The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning

Restoration of Kv7 Channel-Mediated Inhibition Reduces Cued-Reinstatement of Cocaine Seeking

Cocaine addicts display increased sensitivity to drug-associated cues, due in part to changes in the prelimbic prefrontal cortex (PL-PFC). The cellular mechanisms underlying cue-induced reinstatement of cocaine seeking remain unknown. Reinforcement learning for addictive drugs may produce persistent maladaptations in intrinsic excitability within sparse subsets of PFC pyramidal neurons. Using a model of relapse in male rats, we sampled >600 neurons to examine spike frequency adaptation (SFA) and afterhyperpolarizations (AHPs), two systems that attenuate low-frequency inputs to regulate neuronal synchronization. We observed that training to self-administer cocaine or nondrug (sucrose) reinforcers decreased SFA and AHPs in a subpopulation of PL-PFC neurons. Only with cocaine did the resulting hyperexcitability persist through extinction training and increase during reinstatement. In neurons with intact SFA, dopamine enhanced excitability by inhibiting Kv7 potassium channels that mediate SFA. However, dopamine effects were occluded in neurons from cocaine-experienced rats, where SFA and AHPs were reduced. Pharmacological stabilization of Kv7 channels with retigabine restored SFA and Kv7 channel function in neuroadapted cells. When microinjected bilaterally into the PL-PFC 10 min before reinstatement testing, retigabine reduced cue-induced reinstatement of cocaine seeking. Last, using cFos-GFP transgenic rats, we found that the loss of SFA correlated with the expression of cFos-GFP following both extinction and re-exposure to drug-associated cues. Together, these data suggest that cocaine self-administration desensitizes inhibitory Kv7 channels in a subpopulation of PL-PFC neurons. This subpopulation of neurons may represent a persistent neural ensemble responsible for driving drug seeking in response to cues.SIGNIFICANCE STATEMENT Long after the cessation of drug use, cues associated with cocaine still elicit drug-seeking behavior, in part by activation of the prelimbic prefrontal cortex (PL-PFC). The underlying cellular mechanisms governing these activated neurons remain unclear. Using a rat model of relapse to cocaine seeking, we identified a population of PL-PFC neurons that become hyperexcitable following chronic cocaine self-administration. These neurons show persistent loss of spike frequency adaptation, reduced afterhyperpolarizations, decreased sensitivity to dopamine, and reduced Kv7 channel-mediated inhibition. Stabilization of Kv7 channel function with retigabine normalized neuronal excitability, restored Kv7 channel currents, and reduced drug-seeking behavior when administered into the PL-PFC before reinstatement. These data highlight a persistent adaptation in a subset of PL-PFC neurons that may contribute to relapse vulnerability.

Do TRPC channels support working memory? Comparing modulations of TRPC channels and working memory through G-protein coupled receptors and neuromodulators

Working memory is a crucial ability we use in daily life. However, the cellular mechanisms supporting working memory still remain largely unclear. A key component of working memory is persistent neural firing which is believed to serve short-term (hundreds of milliseconds up to tens of seconds) maintenance of necessary information. In this review, we will focus on the role of transient receptor potential canonical (TRPC) channels as a mechanism underlying persistent firing. Many years of in vitro work have been suggesting a crucial role of TRPC channels in working memory and temporal association tasks. If TRPC channels are indeed a central mechanism for working memory, manipulations which impair or facilitate working memory should have a similar effect on TRPC channel modulation. However, modulations of working memory and TRPC channels were never systematically compared, and it remains unanswered whether TRPC channels indeed contribute to working memory in vivo or not. In this article, we review the effects of G-protein coupled receptors (GPCR) and neuromodulators, including acetylcholine, noradrenalin, serotonin and dopamine, on working memory and TRPC channels. Based on comparisons, we argue that GPCR and downstream signaling pathways that activate TRPC, generally support working memory, while those that suppress TRPC channels impair it. However, depending on the channel types, areas, and systems tested, this is not the case in all studies. Further work to clarify involvement of specific TRPC channels in working memory tasks and how they are affected by neuromodulators is still necessary in the future.

Outcome Modulation Across Tasks in the Primate Dorsolateral Prefrontal Cortex

Animals need to learn and to adapt to new and changing environments so that appropriate actions that lead to desirable outcomes are acquired within each context. The prefrontal cortex (PF) is known to underlie such function that directly implies that the outcome of each response must be represented in the brain for behavioral policies update. However, whether such PF signal is context dependent or it is a general representation beyond the specificity of a context is still unclear. Here, we analyzed the activity of neurons in the dorsolateral PF (PFdl) recorded while two monkeys performed two perceptual magnitude discrimination tasks. Both tasks were well known by the monkeys and unexpected changes did not occur but the difficulty of the task varied from trial to trial and thus the monkeys made mistakes in a proportion of trials. We show a context-independent coding of the response outcome with neurons maintaining similar selectivity in both task contexts. Using a classification method of the neural activity, we also show that the trial outcome could be well predicted from the activity of the same neurons in the two contexts. Altogether, our results provide evidence of high degree of outcome generality in PFdl.

Human subthalamic nucleus activity during non-motor decision making

Recent studies have implicated the subthalamic nucleus (STN) in decisions that involve inhibiting movements. Many of the decisions that we make in our daily lives, however, do not involve any motor actions. We studied non-motor decision making by recording intraoperative STN and prefrontal cortex (PFC) electrophysiology as participants perform a novel task that required them to decide whether to encode items into working memory. During all encoding trials, beta band (15-30 Hz) activity decreased in the STN and PFC, and this decrease was progressively enhanced as more items were stored into working memory. Crucially, the STN and lateral PFC beta decrease was significantly attenuated during the trials in which participants were instructed not to encode the presented stimulus. These changes were associated with increase lateral PFC-STN coherence and altered STN neuronal spiking. Our results shed light on why states of altered basal ganglia activity disrupt both motor function and cognition.

Dopamine terminals from the ventral tegmental area gate intrinsic inhibition in the prefrontal cortex

Spike frequency adaptation (SFA or accommodation) and calcium‐activated potassium channels that underlie after‐hyperpolarization potentials (AHP) regulate repetitive firing of neurons. Precisely how neuromodulators such as dopamine from the ventral tegmental area (VTA) regulate SFA and AHP (together referred to as intrinsic inhibition) in the prefrontal cortex (PFC) remains unclear. Using whole cell electrophysiology, we measured intrinsic inhibition in prelimbic (PL) layer 5 pyramidal cells of male adult rats. Results demonstrate that bath application of dopamine reduced intrinsic inhibition (EC₅₀: 25.0 μmol/L). This dopamine action was facilitated by coapplication of cocaine (1 μmol/L), a blocker of dopamine reuptake. To evaluate VTA dopamine terminals in PFC slices, we transfected VTA dopamine cells of TH::Cre rats in vivo with Cre‐dependent AAVs to express channelrhodopsin‐2 (ChR2) or designer receptors exclusively activated by designer drugs (DREADDS). In PFC slices from these animals, stimulation of VTA terminals with either blue light to activate ChR2 or bath application of clozapine‐N‐oxide (CNO) to activate Gq‐DREADDs produced a similar reduction in intrinsic inhibition in PL neurons. Electrophysiological recordings from cells expressing retrograde fluorescent tracers showed that this plasticity occurs in PL neurons projecting to the accumbens core. Collectively, these data highlight an ability of VTA terminals to gate intrinsic inhibition in the PFC, and under appropriate circumstances, enhance PL neuronal firing. These cellular actions of dopamine may be important for dopamine‐dependent behaviors involving cocaine and cue‐reward associations within cortical–striatal circuits.

Neuromodulation of Prefrontal Cortex in Non-Human Primates by Dopaminergic Receptors during Rule-Guided Flexible Behavior and Cognitive Control

The prefrontal cortex (PFC) is indispensable for several higher-order cognitive and executive capacities of primates, including representation of salient stimuli in working memory (WM), maintenance of cognitive task set, inhibition of inappropriate responses and rule-guided flexible behavior. PFC networks are subject to robust neuromodulation from ascending catecholaminergic systems. Disruption of these systems in PFC has been implicated in cognitive deficits associated with several neuropsychiatric disorders. Over the past four decades, a considerable body of work has examined the influence of dopamine on macaque PFC activity representing spatial WM. There has also been burgeoning interest in neuromodulation of PFC circuits involved in other cognitive functions of PFC, including representation of rules to guide flexible behavior. Here, we review recent neuropharmacological investigations conducted in our laboratory and others of the role of PFC dopamine receptors in regulating rule-guided behavior in non-human primates. Employing iontophoresis, we examined the effects of local manipulation of dopaminergic subtypes on neuronal activity during performance of rule-guided pro- and antisaccades, an experimental paradigm sensitive to PFC integrity, wherein deficits in performance are reliably observed in many neuropsychiatric disorders. We found dissociable effects of dopamine receptors on neuronal activity for rule representation and oculomotor responses and discuss these findings in the context of prior studies that have examined the role of dopamine in spatial delayed response tasks, attention, target selection, abstract rules, visuomotor learning and reward. The findings we describe here highlight the common features, as well as heterogeneity and context dependence of dopaminergic neuromodulation in regulating the efficacy of cognitive functions of PFC in health and disease.

The direct basal ganglia pathway is hyperfunctional in focal dystonia

See Fujita and Eidelberg (doi:10.1093/brain/awx305) for a scientific commentary on this article. Focal dystonias are the most common type of isolated dystonia. Although their causative pathophysiology remains unclear, it is thought to involve abnormal functioning of the basal ganglia-thalamo-cortical circuitry. We used high-resolution research tomography with the radioligand 11C-NNC-112 to examine striatal dopamine D1 receptor function in two independent groups of patients, writer’s cramp and laryngeal dystonia, compared to healthy controls. We found that availability of dopamine D1 receptors was significantly increased in bilateral putamen by 19.6–22.5% in writer’s cramp and in right putamen and caudate nucleus by 24.6–26.8% in laryngeal dystonia (all P ≤ 0.009). This suggests hyperactivity of the direct basal ganglia pathway in focal dystonia. Our findings paralleled abnormally decreased dopaminergic function via the indirect basal ganglia pathway and decreased symptom-induced phasic striatal dopamine release in writer’s cramp and laryngeal dystonia. When examining topological distribution of dopamine D1 and D2 receptor abnormalities in these forms of dystonia, we found abnormal separation of direct and indirect pathways within the striatum, with negligible, if any, overlap between the two pathways and with the regions of phasic dopamine release. However, despite topological disorganization of dopaminergic function, alterations of dopamine D1 and D2 receptors were somatotopically localized within the striatal hand and larynx representations in writer’s cramp and laryngeal dystonia, respectively. This finding points to their direct relevance to disorder-characteristic clinical features. Increased D1 receptor availability showed significant negative correlations with dystonia duration but not its severity, likely representing a developmental endophenotype of this disorder. In conclusion, a comprehensive pathophysiological mechanism of abnormal basal ganglia function in focal dystonia is built upon upregulated dopamine D1 receptors that abnormally increase excitation of the direct pathway, downregulated dopamine D2 receptors that abnormally decrease inhibition within the indirect pathway, and weakened nigro-striatal phasic dopamine release during symptomatic task performance. Collectively, these aberrations of striatal dopaminergic function underlie imbalance between direct and indirect basal ganglia pathways and lead to abnormal thalamo-motor-cortical hyperexcitability in dystonia.

Maintained avalanche dynamics during task-induced changes of neuronal activity in nonhuman primates

Sensory events, cognitive processing and motor actions correlate with transient changes in neuronal activity. In cortex, these transients form widespread spatiotemporal patterns with largely unknown statistical regularities. Here, we show that activity associated with behavioral events carry the signature of scale-invariant spatiotemporal clusters, neuronal avalanches. Using high-density microelectrode arrays in nonhuman primates, we recorded extracellular unit activity and the local field potential (LFP) in premotor and prefrontal cortex during motor and cognitive tasks. Unit activity and negative LFP deflections (nLFP) consistently changed in rate at single electrodes during tasks. Accordingly, nLFP clusters on the array deviated from scale-invariance compared to ongoing activity. Scale-invariance was recovered using ‘adaptive binning’, that is identifying clusters at temporal resolution given by task-induced changes in nLFP rate. Measures of LFP synchronization confirmed and computer simulations detailed our findings. We suggest optimization principles identified for avalanches during ongoing activity to apply to cortical information processing during behavior.

Activation of D1/5 Dopamine Receptors in the Dorsal Medial Prefrontal Cortex Promotes Incubated-Like Aversive Responses

It is well established that neurons of the mammalian medial prefrontal cortex (mPFC) modulate different behavioral outputs, including several memory types. This behavioral modulation is, at least in part, under the control of the D1-like Dopamine (DA) receptor (D1/5R) which comprises D1 and D5-specific subtypes (D1R and D5R, respectively). Here, combining a set of behavioral assays with pharmacology, we determined whether the activation of D1/5R in the mPFC during almost neutral or weak negative-valence experiences induces aversive behaviors. The intra mPFC bilateral infusion of the D1/5R agonist SKF 38393 (6.25 μg/side) immediately after exposing rats to the white compartment of a place conditioning apparatus promotes a incubated-like aversive memory when tested 7 days thereafter, but it was not seen 24 h after conditioning. No signs of fear or changes in the anxiety state were observed after the exposure to the white compartment. This aversive response is observed only when the experience paired with the mPFC D1/5R activation has a context component involved. By using specific agonists for D1R or D5R subtypes we suggest that D5R mediate the induction of the aversive behavior. No aversive effects were observed when the D1/5R agonist was infused into the dorsal hippocampus (HP), the nucleus accumbens (NAcc) or the basolateral amygdala (BLA) of rats exposed to the white compartment. Taken together, our present findings endorse the idea that activation of mPFC D1/5R is sufficient to induce incubated-like aversive memories after exposing rats to an apparent neutral or weak negative-valence environment and that mPFC might be considered a key brain region involved in providing adaptive emotional behaviors in response to an ever-changing environment.

Reward and value coding by dopamine neurons in non-human primates

Rewards are fundamental to everyday life. They confer pleasure, support learning, and mediate decisions. Dopamine-releasing neurons in the midbrain are critical for reward processing. These neurons receive input from more than 30 brain areas and send widespread projections to the basal ganglia and frontal cortex. Their phasic responses are tuned to rewards. Specifically, dopamine signals code reward prediction error, the difference between received and predicted rewards. Decades of research in awake, behaving non-human primates (NHP), have shown the importance of these neural signals for learning and decision making. In this review, we will provide an overview of the bedrock findings that support the reward prediction error hypothesis and examine evidence that this signal plays a role in learning and decision making. In addition, we will highlight some of the conceptual challenges in dopamine neurophysiology and identify future areas of research to address these challenges. Keeping with the theme of this special issue, we will focus on the role of NHP studies in understanding dopamine neurophysiology and make the argument that primate models are essential to this line of research.

Cognitive impairment and gene expression alterations in a rodent model of binge eating disorder

Binge eating disorder (BED) is defined as recurrent, distressing over-consumption of palatable food (PF) in a short time period. Clinical studies suggest that individuals with BED may have impairments in cognitive processes, executive functioning, impulse control, and decision-making, which may play a role in sustaining binge eating behavior. These clinical reports, however, are limited and often conflicting. In this study, we used a limited access rat model of binge-like behavior in order to further explore the effects of binge eating on cognition. In binge eating prone (BEP) rats, we found novel object recognition (NOR) as well as Barnes maze reversal learning (BM-RL) deficits. Aberrant gene expression of brain derived neurotrophic factor (Bdnf) and tropomyosin receptor kinase B (TrkB) in the hippocampus (HPC)-prefrontal cortex (PFC) network was observed in BEP rats. Additionally, the NOR deficits were correlated with reductions in the expression of TrkB and insulin receptor (Ir) in the CA3 region of the hippocampus. Furthermore, up-regulation of serotonin-2C (5-HT_2C) receptors in the orbitoprefrontal cortex (OFC) was associated with BM-RL deficit. Finally, in the nucleus accumbens (NAc), we found decreased dopamine receptor 2 (Drd2) expression among BEP rats. Taken together, these data suggest that binge eating vegetable shortening may induce contextual and reversal learning deficits which may be mediated, at least in part, by the altered expression of genes in the CA3-OFC-NAc neural network.

Circuit Mechanisms of Sensorimotor Learning

The relationship between the brain and the environment is flexible, forming the foundation for our ability to learn. Here we review the current state of our understanding of the modifications in the sensorimotor pathway related to sensorimotor learning. We divide the process into three hierarchical levels with distinct goals: (1) sensory perceptual learning, (2) sensorimotor associative learning, and (3) motor skill learning. Perceptual learning optimizes the representations of important sensory stimuli. Associative learning and the initial phase of motor skill learning are ensured by feedback-based mechanisms that permit trial-and-error learning. The later phase of motor skill learning may primarily involve feedback-independent mechanisms operating under the classic Hebbian rule. With these changes under distinct constraints and mechanisms, sensorimotor learning establishes dedicated circuitry for the reproduction of stereotyped neural activity patterns and behavior.

Tonic or Phasic Stimulation of Dopaminergic Projections to Prefrontal Cortex Causes Mice to Maintain or Deviate from Previously Learned Behavioral Strategies

Dopamine neurons in the ventral tegmental area (VTA) encode reward prediction errors and can drive reinforcement learning through their projections to striatum, but much less is known about their projections to prefrontal cortex (PFC). Here, we studied these projections and observed phasic VTA-PFC fiber photometry signals after the delivery of rewards. Next, we studied how optogenetic stimulation of these projections affects behavior using conditioned place preference and a task in which mice learn associations between cues and food rewards and then use those associations to make choices. Neither phasic nor tonic stimulation of dopaminergic VTA-PFC projections elicited place preference. Furthermore, substituting phasic VTA-PFC stimulation for food rewards was not sufficient to reinforce new cue-reward associations nor maintain previously learned ones. However, the same patterns of stimulation that failed to reinforce place preference or cue-reward associations were able to modify behavior in other ways. First, continuous tonic stimulation maintained previously learned cue-reward associations even after they ceased being valid. Second, delivering phasic stimulation either continuously or after choices not previously associated with reward induced mice to make choices that deviated from previously learned associations. In summary, despite the fact that dopaminergic VTA-PFC projections exhibit phasic increases in activity that are time locked to the delivery of rewards, phasic activation of these projections does not necessarily reinforce specific actions. Rather, dopaminergic VTA-PFC activity can control whether mice maintain or deviate from previously learned cue-reward associations.SIGNIFICANCE STATEMENT Dopaminergic inputs from ventral tegmental area (VTA) to striatum encode reward prediction errors and reinforce specific actions; however, it is currently unknown whether dopaminergic inputs to prefrontal cortex (PFC) play similar or distinct roles. Here, we used bulk Ca²⁺ imaging to show that unexpected rewards or reward-predicting cues elicit phasic increases in the activity of dopaminergic VTA-PFC fibers. However, in multiple behavioral paradigms, we failed to observe reinforcing effects after stimulation of these fibers. In these same experiments, we did find that tonic or phasic patterns of stimulation caused mice to maintain or deviate from previously learned cue-reward associations, respectively. Therefore, although they may exhibit similar patterns of activity, dopaminergic inputs to striatum and PFC can elicit divergent behavioral effects.

Projection targets of medial frontal D1DR-expressing neurons

Prefrontal neurons expressing D1-type dopamine receptors (D1DRs) have been implicated in a variety of cognitive processes including working memory and timing. Although D1DRs are most strongly expressed on layer V/VI projection neurons, it is unknown which brain areas are specifically targeted by these projections. Here we selectively marked D1DR neurons using cre-loxP techniques with AAV carrying mCherry fluorescent protein, and traced projection targets of D1DR+ neurons in the mouse medial frontal cortex (MFC). We found relatively strong MFC D1DR+ projections to cortical areas as well as projections to basal ganglia and thalamic nuclei. We found relatively weaker MFC D1DR+ projections to the brainstem, hypothalamus, and other subcortical nuclei. These data intimate that MFC D1DR+ projections are well-positioned to powerfully influence cortical processing and have subcortical specificity. Thus MFC D1DR+ projection neurons may play a key role in tuning cortical networks during goal-directed behavior.

Emergence of visually-evoked reward expectation signals in dopamine neurons via the superior colliculus in V1 lesioned monkeys

Responses of midbrain dopamine (DA) neurons reflecting expected reward from sensory cues are critical for reward-based associative learning. However, critical pathways by which reward-related visual information is relayed to DA neurons remain unclear. To address this question, we investigated Pavlovian conditioning in macaque monkeys with unilateral primary visual cortex (V1) lesions (an animal model of ‘blindsight’). Anticipatory licking responses to obtain juice drops were elicited in response to visual conditioned stimuli (CS) in the affected visual field. Subsequent pharmacological inactivation of the superior colliculus (SC) suppressed the anticipatory licking. Concurrent single unit recordings indicated that DA responses reflecting the reward expectation could be recorded in the absence of V1, and that these responses were also suppressed by SC inactivation. These results indicate that the subcortical visual circuit can relay reward-predicting visual information to DA neurons and integrity of the SC is necessary for visually-elicited classically conditioned responses after V1 lesion.

DOI:http://dx.doi.org/10.7554/eLife.24459.001

To survive and thrive, animals must learn to approach cues in their environment that are likely to lead to a desirable outcome and avoid those that might lead them to harm. A group of brain regions known as the midbrain dopamine system helps many animals to achieve this. Dopamine is the brain’s reward signal. Cues that predict rewards, such as the sight or smell of food, activate midbrain dopamine neurons. However, the details of this process remained unclear.

Takakuwa et al. have now examined how visual information that signals reward reaches the midbrain dopamine neurons. The anatomy of the visual system suggests two main possibilities. Information may travel directly from the eyes to an area of the midbrain called the superior colliculus, and then onto the dopamine neurons. Alternatively, information may travel to the midbrain indirectly via a pathway that includes additional processing in the brain’s outer layer, the visual cortex.

To distinguish between these routes, Takakuwa et al. studied monkeys in which the indirect pathway via the visual cortex had been damaged. Some people with damage to this pathway have a disorder called blindsight. They are able to detect the movement or location of stimuli, but they cannot consciously see those stimuli. The monkeys with damage to visual cortex were able to learn that an image on a screen predicted the delivery of fruit juice. After repeated trials, the monkeys began to lick the spout dispensing the juice whenever the image appeared, even if no juice was delivered. The monkeys’ midbrain dopamine neurons also sent more signals in response to the images, and showed greater activity when the images predicted large rewards than small ones. Takakuwa et al. next inactivated the superior colliculus with a drug and showed that this prevented both the licking behavior and the increased signaling.

Together the findings show that visual information about potential rewards can reach midbrain dopamine neurons via a direct route through the superior colliculus, without needing to pass via the visual cortex. The next step is to determine how and when the visual cortex may get involved in this process to help animals maximize rewards.

DOI:http://dx.doi.org/10.7554/eLife.24459.002

Role of Anterior Cingulate Cortex in Instrumental Learning: Blockade of Dopamine D1 Receptors Suppresses Overt but Not Covert Learning

HIGHLIGHTS

Blockade of dopamine D1 receptors in ACC suppressed instrumental learning when overt responding was required.

Covert learning through observation was not impaired.

After treatment with a dopamine antagonist, instrumental learning recovered but not the rat's pretreatment level of effort tolerance.

ACC dopamine is not necessary for acquisition of task-relevant cues during learning, but regulates energy expenditure and effort based decision.

Dopamine activity in anterior cingulate cortex (ACC) is essential for various aspects of instrumental behavior, including learning and effort based decision making. To dissociate learning from physical effort, we studied both observational (covert) learning, and trial-and-error (overt) learning. If ACC dopamine activity is required for task acquisition, its blockade should impair both overt and covert learning. If dopamine is not required for task acquisition, but solely for regulating the willingness to expend effort for reward, i.e., effort tolerance, blockade should impair overt learning but spare covert learning. Rats learned to push a lever for food rewards either with or without prior observation of an expert conspecific performing the same task. Before daily testing sessions, the rats received bilateral ACC microinfusions of SCH23390, a dopamine D1 receptor antagonist, or saline-control infusions. We found that dopamine blockade suppressed overt responding selectively, leaving covert task acquisition through observational learning intact. In subsequent testing sessions without dopamine blockade, rats recovered their overt-learning capacity but not their pre-treatment level of effort tolerance. These results suggest that ACC dopamine is not required for the acquisition of conditioned behaviors and that apparent learning impairments could instead reflect a reduced level of willingness to expend effort due to cortical dopamine blockade.

Coincident postsynaptic activity gates presynaptic dopamine release to induce plasticity in Drosophila mushroom bodies

Simultaneous stimulation of the antennal lobes (ALs) and the ascending fibers of the ventral nerve cord (AFV), two sensory inputs to the mushroom bodies (MBs), induces long-term enhancement (LTE) of subsequent AL-evoked MB responses. LTE induction requires activation of at least three signaling pathways to the MBs, mediated by nicotinic acetylcholine receptors (nAChRs), NMDA receptors (NRs), and D1 dopamine receptors (D1Rs). Here, we demonstrate that inputs from the AL are transmitted to the MBs through nAChRs, and inputs from the AFV are transmitted by NRs. Dopamine signaling occurs downstream of both nAChR and NR activation, and requires simultaneous stimulation of both pathways. Dopamine release requires the activity of the rutabaga adenylyl cyclase in postsynaptic MB neurons, and release is restricted to MB neurons that receive coincident stimulation. Our results indicate that postsynaptic activity can gate presynaptic dopamine release to regulate plasticity.

Cell-type-specific modulation of targets and distractors by dopamine D1 receptors in primate prefrontal cortex

The prefrontal cortex (PFC) is crucial for maintaining relevant information in working memory and resisting interference. PFC neurons are strongly regulated by dopamine, but it is unknown whether dopamine receptors are involved in protecting target memories from distracting stimuli. We investigated the prefrontal circuit dynamics and dopaminergic modulation of targets and distractors in monkeys trained to ignore interfering stimuli in a delayed-match-to-numerosity task. We found that dopamine D1 receptors (D1Rs) modulate the recovery of task-relevant information following a distracting stimulus. The direction of modulation is cell-type-specific: in putative pyramidal neurons, D1R inhibition enhances and D1R stimulation attenuates coding of the target stimulus after the interference, while the opposite pattern is observed in putative interneurons. Our results suggest that dopaminergic neuromodulation of PFC circuits regulates mental representations of behaviourally relevant stimuli that compete with task-irrelevant input and could play a central role for cognitive functioning in health and disease.

Physiological approaches to understanding molecular actions on dorsolateral prefrontal cortical neurons underlying higher cognitive processing

Revealing how molecular mechanisms influence higher brain circuits in primates will be essential for understanding how genetic insults lead to increased risk of cognitive disorders. Traditionally, modulatory influences on higher cortical circuits have been examined using lesion techniques, where a brain region is depleted of a particular transmitter to determine how its loss impacts cognitive function. For example, depletion of catecholamines or acetylcholine from the dorsolateral prefrontal cortex produces striking deficits in working memory abilities. More directed techniques have utilized direct infusions of drug into a specific cortical site to try to circumvent compensatory changes that are common following transmitter depletion. The effects of drug on neuronal firing patterns are often studied using iontophoresis, where a minute amount of drug is moved into the brain using a tiny electrical current, thus minimizing the fluid flow that generally disrupts neuronal recordings. All of these approaches can be compared to systemic drug administration, which remains a key arena for the development of effective therapeutics for human cognitive disorders. Most recently, viral techniques are being developed to be able to manipulate proteins for which there is no developed pharmacology, and to allow optogenetic manipulations in primate cortex. As the association cortices greatly expand in brain evolution, research in nonhuman primates is particularly important for understanding the modulatory regulation of our highest order cognitive operations.

Targeting Prefrontal Cortical Systems for Drug Development: Potential Therapies for Cognitive Disorders

Medications to treat cognitive disorders are increasingly needed, yet researchers have had few successes in this challenging arena. Cognitive abilities in primates arise from highly evolved N-methyl-d-aspartate (NMDA) receptor circuits in layer III of the dorsolateral prefrontal cortex. These circuits have unique modulatory needs that can differ from the layer V neurons that predominate in rodents, but they offer multiple therapeutic targets. Cognitive improvement often requires low doses that enhance the pattern of information held in working memory, whereas higher doses can produce nonspecific changes that obscure information. Identifying appropriate doses for clinical trials may be helped by assessments in monkeys and by flexible, individualized dose designs. The use of guanfacine (Intuniv) for prefrontal cortical disorders was based on research in monkeys, supporting this approach. Coupling our knowledge of higher primate circuits with the powerful methods now available in drug design will help create effective treatments for cognitive disorders.

Dopamine D1 and D2 Receptors Make Dissociable Contributions to Dorsolateral Prefrontal Cortical Regulation of Rule-Guided Oculomotor Behavior

Studies of neuromodulation of spatial short-term memory have shown that dopamine D1 receptor (D1R) stimulation in dorsolateral prefrontal cortex (DLPFC) dose-dependently modulates memory activity, whereas D2 receptors (D2Rs) selectively modulate activity related to eye movements hypothesized to encode movement feedback. We examined localized stimulation of D1Rs and D2Rs on DLPFC neurons engaged in a task involving rule representation in memory to guide appropriate eye movements toward or away from a visual stimulus. We found dissociable effects of D1R and D2R on DLPFC physiology. D1R stimulation degrades memory activity for the task rule and increases stimulus-related selectivity. In contrast, D2R stimulation affects motor activity tuning only when eye movements are made to the stimulus. Only D1R stimulation degrades task performance and increases impulsive responding. Our results suggest that D1Rs regulate rule representation and impulse control, whereas D2Rs selectively modulate eye-movement-related dynamics and not rule representation in the DLPFC.

Phasic dopamine release in the medial prefrontal cortex enhances stimulus discrimination

Phasic dopamine (DA) release is believed to guide associative learning. Most studies have focused on projections from the ventral tegmental area (VTA) to the striatum, and the action of DA in other VTA target regions remains unclear. Using optogenetic activation of VTA projections, we examined DA function in the medial prefrontal cortex (mPFC). We found that mice perceived optogenetically induced DA release in mPFC as neither rewarding nor aversive, and did not change their previously learned behavior in response to DA transients. However, repetitive temporal pairing of an auditory conditioned stimulus (CS) with mPFC DA release resulted in faster learning of a subsequent task involving discrimination of the same CS against unpaired stimuli. Similar results were obtained using both appetitive and aversive unconditioned stimuli, supporting the notion that DA transients in mPFC do not represent valence. Using extracellular recordings, we found that CS-DA pairings increased firing of mPFC neurons in response to CSs, and administration of D1 or D2 DA-receptor antagonists in mPFC during learning impaired stimulus discrimination. We conclude that DA transients tune mPFC neurons for the recognition of behaviorally relevant events during learning.

Influence of SKF38393 on changes of gene profile in rat prefrontal cortex during chronic paradoxical sleep deprivation

Chronic paradoxical sleep deprivation (CSD) can induce dramatic physiological and neurofunctional changes in rats, including decreased body weight, reduced learning and memory, and declined locomotor function. SKF38393, a dopamine D1 receptor agonist, can reverse the above damages. However, the mechanism of CSD syndrome and reversal role of SKF38393 remains largely unexplained. To preliminarily elucidate the mechanism of the neural dysfunction caused by CSD, in the present study we use gene chips to examine the expression profile of more than 28,000 transcripts in the prefrontal cortex (PFC). Rats were sleep deprived by modified multi-platform method for 3 weeks. Totally 59 transcripts showed differential expressions in CSD group in contrast to controls; they included transcripts coding for caffeine metabolism, circadian rhythm, drug metabolism and some amino acid metabolism pathway. Among the 59 transcripts, 39 increased their expression and 20 decreased. Two transcripts can be specifically reversed with SKF38393, one of them is Homer1, which is related to 20 functional classifications and coding for Glutamatergic synapse pathway. Our findings in the present study indicate that long-term sleep deprivation may trigger the changes of some certain functions and pathways in the PFC, and lead to the dysfunction of this advanced neuron, and the activation of D1 receptor by SKF38393 might ameliorate these changes via modulation of some transcripts such as Homer1, which is involved in the Ca(2+) pathway and MAPK pathway related to Glutamatergic synapse pathway.

Dopamine neurons learn relative chosen value from probabilistic rewards

Economic theories posit reward probability as one of the factors defining reward value. Individuals learn the value of cues that predict probabilistic rewards from experienced reward frequencies. Building on the notion that responses of dopamine neurons increase with reward probability and expected value, we asked how dopamine neurons in monkeys acquire this value signal that may represent an economic decision variable. We found in a Pavlovian learning task that reward probability-dependent value signals arose from experienced reward frequencies. We then assessed neuronal response acquisition during choices among probabilistic rewards. Here, dopamine responses became sensitive to the value of both chosen and unchosen options. Both experiments showed also the novelty responses of dopamine neurones that decreased as learning advanced. These results show that dopamine neurons acquire predictive value signals from the frequency of experienced rewards. This flexible and fast signal reflects a specific decision variable and could update neuronal decision mechanisms.

Dopaminergic impact on local and global cortical circuit processing during learning

We have learned to detect, predict and behaviorally respond to important changes in our environment on short and longer time scales. Therefore, brains of humans and higher animals build upon a perceptual and semantic salience stored in their memories mainly generated by associative reinforcement learning. Functionally, the brain needs to extract and amplify a small number of features of sensory input with behavioral relevance to a particular situation in order to guide behavior. In this review, I argue that dopamine action, particularly in sensory cortex, orchestrates layer-dependent local and long-range cortical circuits integrating sensory associated bottom-up and semantically relevant top-down information, respectively. Available evidence reveals that dopamine thereby controls both the selection of perceptually or semantically salient signals as well as feedback processing from higher-order areas in the brain. Sensory cortical dopamine thereby governs the integration of selected sensory information within a behavioral context. This review proposes that dopamine enfolds this function by temporally distinct actions on particular layer-dependent local and global cortical circuits underlying the integration of sensory, and non-sensory cognitive and behavioral variables.

Associative learning rapidly establishes neuronal representations of upcoming behavioral choices in crows

The ability to form associations between behaviorally relevant sensory stimuli is fundamental for goal-directed behaviors. We investigated neuronal activity in the telencephalic area nidopallium caudolaterale (NCL) while two crows (Corvus corone) performed a delayed association task. Whereas some paired associates were familiar to the crows, novel associations had to be learned and mapped to the same target stimuli within a single session. We found neurons that prospectively encoded the chosen test item during the delay for both familiar and newly learned associations. These neurons increased their selectivity during learning in parallel with the crows' increased behavioral performance. Thus, sustained activity in the NCL actively processes information for the upcoming behavioral choice. These data provide new insights into memory representations of behaviorally meaningful stimuli in birds, and how such representations are formed during learning. The findings suggest that the NCL plays a role in learning arbitrary associations, a cornerstone of corvids' remarkable behavioral flexibility and adaptability.

The impact of Val108/158Met polymorphism of catechol-O-methyltransferase on brain oscillations during working memory

This study investigated whether catechol-O-methyltransferase (COMT) Val/Met polymorphism was associated with variation in event-related desynchronization/synchronization (ERD/ERS) of responses during working memory (WM). 11 Val/Val and 11 Met/Met homozygous participants underwent magnetoencephalography (MEG) while performing a WM task. In contrast to small effects behaviourally, during the delay period Val/Val individuals showed lower ERS in the gamma band (Hz 30-50) in frontal regions, increased ERS in the alpha band (Hz 8-12) in the right frontal and parietal regions and increased ERD in the beta band (Hz 14-30) in the left fronto-temporal regions as compared with Met/Met homozygous individuals. During the response period Val/Val participants showed greater beta ERD in the prefrontal and parietotemporal regions. These results demonstrate that COMT genotype has a strong impact on brain responses (oscillatory activity) during WM performance likely a consequence of compensatory activity during the delay and response periods.

Components and characteristics of the dopamine reward utility signal

Rewards are defined by their behavioral functions in learning (positive reinforcement), approach behavior, economic choices, and emotions. Dopamine neurons respond to rewards with two components, similar to higher order sensory and cognitive neurons. The initial, rapid, unselective dopamine detection component reports all salient environmental events irrespective of their reward association. It is highly sensitive to factors related to reward and thus detects a maximal number of potential rewards. It also senses aversive stimuli but reports their physical impact rather than their aversiveness. The second response component processes reward value accurately and starts early enough to prevent confusion with unrewarded stimuli and objects. It codes reward value as a numeric, quantitative utility prediction error, consistent with formal concepts of economic decision theory. Thus, the dopamine reward signal is fast, highly sensitive and appropriate for driving and updating economic decisions.

The DA antagonist tiapride impairs context-related extinction learning in a novel context without affecting renewal

Renewal describes the recovery of an extinguished response if recall is tested in a context different from the extinction context. Behavioral studies demonstrated that attention to relevant context strengthens renewal. Neurotransmitters mediating attention and learning such as the dopaminergic (DA) system presumably modulate extinction learning and renewal. However, the role of DA for non-fear-based extinction learning and renewal in humans has not yet been investigated. This fMRI study investigated effects of DA-antagonism upon context-related extinction in a predictive learning task in which extinction occurred either in a novel (ABA) or an unchanged (AAA) context. The tiapride-treated group (TIA) showed significantly impaired ABA extinction learning and a significant within-group difference between ABA and AAA extinction, compared to placebo (PLAC). Groups did not differ in their level of ABA renewal. In ABA extinction, TIA showed reduced activation in dlPFC and OFC, hippocampus, and temporal regions. Across groups, activation in PFC and hippocampus correlated negatively with ABA extinction errors. Results suggest that in context-related extinction learning DA in PFC and hippocampus is involved in readjusting the cue-outcome relationship in the presence of a novel context. However, relating context to the appropriate association during recall does not appear to rely exclusively on DA signaling.

Doping the Mind: Dopaminergic Modulation of Prefrontal Cortical Cognition

The prefrontal cortex is the center of cognitive control. Processing in prefrontal cortical circuits enables us to direct attention to behaviorally relevant events; to memorize, structure, and categorize information; and to learn new concepts. The prefrontal cortex receives strong projections from midbrain neurons that use dopamine as a transmitter. In this article, we review the crucial role dopamine plays as a modulator of prefrontal cognitive functions, in the primate brain in particular. Following a summary of the anatomy and physiology of the midbrain dopamine system, we focus on recent studies that investigated dopaminergic effects in prefrontal cortex at the cellular level. We then discuss how unregulated prefrontal dopamine signaling could contribute to major disorders of cognition. The studies highlighted in this review demonstrate the powerful influence dopamine exerts on the mind.

Dopamine D1/D5 receptor signaling regulates synaptic cooperation and competition in hippocampal CA1 pyramidal neurons via sustained ERK1/2 activation

Synaptic cooperation and competition are important components of synaptic plasticity that tune synapses for the formation of associative long‐term plasticity, a cellular correlate of associative long‐term memory. We have recently reported that coincidental activation of weak synapses within the vicinity of potentiated synapses will alter the cooperative state of synapses to a competitive state thus leading to the slow decay of long‐term plasticity, but the molecular mechanism underlying this is still unknown. Here, using acute hippocampal slices of rats, we have examined how increasing extracellular dopamine concentrations interact and/or affect electrically induced long‐term potentiation (LTP) in the neighboring synapses. We demonstrate that D1/D5‐receptor‐mediated potentiation at the CA1 Schaffer collateral synapses differentially regulates synaptic co‐operation and competition. Further investigating the molecular players involved, we reveal an important role for extracellular signal‐regulated kinases‐1 and 2 (ERK1/2) as signal integrators and dose‐sensors. Interestingly, a sustained activation of ERK1/2 pathway seems to be involved in the differential regulation of synaptic associativity. The concentration‐dependent effects of the modulatory transmitter, as demonstrated for dopaminergic signaling in the present study, might offer additional computational power by fine tuning synaptic associativity processes for establishing long‐term associative memory in neural networks. © 2015 The Authors Hippocampus Published by Wiley Periodicals, Inc.

Cholinergic modulation of dopamine pathways through nicotinic acetylcholine receptors

Nicotine addiction is highly prevalent in current society and is often comorbid with other diseases. In the central nervous system, nicotine acts as an agonist for nicotinic acetylcholine receptors (nAChRs) and its effects depend on location and receptor composition. Although nicotinic receptors are found in most brain regions, many studies on addiction have focused on the mesolimbic system and its reported behavioral correlates such as reward processing and reinforcement learning. Profound modulatory cholinergic input from the pedunculopontine and laterodorsal tegmentum to dopaminergic midbrain nuclei as well as local cholinergic interneuron projections to dopamine neuron axons in the striatum may play a major role in the effects of nicotine. Moreover, an indirect mesocorticolimbic feedback loop involving the medial prefrontal cortex may be involved in behavioral characteristics of nicotine addiction. Therefore, this review will highlight current understanding of the effects of nicotine on the function of mesolimbic and mesocortical dopamine projections in the mesocorticolimbic circuit.

Serotonin Modulation of Prefronto-Hippocampal Rhythms in Health and Disease

There is mounting evidence that most cognitive functions depend upon the coordinated activity of neuronal networks often located far from each other in the brain. Ensembles of neurons synchronize their activity, generating oscillations at different frequencies that may encode behavior by allowing an efficient communication between brain areas. The serotonin system, by virtue of the widespread arborisation of serotonergic neurons, is in an excellent position to exert strong modulatory actions on brain rhythms. These include specific oscillatory activities in the prefrontal cortex and the hippocampus, two brain areas essential for many higher-order cognitive functions. Psychiatric patients show abnormal oscillatory activities in these areas, notably patients with schizophrenia who display psychotic symptoms as well as affective and cognitive impairments. Synchronization of neural activity between the prefrontal cortex and the hippocampus seems to be important for cognition and, in fact, reduced prefronto-hippocampal synchrony has been observed in a genetic mouse model of schizophrenia. Here, we review recent advances in the field of neuromodulation of brain rhythms by serotonin, focusing on the actions of serotonin in the prefrontal cortex and the hippocampus. Considering that the serotonergic system plays a crucial role in cognition and mood and is a target of many psychiatric treatments, it is surprising that this field of research is still in its infancy. In that regard, we point to future investigations that are much needed in this field.

Activation of Phosphatidylinositol-Linked Dopamine Receptors Induces a Facilitation of Glutamate-Mediated Synaptic Transmission in the Lateral Entorhinal Cortex

The lateral entorhinal cortex receives strong inputs from midbrain dopamine neurons that can modulate its sensory and mnemonic function. We have previously demonstrated that 1 µM dopamine facilitates synaptic transmission in layer II entorhinal cortex cells via activation of D1-like receptors, increased cAMP-PKA activity, and a resulting enhancement of AMPA-receptor mediated currents. The present study assessed the contribution of phosphatidylinositol (PI)-linked D1 receptors to the dopaminergic facilitation of transmission in layer II of the rat entorhinal cortex, and the involvement of phospholipase C activity and release of calcium from internal stores. Whole-cell patch-clamp recordings of glutamate-mediated evoked excitatory postsynaptic currents were obtained from pyramidal and fan cells. Activation of D1-like receptors using SKF38393, SKF83959, or 1 µM dopamine induced a reversible facilitation of EPSCs which was abolished by loading cells with either the phospholipase C inhibitor U-73122 or the Ca2+ chelator BAPTA. Neither the L-type voltage-gated Ca2+ channel blocker nifedipine, nor the L/N-type channel blocker cilnidipine, blocked the facilitation of synaptic currents. However, the facilitation was blocked by blocking Ca2+ release from internal stores via inositol 1,4,5-trisphosphate (InsP3) receptors or ryanodine receptors. Follow-up studies demonstrated that inhibiting CaMKII activity with KN-93 failed to block the facilitation, but that application of the protein kinase C inhibitor PKC(19-36) completely blocked the dopamine-induced facilitation. Overall, in addition to our previous report indicating a role for the cAMP-PKA pathway in dopamine-induced facilitation of synaptic transmission, we demonstrate here that the dopaminergic facilitation of synaptic responses in layer II entorhinal neurons also relies on a signaling cascade dependent on PI-linked D1 receptors, PLC, release of Ca2+ from internal stores, and PKC activation which is likely dependent upon both DAG and enhanced intracellular Ca2+. These signaling pathways may collaborate to enhance sensory and mnemonic function in the entorhinal cortex during tonic release of dopamine.

Dopamine's Actions in Primate Prefrontal Cortex: Challenges for Treating Cognitive Disorders

The prefrontal cortex (PFC) elaborates and differentiates in primates, and there is a corresponding elaboration in cortical dopamine (DA). DA cells that fire to both aversive and rewarding stimuli likely project to the dorsolateral PFC (dlPFC), signaling a salient event. Since 1979, we have known that DA has an essential influence on dlPFC working memory functions. DA has differing effects via D1 (D1R) versus D2 receptor (D2R) families. D1R are concentrated on dendritic spines, and D1/5R stimulation produces an inverted U-shaped dose response on visuospatial working memory performance and Delay cell firing, the neurons that generate representations of visual space. Optimal levels of D1R stimulation gate out "noise," whereas higher levels, e.g., during stress, suppress Delay cell firing. These effects likely involve hyperpolarization-activated cyclic nucleotide-gated channel opening, activation of GABA interneurons, and reduced glutamate release. Dysregulation of D1R has been related to cognitive deficits in schizophrenia, and there is a need for new, lower-affinity D1R agonists that may better mimic endogenous DA to enhance mental representations and improve cognition. In contrast to D1R, D2R are primarily localized on layer V pyramidal cell dendrites, and D2/3R stimulation speeds and magnifies the firing of Response cells, including Response Feedback cells. Altered firing of Feedback neurons may relate to positive symptoms in schizophrenia. Emerging research suggests that DA may have similar effects in the ventrolateral PFC and frontal eye fields. Research on the orbital PFC in monkeys is just beginning and could be a key area for future discoveries.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Modulating the map: dopaminergic tuning of hippocampal spatial coding and interactions

Salient events activate the midbrain dopaminergic system and have important impacts on various aspects of mnemonic function, including the stability of hippocampus-dependent memories. Dopamine is also central to modulation of neocortical memory processing, particularly during prefrontal cortex-dependent working memory. Here, we review the current state of the circuitry and physiology underlying dopamine's actions, suggesting that--alongside local effects within hippocampus and prefrontal cortex--dopamine released from the midbrain ventral tegmental area is well positioned to dynamically tune interactions between limbic-cortical circuits through modulation of rhythmic network activity.

Dopamine D2-receptor blockade enhances decoding of prefrontal signals in humans

The prefrontal cortex houses representations critical for ongoing and future behavior expressed in the form of patterns of neural activity. Dopamine has long been suggested to play a key role in the integrity of such representations, with D2-receptor activation rendering them flexible but weak. However, it is currently unknown whether and how D2-receptor activation affects prefrontal representations in humans. In the current study, we use dopamine receptor-specific pharmacology and multivoxel pattern-based functional magnetic resonance imaging to test the hypothesis that blocking D2-receptor activation enhances prefrontal representations. Human subjects performed a simple reward prediction task after double-blind and placebo controlled administration of the D2-receptor antagonist amisulpride. Using a whole-brain searchlight decoding approach we show that D2-receptor blockade enhances decoding of reward signals in the medial orbitofrontal cortex. Examination of activity patterns suggests that amisulpride increases the separation of activity patterns related to reward versus no reward. Moreover, consistent with the cortical distribution of D2 receptors, post hoc analyses showed enhanced decoding of motor signals in motor cortex, but not of visual signals in visual cortex. These results suggest that D2-receptor blockade enhances content-specific representations in frontal cortex, presumably by a dopamine-mediated increase in pattern separation. These findings are in line with a dual-state model of prefrontal dopamine, and provide new insights into the potential mechanism of action of dopaminergic drugs.

Learning by observation in the macaque monkey under high experimental constraints

While neuroscience research has tremendously advanced our knowledge about the neural mechanisms of individual learning, i.e. through trial-and-error, it is only recently that neuroscientists have begun to study observational learning, and thus little is known about its neural mechanisms. One limitation is that observational learning has been addressed under unconstrained experimental conditions, not compatible with neuronal recordings. This study examined observational learning in macaque monkeys under the constraining conditions of behavioral neurophysiology. Two animals sat in primate chairs facing each other, with their head fixed. A touch screen was placed face up between the chairs at arm's reach, and the monkeys were trained on an abstract visuomotor associative task. In one experiment, the monkeys alternated the roles of "actor" and "observer". The actor learned to associate visual cues with reaching targets, while the observer "watched" freely. Then, the observer was given the same cue-target associations just performed by the actor, or had to learn new, not previously observed ones. The results show that learning performance is better after observation. In experiment 2, one monkey learned from a human actor who performed the task with errors only, or with successes only in separate blocks. The monkey's gain in performance was higher after observation of errors than after successes. The findings suggest that observational learning can occur even under highly constraining conditions, and open the way for investigating the neuronal correlates of social learning using the methods of behavioral neurophysiology.

Effects of chronic REM sleep restriction on D1 receptor and related signal pathways in rat prefrontal cortex

The prefrontal cortex (PFC) mediates cognitive function that is sensitive to disruption by sleep loss, and molecular mechanisms regulating neural dysfunction induced by chronic sleep restriction (CSR), particularly in the PFC, have yet to be completely understood. The aim of the present study was to investigate the effect of chronic REM sleep restriction (REM-CSR) on the D₁ receptor (D₁R) and key molecules in D₁R' signal pathways in PFC. We employed the modified multiple platform method to create the REM-CSR rat model. The ultrastructure of PFC was observed by electron microscopy. HPLC was performed to measure the DA level in PFC. The expressions of genes and proteins of related molecules were assayed by real-time PCR and Western blot, respectively. The general state and morphology of PFC in rats were changed by CSR, and DA level and the expression of D₁R in PFC were markedly decreased (P < 0.01, P < 0.05); the expression of phosphor-PKAcα was significantly lowered in CSR rats (P < 0.05). The present results suggested that the alteration of neuropathology and D₁R expression in PFC may be associated with CSR induced cognitive dysfunction, and the PKA pathway of D₁R may play an important role in the impairment of advanced neural function.

Dopamine signals and physiological origin of cognitive dysfunction in Parkinson's disease

The pathological hallmark of Parkinson's disease (PD) is the degeneration of midbrain dopamine neurons. Cognitive dysfunction is a feature of PD patients even at the early stages of the disease. Electrophysiological studies on dopamine neurons in awake animals provide contradictory accounts of the role of dopamine. These studies have established that dopamine neurons convey a unique signal associated with rewards rather than cognitive functions. Emphasizing their role in reward processing leads to difficulty in developing hypothesis as to how cognitive impairments in PD are associated with the degeneration of dopamine circuitry. A hint to resolve this contradiction came from recent electrophysiological studies reporting that dopamine neurons transmit more diverse signals than previously thought. These studies suggest that dopamine neurons are divided into at least two functional subgroups, one signaling "motivational value" and the other signaling "salience." The former subgroup fits well with the conventional reward theory, whereas the latter subgroup has been shown to transmit signals related to salient but non-rewarding experiences such as aversive stimulations and cognitively demanding situations. This article reviews recent advances in understanding the non-reward functions of dopamine, and then discusses the possibility that cognitive dysfunction in PD is at least partially caused by the degeneration of the dopamine neuron subgroup signaling the salience of events in the environment.

Effect of dietary iron loading on recognition memory in growing rats

While nutritional and neurobehavioral problems are associated with both iron deficiency during growth and overload in the elderly, the effect of iron loading in growing ages on neurobehavioral performance has not been fully explored. To characterize the role of dietary iron loading in memory function in the young, weanling rats were fed iron-loading diet (10,000 mg iron/kg diet) or iron-adequate control diet (50 mg/kg) for one month, during which a battery of behavioral tests were conducted. Iron-loaded rats displayed elevated non-heme iron levels in serum and liver, indicating a condition of systemic iron overload. In the brain, non-heme iron was elevated in the prefrontal cortex of iron-loaded rats compared with controls, whereas there was no difference in iron content in other brain regions between the two diet groups. While iron loading did not alter motor coordination or anxiety-like behavior, iron-loaded rats exhibited a better recognition memory, as represented by an increased novel object recognition index (22% increase from the reference value) than control rats (12% increase; P=0.047). Western blot analysis showed an up-regulation of dopamine receptor 1 in the prefrontal cortex from iron-loaded rats (142% increase; P=0.002). Furthermore, levels of glutamate receptors (both NMDA and AMPA) and nicotinic acetylcholine receptor (nAChR) were significantly elevated in the prefrontal cortex of iron-loaded rats (62% increase in NR1; 70% increase in Glu1A; 115% increase in nAChR). Dietary iron loading also increased the expression of NMDA receptors and nAChR in the hippocampus. These results support the idea that iron is essential for learning and memory and further reveal that iron supplementation during developmental and rapidly growing periods of life improves memory performance. Our investigation also demonstrates that both cholinergic and glutamatergic neurotransmission pathways are regulated by dietary iron and provides a molecular basis for the role of iron loading in improved memory.

Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control

Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.

Prefrontal dopamine in associative learning and memory

Learning to associate specific objects or actions with rewards and remembering the associations are everyday tasks crucial for our flexible adaptation to the environment. These higher-order cognitive processes depend on the prefrontal cortex (PFC) and frontostriatal circuits that connect areas in the frontal lobe with the striatum in the basal ganglia. Both structures are densely innervated by dopamine (DA) afferents that originate in the midbrain. Although the activity of DA neurons is thought to be important for learning, the exact role of DA transmission in frontostriatal circuits during learning-related tasks is still unresolved. Moreover, the neural substrates of this modulation are poorly understood. Here, we review our recent work in monkeys utilizing local pharmacology of DA agents in the PFC to investigate the cellular mechanisms of DA modulation of associative learning and memory. We show that blocking both D1 and D2 receptors in the lateral PFC impairs learning of new stimulus-response associations and cognitive flexibility, but not the memory of highly familiar associations. In addition, D2 receptors may also contribute to motivation. The learning deficits correlated with reductions of neural information about the associations in PFC neurons, alterations in global excitability and spike synchronization, and exaggerated alpha and beta neural oscillations. Our findings provide new insights into how DA transmission modulates associative learning and memory processes in frontostriatal systems.

Dopamine modulation of learning and memory in the prefrontal cortex: insights from studies in primates, rodents, and birds

In this review, we provide a brief overview over the current knowledge about the role of dopamine transmission in the prefrontal cortex during learning and memory. We discuss work in humans, monkeys, rats, and birds in order to provide a basis for comparison across species that might help identify crucial features and constraints of the dopaminergic system in executive function. Computational models of dopamine function are introduced to provide a framework for such a comparison. We also provide a brief evolutionary perspective showing that the dopaminergic system is highly preserved across mammals. Even birds, following a largely independent evolution of higher cognitive abilities, have evolved a comparable dopaminergic system. Finally, we discuss the unique advantages and challenges of using different animal models for advancing our understanding of dopamine function in the healthy and diseased brain.