1
|
Burwell SCV, Yan H, Lim SSX, Shields BC, Tadross MR. Reward perseveration is shaped by GABA A -mediated dopamine pauses. bioRxiv 2024:2024.05.09.593320. [PMID: 38766037 PMCID: PMC11100816 DOI: 10.1101/2024.05.09.593320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
Extinction learning is an essential form of cognitive flexibility, which enables obsolete reward associations to be discarded. Its downregulation can lead to perseveration, a symptom seen in several neuropsychiatric disorders. This balance is regulated by dopamine from VTA DA (ventral tegmental area dopamine) neurons, which in turn are largely controlled by GABA (gamma amino-butyric acid) synapses. However, the causal relationship of these circuit elements to extinction and perseveration remain incompletely understood. Here, we employ an innovative drug-targeting technology, DART (drug acutely restricted by tethering), to selectively block GABA A receptors on VTA DA neurons as mice engage in Pavlovian learning. DART eliminated GABA A -mediated pauses-brief decrements in VTA DA activity canonically thought to drive extinction learning. However, contrary to the hypothesis that blocking VTA DA pauses should eliminate extinction learning, we observed the opposite-accelerated extinction learning. Specifically, DART eliminated the naturally occurring perseveration seen in half of control mice. We saw no impact on Pavlovian conditioning, nor on other aspects of VTA DA neural firing. These findings challenge canonical theories, recasting GABA A -mediated VTA DA pauses from presumed facilitators of extinction to drivers of perseveration. More broadly, this study showcases the merits of targeted synaptic pharmacology, while hinting at circuit interventions for pathological perseveration.
Collapse
|
2
|
Lindsey J, Markowitz JE, Datta SR, Litwin-Kumar A. Dynamics of striatal action selection and reinforcement learning. bioRxiv 2024:2024.02.14.580408. [PMID: 38464083 PMCID: PMC10925202 DOI: 10.1101/2024.02.14.580408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
Spiny projection neurons (SPNs) in dorsal striatum are often proposed as a locus of reinforcement learning in the basal ganglia. Here, we identify and resolve a fundamental inconsistency between striatal reinforcement learning models and known SPN synaptic plasticity rules. Direct-pathway (dSPN) and indirect-pathway (iSPN) neurons, which promote and suppress actions, respectively, exhibit synaptic plasticity that reinforces activity associated with elevated or suppressed dopamine release. We show that iSPN plasticity prevents successful learning, as it reinforces activity patterns associated with negative outcomes. However, this pathological behavior is reversed if functionally opponent dSPNs and iSPNs, which promote and suppress the current behavior, are simultaneously activated by efferent input following action selection. This prediction is supported by striatal recordings and contrasts with prior models of SPN representations. In our model, learning and action selection signals can be multiplexed without interference, enabling learning algorithms beyond those of standard temporal difference models.
Collapse
Affiliation(s)
- Jack Lindsey
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jeffrey E Markowitz
- Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | | | - Ashok Litwin-Kumar
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| |
Collapse
|
3
|
Tully J, Pereira AC, Sethi A, Griem J, Cross B, Williams SC, Blair RJ, Murphy D, Blackwood N. Impaired striatal glutamate/GABA regulation in violent offenders with antisocial personality disorder and psychopathy. Mol Psychiatry 2024:10.1038/s41380-024-02437-4. [PMID: 38326560 DOI: 10.1038/s41380-024-02437-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 01/09/2024] [Accepted: 01/16/2024] [Indexed: 02/09/2024]
Abstract
Men with antisocial personality disorder (ASPD) with or without psychopathy (+/-P) are responsible for most violent crime in society. Development of effective treatments is hindered by poor understanding of the neurochemical underpinnings of the condition. Men with ASPD with and without psychopathy demonstrate impulsive decision-making, associated with striatal abnormalities in functional neuroimaging studies. However, to date, no study has directly examined the potential neurochemical underpinnings of such abnormalities. We therefore investigated striatal glutamate: GABA ratio using Magnetic Resonance Spectroscopy in 30 violent offenders (16 ASPD-P, 14 ASPD + P) and 21 healthy non-offenders. Men with ASPD +/- P had a significant reduction in striatal glutamate : GABA ratio compared to non-offenders. We report, for the first time, striatal Glutamate/GABA dysregulation in ASPD +/- P, and discuss how this may be related to core behavioral abnormalities in the disorders.
Collapse
Affiliation(s)
- John Tully
- Academic Unit of Mental Health and Clinical Neurosciences, School of Medicine, University of Nottingham, Jubilee Campus, University of Nottingham, Wollaton Rd, Lenton, Nottingham, NG8 1BB, United Kingdom.
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom.
| | - Andreia C Pereira
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom
| | - Arjun Sethi
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom
| | - Julia Griem
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom
| | - Ben Cross
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom
| | - Steve Cr Williams
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE58AF, United Kingdom
| | - Robert James Blair
- Child and Adolescent Mental Health Centre, Mental Health Services, Capital Region of Denmark, Copenhagen, Denmark
| | - Declan Murphy
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom
| | - Nigel Blackwood
- Department of Forensic and Neurodevelopmental Sciences, Institute of Psychiatry, Psychology and Neuroscience, Kings College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom
| |
Collapse
|
4
|
Tang JCY, Paixao V, Carvalho F, Silva A, Klaus A, da Silva JA, Costa RM. Dynamic behaviour restructuring mediates dopamine-dependent credit assignment. Nature 2024; 626:583-592. [PMID: 38092040 PMCID: PMC10866702 DOI: 10.1038/s41586-023-06941-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 12/06/2023] [Indexed: 02/02/2024]
Abstract
Animals exhibit a diverse behavioural repertoire when exploring new environments and can learn which actions or action sequences produce positive outcomes. Dopamine release after encountering a reward is critical for reinforcing reward-producing actions1-3. However, it has been challenging to understand how credit is assigned to the exact action that produced the dopamine release during continuous behaviour. Here we investigated this problem in mice using a self-stimulation paradigm in which specific spontaneous movements triggered optogenetic stimulation of dopaminergic neurons. Dopamine self-stimulation rapidly and dynamically changes the structure of the entire behavioural repertoire. Initial stimulations reinforced not only the stimulation-producing target action, but also actions similar to the target action and actions that occurred a few seconds before stimulation. Repeated pairings led to a gradual refinement of the behavioural repertoire to home in on the target action. Reinforcement of action sequences revealed further temporal dependencies of refinement. Action pairs spontaneously separated by long time intervals promoted a stepwise credit assignment, with early refinement of actions most proximal to stimulation and subsequent refinement of more distal actions. Thus, a retrospective reinforcement mechanism promotes not only reinforcement, but also gradual refinement of the entire behavioural repertoire to assign credit to specific actions and action sequences that lead to dopamine release.
Collapse
Affiliation(s)
- Jonathan C Y Tang
- Department of Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Seattle Children's Research Institute, Center for Integrative Brain Research, Seattle, WA, USA
- Department of Pediatrics, University of Washington School of Medicine, Seattle, WA, USA
| | - Vitor Paixao
- Champalimaud Neuroscience Programme, Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
- Kinetikos, Coimbra, Portugal
| | - Filipe Carvalho
- Champalimaud Neuroscience Programme, Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
- Open Ephys Production Site, Lisbon, Portugal
| | - Artur Silva
- Champalimaud Neuroscience Programme, Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
| | - Andreas Klaus
- Champalimaud Neuroscience Programme, Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
| | - Joaquim Alves da Silva
- Champalimaud Neuroscience Programme, Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
- Champalimaud Experimental Clinical Research Programme, Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
- NOVA Medical School, Universidade NOVA de Lisboa, Lisbon, Portugal
| | - Rui M Costa
- Department of Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA.
- Aligning Science Across Parkinson's Collaborative Research Network, Chevy Chase, MD, USA.
- Allen Institute, Seattle, WA, USA.
| |
Collapse
|
5
|
Favila N, Gurney K, Overton PG. Role of the basal ganglia in innate and learned behavioural sequences. Rev Neurosci 2024; 35:35-55. [PMID: 37437141 DOI: 10.1515/revneuro-2023-0038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 06/24/2023] [Indexed: 07/14/2023]
Abstract
Integrating individual actions into coherent, organised behavioural units, a process called chunking, is a fundamental, evolutionarily conserved process that renders actions automatic. In vertebrates, evidence points to the basal ganglia - a complex network believed to be involved in action selection - as a key component of action sequence encoding, although the underlying mechanisms are only just beginning to be understood. Central pattern generators control many innate automatic behavioural sequences that form some of the most basic behaviours in an animal's repertoire, and in vertebrates, brainstem and spinal pattern generators are under the control of higher order structures such as the basal ganglia. Evidence suggests that the basal ganglia play a crucial role in the concatenation of simpler behaviours into more complex chunks, in the context of innate behavioural sequences such as chain grooming in rats, as well as sequences in which innate capabilities and learning interact such as birdsong, and sequences that are learned from scratch, such as lever press sequences in operant behaviour. It has been proposed that the role of the striatum, the largest input structure of the basal ganglia, might lie in selecting and allowing the relevant central pattern generators to gain access to the motor system in the correct order, while inhibiting other behaviours. As behaviours become more complex and flexible, the pattern generators seem to become more dependent on descending signals. Indeed, during learning, the striatum itself may adopt the functional characteristics of a higher order pattern generator, facilitated at the microcircuit level by striatal neuropeptides.
Collapse
Affiliation(s)
- Natalia Favila
- German Center for Neurodegenerative Diseases, 53127 Bonn, Germany
| | - Kevin Gurney
- Department of Psychology, The University of Sheffield, Sheffield S1 2LT, UK
| | - Paul G Overton
- Department of Psychology, The University of Sheffield, Sheffield S1 2LT, UK
| |
Collapse
|
6
|
Xie X, Lu J, Ma T, Cheng Y, Woodson K, Bonifacio J, Bego K, Wang X, Wang J. Linking input- and cell-type-specific synaptic plasticity to the reinforcement of alcohol-seeking behavior. Neuropharmacology 2023; 237:109619. [PMID: 37290535 DOI: 10.1016/j.neuropharm.2023.109619] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 05/15/2023] [Accepted: 05/27/2023] [Indexed: 06/10/2023]
Abstract
The reinforcement of voluntary alcohol-seeking behavior requires dopamine-dependent long-term synaptic plasticity in the striatum. Specifically, the long-term potentiation (LTP) of direct-pathway medium spiny neurons (dMSNs) in the dorsomedial striatum (DMS) promotes alcohol drinking. However, it remains unclear whether alcohol induces input-specific plasticity onto dMSNs and whether this plasticity directly drives instrumental conditioning. In this study, we found that voluntary alcohol intake selectively strengthened glutamatergic transmission from the medial prefrontal cortex (mPFC) to DMS dMSNs in mice. Importantly, mimicking this alcohol-induced potentiation by optogenetically self-stimulating mPFC→dMSN synapse with an LTP protocol was sufficient to drive the reinforcement of lever pressing in operant chambers. Conversely, induction of a post-pre spike timing-dependent LTD at this synapse time-locked to alcohol delivery during operant conditioning persistently decreased alcohol-seeking behavior. Our results establish a causal relationship between input- and cell-type-specific corticostriatal plasticity and the reinforcement of alcohol-seeking behavior. This provides a potential therapeutic strategy to restore normal cortical control of dysregulated basal ganglia circuitries in alcohol use disorder.
Collapse
Affiliation(s)
- Xueyi Xie
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Jiayi Lu
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Tengfei Ma
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Yifeng Cheng
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Kayla Woodson
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Jordan Bonifacio
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Kassidy Bego
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Xuehua Wang
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA
| | - Jun Wang
- Department of Neuroscience and Experimental Therapeutics, School of Medicine, Texas A&M University Health Science Center, Bryan, TX, 77807, USA.
| |
Collapse
|
7
|
Chantranupong L, Beron CC, Zimmer JA, Wen MJ, Wang W, Sabatini BL. Dopamine and glutamate regulate striatal acetylcholine in decision-making. Nature 2023; 621:577-585. [PMID: 37557915 PMCID: PMC10511323 DOI: 10.1038/s41586-023-06492-9] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 07/28/2023] [Indexed: 08/11/2023]
Abstract
Striatal dopamine and acetylcholine are essential for the selection and reinforcement of motor actions and decision-making1. In vitro studies have revealed an intrastriatal circuit in which acetylcholine, released by cholinergic interneurons (CINs), drives the release of dopamine, and dopamine, in turn, inhibits the activity of CINs through dopamine D2 receptors (D2Rs). Whether and how this circuit contributes to striatal function in vivo is largely unknown. Here, to define the role of this circuit in a living system, we monitored acetylcholine and dopamine signals in the ventrolateral striatum of mice performing a reward-based decision-making task. We establish that dopamine and acetylcholine exhibit multiphasic and anticorrelated transients that are modulated by decision history and reward outcome. Dopamine dynamics and reward encoding do not require the release of acetylcholine by CINs. However, dopamine inhibits acetylcholine transients in a D2R-dependent manner, and loss of this regulation impairs decision-making. To determine how other striatal inputs shape acetylcholine signals, we assessed the contribution of cortical and thalamic projections, and found that glutamate release from both sources is required for acetylcholine release. Altogether, we uncover a dynamic relationship between dopamine and acetylcholine during decision-making, and reveal multiple modes of CIN regulation. These findings deepen our understanding of the neurochemical basis of decision-making and behaviour.
Collapse
Affiliation(s)
- Lynne Chantranupong
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, USA
| | - Celia C Beron
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, USA
| | - Joshua A Zimmer
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, USA
| | - Michelle J Wen
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, USA
| | - Wengang Wang
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, USA
| | - Bernardo L Sabatini
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, USA.
| |
Collapse
|
8
|
Wärnberg E, Kumar A. Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia. Proc Natl Acad Sci U S A 2023; 120:e2221994120. [PMID: 37527344 PMCID: PMC10410740 DOI: 10.1073/pnas.2221994120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Accepted: 06/08/2023] [Indexed: 08/03/2023] Open
Abstract
It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small discrete set of actions (e.g., no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of continuous outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). We implement a local, "three-factor" corticostriatal plasticity rule involving the presynaptic firing rate, a postsynaptic factor, and the unique dopamine concentration perceived by each striatal neuron. With this learning rule, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. Crucially, we demonstrate that this plasticity rule does not require precise nigrostriatal synapses but remains compatible with experimental observations of random placement of varicosities and diffuse volume transmission of dopamine.
Collapse
Affiliation(s)
- Emil Wärnberg
- Department of Neuroscience, Karolinska Institutet, 171 77Stockholm, Sweden
- Division of Computational Science and Technology, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, 114 28Stockholm, Sweden
| | - Arvind Kumar
- Division of Computational Science and Technology, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, 114 28Stockholm, Sweden
| |
Collapse
|
9
|
Vautrelle N, Coizet V, Leriche M, Dahan L, Schulz JM, Zhang YF, Zeghbib A, Overton PG, Bracci E, Redgrave P, Reynolds JN. Sensory Reinforced Corticostriatal Plasticity. Curr Neuropharmacol 2023; 22:CN-EPUB-133306. [PMID: 37533245 PMCID: PMC11097983 DOI: 10.2174/1570159x21666230801110359] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2022] [Revised: 02/04/2023] [Accepted: 02/10/2023] [Indexed: 08/04/2023] Open
Abstract
BACKGROUND Regional changes in corticostriatal transmission induced by phasic dopaminergic signals are an essential feature of the neural network responsible for instrumental reinforcement during discovery of an action. However, the timing of signals that are thought to contribute to the induction of corticostriatal plasticity is difficult to reconcile within the framework of behavioural reinforcement learning, because the reinforcer is normally delayed relative to the selection and execution of causally-related actions. OBJECTIVE While recent studies have started to address the relevance of delayed reinforcement signals and their impact on corticostriatal processing, our objective was to establish a model in which a sensory reinforcer triggers appropriately delayed reinforcement signals relayed to the striatum via intact neuronal pathways and to investigate the effects on corticostriatal plasticity. METHODS We measured corticostriatal plasticity with electrophysiological recordings using a light flash as a natural sensory reinforcer, and pharmacological manipulations were applied in an in vivo anesthetized rat model preparation. RESULTS We demonstrate that the spiking of striatal neurons evoked by single-pulse stimulation of the motor cortex can be potentiated by a natural sensory reinforcer, operating through intact afferent pathways, with signal timing approximating that required for behavioural reinforcement. The pharmacological blockade of dopamine receptors attenuated the observed potentiation of corticostriatal neurotransmission. CONCLUSION This novel in vivo model of corticostriatal plasticity offers a behaviourally relevant framework to address the physiological, anatomical, cellular, and molecular bases of instrumental reinforcement learning.
Collapse
Affiliation(s)
- Nicolas Vautrelle
- Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
| | - Véronique Coizet
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
- Institut des Neurosciences de Grenoble, Université Joseph Fourier, Inserm, U1216, 38706 La Tronche Cedex, France
| | - Mariana Leriche
- Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
| | - Lionel Dahan
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
- Centre de Recherches sur la Cognition Animale, Université de Toulouse, UPS, 118 Route de Narbonne, F-31062 Toulouse Cedex 9, France
| | - Jan M. Schulz
- Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand
- Department of Biomedicine, University of Basel, CH - 4056 Basel, Switzerland
| | - Yan-Feng Zhang
- Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand
- Department of Clinical and Biomedical Sciences, University of Exeter Medical School, Hatherly Laboratories, Exeter EX4 4PS, United Kingdom
| | - Abdelhafid Zeghbib
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
| | - Paul G. Overton
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
| | - Enrico Bracci
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
| | - Peter Redgrave
- Department of Psychology, University of Sheffield, Sheffield, S10 2TP, UK
| | - John N.J. Reynolds
- Department of Anatomy, Brain Health Research Centre, University of Otago, Dunedin 9054, New Zealand
| |
Collapse
|
10
|
Yagishita S. Cellular bases for reward-related dopamine actions. Neurosci Res 2023; 188:1-9. [PMID: 36496085 DOI: 10.1016/j.neures.2022.12.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2022] [Revised: 11/09/2022] [Accepted: 12/06/2022] [Indexed: 12/12/2022]
Abstract
Dopamine neurons exhibit transient increases and decreases in their firing rate upon reward and punishment for learning. This bidirectional modulation of dopamine dynamics occurs on the order of hundreds of milliseconds, and it is sensitively detected to reinforce the preceding sensorimotor events. These observations indicate that the mechanisms of dopamine detection at the projection sites are of remarkable precision, both in time and concentration. A major target of dopamine projection is the striatum, including the ventral region of the nucleus accumbens, which mainly comprises dopamine D1 and D2 receptor (D1R and D2R)-expressing spiny projection neurons. Although the involvement of D1R and D2R in dopamine-dependent learning has been suggested, the exact cellular bases for detecting transient dopamine signaling remain unclear. This review discusses recent cellular studies on the novel synaptic mechanisms for detecting dopamine transient signals associated with learning. Analyses of behavior based on these mechanisms have further revealed new behavioral aspects that are closely associated with these synaptic mechanisms. Thus, it is gradually possible to mechanistically explain behavioral learning via synaptic and cellular bases in rodents.
Collapse
Affiliation(s)
- Sho Yagishita
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, The University of Tokyo, Bunkyo-ku, Tokyo, Japan; International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan.
| |
Collapse
|
11
|
Dorman DB, Blackwell KT. Synaptic Plasticity Is Predicted by Spatiotemporal Firing Rate Patterns and Robust to In Vivo-like Variability. Biomolecules 2022; 12:1402. [PMID: 36291612 PMCID: PMC9599115 DOI: 10.3390/biom12101402] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 09/13/2022] [Accepted: 09/28/2022] [Indexed: 11/22/2022] Open
Abstract
Synaptic plasticity, the experience-induced change in connections between neurons, underlies learning and memory in the brain. Most of our understanding of synaptic plasticity derives from in vitro experiments with precisely repeated stimulus patterns; however, neurons exhibit significant variability in vivo during repeated experiences. Further, the spatial pattern of synaptic inputs to the dendritic tree influences synaptic plasticity, yet is not considered in most synaptic plasticity rules. Here, we investigate how spatiotemporal synaptic input patterns produce plasticity with in vivo-like conditions using a data-driven computational model with a plasticity rule based on calcium dynamics. Using in vivo spike train recordings as inputs to different size clusters of spines, we show that plasticity is strongly robust to trial-to-trial variability of spike timing. In addition, we derive general synaptic plasticity rules describing how spatiotemporal patterns of synaptic inputs control the magnitude and direction of plasticity. Synapses that strongly potentiated have greater firing rates and calcium concentration later in the trial, whereas strongly depressing synapses have hiring firing rates early in the trial. The neighboring synaptic activity influences the direction and magnitude of synaptic plasticity, with small clusters of spines producing the greatest increase in synaptic strength. Together, our results reveal that calcium dynamics can unify diverse plasticity rules and reveal how spatiotemporal firing rate patterns control synaptic plasticity.
Collapse
Affiliation(s)
- Daniel B. Dorman
- Interdisciplinary Program in Neuroscience, George Mason University, Fairfax, VA 22030, USA
| | - Kim T. Blackwell
- Interdisciplinary Program in Neuroscience, George Mason University, Fairfax, VA 22030, USA
- Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, VA 22030, USA
| |
Collapse
|
12
|
Mizrahi-Kliger AD, Kaplan A, Israel Z, Bergman H. Entrainment to sleep spindles reflects dissociable patterns of connectivity between cortex and basal ganglia. Cell Rep 2022; 40:111367. [PMID: 36130495 DOI: 10.1016/j.celrep.2022.111367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 07/20/2022] [Accepted: 08/25/2022] [Indexed: 11/18/2022] Open
Abstract
Sleep spindles are crucial for learning in the cortex and basal ganglia (BG) because they facilitate the reactivation of previously active neuronal ensembles. Studying field potentials (FPs) and spiking in the cortex and BG during sleep in non-human primates following pre-sleep learning, we show that FP sleep spindles are widespread in the BG and are similar to cortical spindles in morphology, spectral content, and response to the pre-sleep task. Further, BG spindles are concordant with electroencephalogram (EEG) spindles and associated with increased cortico-BG correlation. However, spindles across the BG differ markedly in their entrainment of local spiking. The spiking activity of striatal projection neurons exhibits consistent phase locking to striatal and EEG spindles, producing phase windows of peaked cross-region spindling. In contrast, firing in other BG nuclei is not entrained to either local or EEG sleep spindles. These results suggest corticostriatal synapses as the main hub for offline cortico-BG communication.
Collapse
Affiliation(s)
- Aviv D Mizrahi-Kliger
- Department of Neurobiology, Institute of Medical Research Israel-Canada, Hadassah Medical School, The Hebrew University of Jerusalem, 9112001 Jerusalem, Israel.
| | - Alexander Kaplan
- Department of Neurobiology, Institute of Medical Research Israel-Canada, Hadassah Medical School, The Hebrew University of Jerusalem, 9112001 Jerusalem, Israel; The Edmond and Lily Safra Center for Brain Sciences, The Hebrew University, 9190401 Jerusalem, Israel
| | - Zvi Israel
- Department of Neurosurgery, Hadassah University Hospital, 9112001 Jerusalem, Israel
| | - Hagai Bergman
- Department of Neurobiology, Institute of Medical Research Israel-Canada, Hadassah Medical School, The Hebrew University of Jerusalem, 9112001 Jerusalem, Israel; The Edmond and Lily Safra Center for Brain Sciences, The Hebrew University, 9190401 Jerusalem, Israel; Department of Neurosurgery, Hadassah University Hospital, 9112001 Jerusalem, Israel
| |
Collapse
|
13
|
Laverne G, Pesce J, Reynders A, Combrisson E, Gascon E, Melon C, Kerkerian-Le Goff L, Maurice N, Beurrier C. Cholinergic interneuron inhibition potentiates corticostriatal transmission in direct medium spiny neurons and rescues motor learning in parkinsonism. Cell Rep 2022; 40:111034. [PMID: 35793632 DOI: 10.1016/j.celrep.2022.111034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 04/27/2022] [Accepted: 06/11/2022] [Indexed: 11/30/2022] Open
Abstract
Striatal cholinergic interneurons (CINs) respond to salient or reward prediction-related stimuli after conditioning with brief pauses in their activity, implicating them in learning and action selection. This pause is lost in animal models of Parkinson's disease. How this signal regulates the striatal network remains an open question. Here, we examine the impact of CIN firing inhibition on glutamatergic transmission between the cortex and the medium spiny neurons expressing dopamine D1 receptor (D1 MSNs). Brief interruption of CIN activity has no effect in control conditions, whereas it increases glutamatergic responses in D1 MSNs after dopamine denervation. This potentiation depends upon M4 muscarinic receptor and protein kinase A. Decreasing CIN firing by optogenetics/chemogenetics in vivo partially rescues long-term potentiation in MSNs and motor learning deficits in parkinsonian mice. Our findings demonstrate that the control exerted by CINs on corticostriatal transmission and striatal-dependent motor-skill learning depends on the integrity of dopaminergic inputs.
Collapse
Affiliation(s)
- Gwenaëlle Laverne
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France
| | - Jonathan Pesce
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France
| | - Ana Reynders
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France
| | - Etienne Combrisson
- Aix Marseille University, CNRS, Institut de Neurosciences de la Timone (INT), 13005 Marseille, France
| | - Eduardo Gascon
- Aix Marseille University, CNRS, Institut de Neurosciences de la Timone (INT), 13005 Marseille, France
| | - Christophe Melon
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France
| | - Lydia Kerkerian-Le Goff
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France
| | - Nicolas Maurice
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France
| | - Corinne Beurrier
- Aix Marseille University, CNRS, Institut de Biologie du Développement (IBDM), 13009 Marseille, France.
| |
Collapse
|
14
|
Hong SZ, Mesik L, Grossman CD, Cohen JY, Lee B, Severin D, Lee HK, Hell JW, Kirkwood A. Norepinephrine potentiates and serotonin depresses visual cortical responses by transforming eligibility traces. Nat Commun 2022; 13:3202. [PMID: 35680879 PMCID: PMC9184610 DOI: 10.1038/s41467-022-30827-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Accepted: 05/19/2022] [Indexed: 11/18/2022] Open
Abstract
Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in mouse V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.
Collapse
Affiliation(s)
- Su Z Hong
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Lukas Mesik
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Cooper D Grossman
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Jeremiah Y Cohen
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Boram Lee
- Department of Pharmacology, University of California at Davis, Davis, CA, 95616, USA
| | - Daniel Severin
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Hey-Kyoung Lee
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Johannes W Hell
- Department of Pharmacology, University of California at Davis, Davis, CA, 95616, USA
| | - Alfredo Kirkwood
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA.
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA.
| |
Collapse
|
15
|
Parker NF, Baidya A, Cox J, Haetzel LM, Zhukovskaya A, Murugan M, Engelhard B, Goldman MS, Witten IB. Choice-selective sequences dominate in cortical relative to thalamic inputs to NAc to support reinforcement learning. Cell Rep 2022; 39:110756. [PMID: 35584665 PMCID: PMC9218875 DOI: 10.1016/j.celrep.2022.110756] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2019] [Revised: 02/18/2022] [Accepted: 04/07/2022] [Indexed: 11/25/2022] Open
Abstract
How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens, which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex and midline regions of the thalamus. However, little is known about whether and how representations differ across these input pathways. By comparing these inputs during a reinforcement learning task in mice, we discovered that prelimbic cortical inputs preferentially represent actions and choices, whereas midline thalamic inputs preferentially represent cues. Choice-selective activity in the prelimbic cortical inputs is organized in sequences that persist beyond the outcome. Through computational modeling, we demonstrate that these sequences can support the neural implementation of reinforcement-learning algorithms, in both a circuit model based on synaptic plasticity and one based on neural dynamics. Finally, we test and confirm a prediction of our circuit models by direct manipulation of nucleus accumbens input neurons.
Collapse
Affiliation(s)
- Nathan F Parker
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Avinash Baidya
- Center for Neuroscience, University of California, Davis, Davis, CA 95616, USA; Department of Physics and Astronomy, University of California, Davis, Davis, CA 95616, USA
| | - Julia Cox
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Laura M Haetzel
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Anna Zhukovskaya
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Malavika Murugan
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Ben Engelhard
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Mark S Goldman
- Center for Neuroscience, University of California, Davis, Davis, CA 95616, USA; Department of Neurobiology, Physiology and Behavior, University of California, Davis, Davis, CA 95616, USA; Department of Ophthalmology and Vision Science, University of California, Davis, Davis, CA 95616, USA.
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Department of Psychology, Princeton University, Princeton, NJ 08544, USA.
| |
Collapse
|
16
|
Möller M, Manohar S, Bogacz R. Uncertainty-guided learning with scaled prediction errors in the basal ganglia. PLoS Comput Biol 2022; 18:e1009816. [PMID: 35622863 PMCID: PMC9182698 DOI: 10.1371/journal.pcbi.1009816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Revised: 06/09/2022] [Accepted: 05/05/2022] [Indexed: 11/19/2022] Open
Abstract
To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.
Collapse
Affiliation(s)
- Moritz Möller
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Sanjay Manohar
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Rafal Bogacz
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
17
|
Perez S, Cui Y, Vignoud G, Perrin E, Mendes A, Zheng Z, Touboul J, Venance L. Striatum expresses region-specific plasticity consistent with distinct memory abilities. Cell Rep 2022; 38:110521. [PMID: 35294877 DOI: 10.1016/j.celrep.2022.110521] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Revised: 12/23/2021] [Accepted: 02/21/2022] [Indexed: 11/24/2022] Open
Abstract
The striatum mediates two learning modalities: goal-directed behavior in dorsomedial (DMS) and habits in dorsolateral (DLS) striata. The synaptic bases of these learnings are still elusive. Indeed, while ample research has described DLS plasticity, little remains known about DMS plasticity and its involvement in procedural learning. Here, we find symmetric and asymmetric anti-Hebbian spike-timing-dependent plasticity (STDP) in DMS and DLS, respectively, with opposite plasticity dominance upon increasing corticostriatal activity. During motor-skill learning, plasticity is engaged in DMS and striatonigral DLS neurons only during early learning stages, whereas striatopallidal DLS neurons are mobilized only during late phases. With a mathematical modeling approach, we find that symmetric anti-Hebbian STDP favors memory flexibility, while asymmetric anti-Hebbian STDP favors memory maintenance, consistent with memory processes at play in procedural learning.
Collapse
Affiliation(s)
- Sylvie Perez
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France
| | - Yihui Cui
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France; Department of Neurobiology, Department of Neurology of Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou 310058, China
| | - Gaëtan Vignoud
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France; MAMBA-Modelling and Analysis for Medical and Biological Applications, Inria Paris, LJLL (UMR-7598) -Laboratory Jacques-Louis Lions, Paris, France
| | - Elodie Perrin
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France
| | - Alexandre Mendes
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France
| | - Zhiwei Zheng
- Department of Neurobiology, Department of Neurology of Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou 310058, China
| | - Jonathan Touboul
- Department of Mathematics and Volen National Center for Complex Systems, Brandeis University, Waltham, MA, USA
| | - Laurent Venance
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France.
| |
Collapse
|
18
|
Reynolds JNJ, Avvisati R, Dodson PD, Fisher SD, Oswald MJ, Wickens JR, Zhang YF. Coincidence of cholinergic pauses, dopaminergic activation and depolarisation of spiny projection neurons drives synaptic plasticity in the striatum. Nat Commun 2022; 13:1296. [PMID: 35277506 PMCID: PMC8917208 DOI: 10.1038/s41467-022-28950-0] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Accepted: 02/18/2022] [Indexed: 11/17/2022] Open
Abstract
Dopamine-dependent long-term plasticity is believed to be a cellular mechanism underlying reinforcement learning. In response to reward and reward-predicting cues, phasic dopamine activity potentiates the efficacy of corticostriatal synapses on spiny projection neurons (SPNs). Since phasic dopamine activity also encodes other behavioural variables, it is unclear how postsynaptic neurons identify which dopamine event is to induce long-term plasticity. Additionally, it is unknown how phasic dopamine released from arborised axons can potentiate targeted striatal synapses through volume transmission. To examine these questions we manipulated striatal cholinergic interneurons (ChIs) and dopamine neurons independently in two distinct in vivo paradigms. We report that long-term potentiation (LTP) at corticostriatal synapses with SPNs is dependent on the coincidence of pauses in ChIs and phasic dopamine activation, critically accompanied by SPN depolarisation. Thus, the ChI pause defines the time window for phasic dopamine to induce plasticity, while depolarisation of SPNs constrains the synapses eligible for plasticity. It remains unclear how corticostriatal synapses utilize reward prediction error signaling in order to reinforce reward-related behaviors. Here, the authors show that potentiation of corticostriatal synapses requires phasic dopamine activation, pauses in striatal cholinergic interneuron firing, and depolarization of spiny projection neurons.
Collapse
Affiliation(s)
- John N J Reynolds
- Department of Anatomy, University of Otago, School of Biomedical Sciences, Brain Health Research Centre, P.O. Box 913, Dunedin, New Zealand.
| | - Riccardo Avvisati
- School of Physiology, Pharmacology & Neuroscience, University of Bristol, Bristol, BS8 1TD, UK
| | - Paul D Dodson
- School of Physiology, Pharmacology & Neuroscience, University of Bristol, Bristol, BS8 1TD, UK
| | - Simon D Fisher
- Department of Anatomy, University of Otago, School of Biomedical Sciences, Brain Health Research Centre, P.O. Box 913, Dunedin, New Zealand
| | - Manfred J Oswald
- Department of Anatomy, University of Otago, School of Biomedical Sciences, Brain Health Research Centre, P.O. Box 913, Dunedin, New Zealand
| | - Jeffery R Wickens
- Department of Anatomy, University of Otago, School of Biomedical Sciences, Brain Health Research Centre, P.O. Box 913, Dunedin, New Zealand.,Okinawa Institute of Science and Technology, Okinawa, 904-2234, Japan
| | - Yan-Feng Zhang
- Department of Anatomy, University of Otago, School of Biomedical Sciences, Brain Health Research Centre, P.O. Box 913, Dunedin, New Zealand. .,Department of Physiology, Anatomy & Genetics, University of Oxford, Oxford, OX1 3PT, UK.
| |
Collapse
|
19
|
Yamaguchi K, Maeda Y, Sawada T, Iino Y, Tajiri M, Nakazato R, Ishii S, Kasai H, Yagishita S. A behavioural correlate of the synaptic eligibility trace in the nucleus accumbens. Sci Rep 2022; 12:1921. [PMID: 35121769 PMCID: PMC8817024 DOI: 10.1038/s41598-022-05637-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Accepted: 01/17/2022] [Indexed: 11/09/2022] Open
Abstract
Reward reinforces the association between a preceding sensorimotor event and its outcome. Reinforcement learning (RL) theory and recent brain slice studies explain the delayed reward action such that synaptic activities triggered by sensorimotor events leave a synaptic eligibility trace for 1 s. The trace produces a sensitive period for reward-related dopamine to induce synaptic plasticity in the nucleus accumbens (NAc). However, the contribution of the synaptic eligibility trace to behaviour remains unclear. Here we examined a reward-sensitive period to brief pure tones with an accurate measurement of an effective timing of water reward in head-fixed Pavlovian conditioning, which depended on the plasticity-related signaling in the NAc. We found that the reward-sensitive period was within 1 s after the pure tone presentation and optogenetically-induced presynaptic activities at the NAc, showing that the short reward-sensitive period was in conformity with the synaptic eligibility trace in the NAc. These findings support the application of the synaptic eligibility trace to construct biologically plausible RL models.
Collapse
Affiliation(s)
- Kenji Yamaguchi
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan.,Department of Psychology, Waseda University, Shinjuku-ku, Tokyo, Japan
| | - Yoshitomo Maeda
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
| | - Takeshi Sawada
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
| | - Yusuke Iino
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
| | - Mio Tajiri
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
| | - Ryosuke Nakazato
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
| | - Shin Ishii
- International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan.,Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Kyoto, Japan
| | - Haruo Kasai
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan
| | - Sho Yagishita
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, Faculty of Medicine Bldg, The University of Tokyo, 1 #NC207, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan. .,International Research Center for Neurointelligence (WPI-IRCN), UTIAS, The University of Tokyo, Bunkyo-ku, Tokyo, Japan.
| |
Collapse
|
20
|
Scholl C, Baladron J, Vitay J, Hamker FH. Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model. Brain Struct Funct 2022. [PMID: 35113242 DOI: 10.1007/s00429-021-02446-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Accepted: 12/15/2021] [Indexed: 12/28/2022]
Abstract
Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.
Collapse
|
21
|
Holly EN, Davatolhagh MF, España RA, Fuccillo MV. Striatal low-threshold spiking interneurons locally gate dopamine. Curr Biol 2021; 31:4139-4147.e6. [PMID: 34302742 DOI: 10.1016/j.cub.2021.06.081] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2020] [Revised: 05/02/2021] [Accepted: 06/25/2021] [Indexed: 11/27/2022]
Abstract
The dorsomedial striatum (DMS) is a central hub supporting goal-directed learning and motor performance. Recent evidence has revealed unexpected roles for local inhibitory GABAergic networks in modulating striatal output and behavior.1 The sparse low-threshold spiking interneuron subtype (LTSI), which exhibits robust reward-circumscribed population activity, is a bidirectional regulator of initial goal-directed learning.2 Striatal dopamine signaling is a central reward-related neuromodulatory system mediating goal-directed action and performance, serving as a teaching signal,3 facilitating synaptic plasticity,4 and invigorating motor behaviors.5 Given the dynamic modulation of LTSIs during goal-directed behavior, we hypothesized that they could provide a novel GABAergic mechanism of local striatal dopaminergic regulation to shape early learning. We provide anatomical evidence for close proximation of LTSI terminals and dopaminergic processes in striatum, suggesting that LTSIs directly control dopaminergic axon activity. Using in vitro fast scan cyclic voltammetry, we demonstrate that LTSIs directly attenuate optogenetically evoked dopamine via GABAB receptor signaling. In vivo, GRABDA dopamine sensor imaging shows that LTSIs strongly modulate striatal dopamine dynamics during operant learning, while pharmacological stabilization of dopamine via intra-striatal aripiprazole microinjection suppresses the effects of LTSI inhibition on learning. Together, these results uncover an unexpected function for LTSIs in gating striatal dopamine to facilitate goal-directed learning.
Collapse
Affiliation(s)
- Elizabeth N Holly
- Department of Neuroscience, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| | - M Felicia Davatolhagh
- Department of Neuroscience, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Neuroscience Graduate Group, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Rodrigo A España
- Department of Neurobiology and Anatomy, Drexel University College of Medicine, Philadelphia, PA, USA
| | - Marc V Fuccillo
- Department of Neuroscience, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| |
Collapse
|
22
|
Urakubo H, Yagishita S, Kasai H, Kubota Y, Ishii S. The critical balance between dopamine D2 receptor and RGS for the sensitive detection of a transient decay in dopamine signal. PLoS Comput Biol 2021; 17:e1009364. [PMID: 34591840 PMCID: PMC8483376 DOI: 10.1371/journal.pcbi.1009364] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Accepted: 08/18/2021] [Indexed: 12/19/2022] Open
Abstract
In behavioral learning, reward-related events are encoded into phasic dopamine (DA) signals in the brain. In particular, unexpected reward omission leads to a phasic decrease in DA (DA dip) in the striatum, which triggers long-term potentiation (LTP) in DA D2 receptor (D2R)-expressing spiny-projection neurons (D2 SPNs). While this LTP is required for reward discrimination, it is unclear how such a short DA-dip signal (0.5-2 s) is transferred through intracellular signaling to the coincidence detector, adenylate cyclase (AC). In the present study, we built a computational model of D2 signaling to determine conditions for the DA-dip detection. The DA dip can be detected only if the basal DA signal sufficiently inhibits AC, and the DA-dip signal sufficiently disinhibits AC. We found that those two requirements were simultaneously satisfied only if two key molecules, D2R and regulators of G protein signaling (RGS) were balanced within a certain range; this balance has indeed been observed in experimental studies. We also found that high level of RGS was required for the detection of a 0.5-s short DA dip, and the analytical solutions for these requirements confirmed their universality. The imbalance between D2R and RGS is associated with schizophrenia and DYT1 dystonia, both of which are accompanied by abnormal striatal LTP. Our simulations suggest that D2 SPNs in patients with schizophrenia and DYT1 dystonia cannot detect short DA dips. We finally discussed that such psychiatric and movement disorders can be understood in terms of the imbalance between D2R and RGS.
Collapse
Affiliation(s)
- Hidetoshi Urakubo
- Integrated Systems Biology Laboratory, Department of Systems Science, Graduate School of Informatics, Kyoto University, Kyoto, Japan
- Section of Electron Microscopy, National Institute for Physiological Sciences, Okazaki, Aichi, Japan
| | - Sho Yagishita
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, University of Tokyo, Bunkyo-ku, Tokyo, Japan
- International Research Center for Neurointelligence (WPI-IRCN), University of Tokyo Institutes for Advanced Study (UTIAS), Tokyo, Japan
| | - Haruo Kasai
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, University of Tokyo, Bunkyo-ku, Tokyo, Japan
- International Research Center for Neurointelligence (WPI-IRCN), University of Tokyo Institutes for Advanced Study (UTIAS), Tokyo, Japan
| | - Yoshiyuki Kubota
- Section of Electron Microscopy, National Institute for Physiological Sciences, Okazaki, Aichi, Japan
- Department of Physiological Sciences, The Graduate University for Advanced Studies (SOKENDAI), Okazaki, Aichi, Japan
| | - Shin Ishii
- Integrated Systems Biology Laboratory, Department of Systems Science, Graduate School of Informatics, Kyoto University, Kyoto, Japan
- International Research Center for Neurointelligence (WPI-IRCN), University of Tokyo Institutes for Advanced Study (UTIAS), Tokyo, Japan
| |
Collapse
|
23
|
Mihalas S, Ardiles A, He K, Palacios A, Kirkwood A. A Multisubcellular Compartment Model of AMPA Receptor Trafficking for Neuromodulation of Hebbian Synaptic Plasticity. Front Synaptic Neurosci 2021; 13:703621. [PMID: 34456706 PMCID: PMC8385783 DOI: 10.3389/fnsyn.2021.703621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 07/05/2021] [Indexed: 11/13/2022] Open
Abstract
Neuromodulation can profoundly impact the gain and polarity of postsynaptic changes in Hebbian synaptic plasticity. An emerging pattern observed in multiple central synapses is a pull–push type of control in which activation of receptors coupled to the G-protein Gs promote long-term potentiation (LTP) at the expense of long-term depression (LTD), whereas receptors coupled to Gq promote LTD at the expense of LTP. Notably, coactivation of both Gs- and Gq-coupled receptors enhances the gain of both LTP and LTD. To account for these observations, we propose a simple kinetic model in which AMPA receptors (AMPARs) are trafficked between multiple subcompartments in and around the postsynaptic spine. In the model AMPARs in the postsynaptic density compartment (PSD) are the primary contributors to synaptic conductance. During LTP induction, AMPARs are trafficked to the PSD primarily from a relatively small perisynaptic (peri-PSD) compartment. Gs-coupled receptors promote LTP by replenishing peri-PSD through increased AMPAR exocytosis from a pool of endocytic AMPAR. During LTD induction AMPARs are trafficked in the reverse direction, from the PSD to the peri-PSD compartment, and Gq-coupled receptors promote LTD by clearing the peri-PSD compartment through increased AMPAR endocytosis. We claim that the model not only captures essential features of the pull–push neuromodulation of synaptic plasticity, but it is also consistent with other actions of neuromodulators observed in slice experiments and is compatible with the current understanding of AMPAR trafficking.
Collapse
Affiliation(s)
- Stefan Mihalas
- Allen Institute for Brain Science, Seattle, WA, United States
| | - Alvaro Ardiles
- Centro Interdisciplinario de Neurociencia de Valparaíso, Facultad de Ciencias, Universidad de Valparaíso, Valparaíso, Chile.,Centro de Neurología Traslacional, Facultad de Medicina, Universidad de Valparaíso, Valparaíso, Chile
| | - Kaiwen He
- Mind Brain Institute, Johns Hopkins University, Baltimore, MD, United States
| | - Adrian Palacios
- Centro Interdisciplinario de Neurociencia de Valparaíso, Facultad de Ciencias, Universidad de Valparaíso, Valparaíso, Chile
| | - Alfredo Kirkwood
- Mind Brain Institute, Johns Hopkins University, Baltimore, MD, United States
| |
Collapse
|
24
|
Woolrych A, Vautrelle N, Reynolds JNJ, Parr-Brownlie LC. Throwing open the doors of perception: The role of dopamine in visual processing. Eur J Neurosci 2021; 54:6135-6146. [PMID: 34340265 DOI: 10.1111/ejn.15408] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2020] [Revised: 07/05/2021] [Accepted: 07/18/2021] [Indexed: 01/11/2023]
Abstract
Animals form associations between visual cues and behaviours. Although dopamine is known to be critical in many areas of the brain to bind sensory information with appropriate responses, dopamine's role in the visual system is less well understood. Visual signals, which indicate the likely occurrence of a rewarding or aversive stimulus or indicate the context within which such stimuli may arrive, modulate activity in the superior colliculus and alter behaviour. However, such signals primarily originate in cortical and basal ganglia circuits, and evidence of direct signalling from midbrain dopamine neurons to superior colliculus is lacking. Instead, hypothalamic A13 dopamine neurons innervate the superior colliculus, and dopamine receptors are differentially expressed in the superior colliculus, with D1 receptors in superficial layers and D2 receptors in deep layers. However, it remains unknown if A13 dopamine neurons control behaviours through their effect on afferents within the superior colliculus. We propose that A13 dopamine neurons may play a critical role in processing information in the superior colliculus, modifying behavioural responses to visual cues, and propose some testable hypotheses regarding dopamine's effect on visual perception.
Collapse
Affiliation(s)
- Alexander Woolrych
- Department of Anatomy, School of Biomedical Sciences, Brain Health Research Centre, University of Otago, Dunedin, New Zealand
| | - Nicolas Vautrelle
- Department of Anatomy, School of Biomedical Sciences, Brain Health Research Centre, University of Otago, Dunedin, New Zealand
| | - John N J Reynolds
- Department of Anatomy, School of Biomedical Sciences, Brain Health Research Centre, University of Otago, Dunedin, New Zealand
| | - Louise C Parr-Brownlie
- Department of Anatomy, School of Biomedical Sciences, Brain Health Research Centre, University of Otago, Dunedin, New Zealand
| |
Collapse
|
25
|
Maith O, Schwarz A, Hamker FH. Optimal attention tuning in a neuro-computational model of the visual cortex-basal ganglia-prefrontal cortex loop. Neural Netw 2021; 142:534-547. [PMID: 34314999 DOI: 10.1016/j.neunet.2021.07.008] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2021] [Revised: 06/11/2021] [Accepted: 07/05/2021] [Indexed: 11/29/2022]
Abstract
Visual attention is widely considered a vital factor in the perception and analysis of a visual scene. Several studies explored the effects and mechanisms of top-down attention, but the mechanisms that determine the attentional signal are less explored. By developing a neuro-computational model of visual attention including the visual cortex-basal ganglia loop, we demonstrate how attentional alignment can evolve based on dopaminergic reward during a visual search task. Unlike most previous modeling studies of feature-based attention, we do not implement a manually predefined attention template. Dopamine-modulated covariance learning enable the basal ganglia to learn rewarded associations between the visual input and the attentional gain represented in the PFC of the model. Hence, the model shows human-like performance on a visual search task by optimally tuning the attention signal. In particular, similar as in humans, this reward-based tuning in the model leads to an attentional template that is not centered on the target feature, but a relevant feature deviating away from the target due to the presence of highly similar distractors. Further analyses of the model shows, attention is mainly guided by the signal-to-noise ratio between target and distractors.
Collapse
Affiliation(s)
- Oliver Maith
- Chemnitz University of Technology, Department of Computer Science, 09107 Chemnitz, Germany.
| | - Alex Schwarz
- Chemnitz University of Technology, Department of Computer Science, 09107 Chemnitz, Germany.
| | - Fred H Hamker
- Chemnitz University of Technology, Department of Computer Science, 09107 Chemnitz, Germany.
| |
Collapse
|
26
|
Mancini A, Ghiglieri V, Parnetti L, Calabresi P, Di Filippo M. Neuro-Immune Cross-Talk in the Striatum: From Basal Ganglia Physiology to Circuit Dysfunction. Front Immunol 2021; 12:644294. [PMID: 33953715 PMCID: PMC8091963 DOI: 10.3389/fimmu.2021.644294] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2020] [Accepted: 03/16/2021] [Indexed: 01/02/2023] Open
Abstract
The basal ganglia network is represented by an interconnected group of subcortical nuclei traditionally thought to play a crucial role in motor learning and movement execution. During the last decades, knowledge about basal ganglia physiology significantly evolved and this network is now considered as a key regulator of important cognitive and emotional processes. Accordingly, the disruption of basal ganglia network dynamics represents a crucial pathogenic factor in many neurological and psychiatric disorders. The striatum is the input station of the circuit. Thanks to the synaptic properties of striatal medium spiny neurons (MSNs) and their ability to express synaptic plasticity, the striatum exerts a fundamental integrative and filtering role in the basal ganglia network, influencing the functional output of the whole circuit. Although it is currently established that the immune system is able to regulate neuronal transmission and plasticity in specific cortical areas, the role played by immune molecules and immune/glial cells in the modulation of intra-striatal connections and basal ganglia activity still needs to be clarified. In this manuscript, we review the available evidence of immune-based regulation of synaptic activity in the striatum, also discussing how an abnormal immune activation in this region could be involved in the pathogenesis of inflammatory and degenerative central nervous system (CNS) diseases.
Collapse
Affiliation(s)
- Andrea Mancini
- Section of Neurology, Department of Medicine and Surgery, Università degli Studi di Perugia, Perugia, Italy
| | | | - Lucilla Parnetti
- Section of Neurology, Department of Medicine and Surgery, Università degli Studi di Perugia, Perugia, Italy
| | - Paolo Calabresi
- Section of Neurology, Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy.,Department of Neuroscience, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Massimiliano Di Filippo
- Section of Neurology, Department of Medicine and Surgery, Università degli Studi di Perugia, Perugia, Italy
| |
Collapse
|
27
|
Abstract
Traditional synaptic plasticity experiments and models depend on tight temporal correlations between pre- and postsynaptic activity. These tight temporal correlations, on the order of tens of milliseconds, are incompatible with significantly longer behavioral time scales, and as such might not be able to account for plasticity induced by behavior. Indeed, recent findings in hippocampus suggest that rapid, bidirectional synaptic plasticity which modifies place fields in CA1 operates at behavioral time scales. These experimental results suggest that presynaptic activity generates synaptic eligibility traces both for potentiation and depression, which last on the order of seconds. These traces can be converted to changes in synaptic efficacies by the activation of an instructive signal that depends on naturally occurring or experimentally induced plateau potentials. We have developed a simple mathematical model that is consistent with these observations. This model can be fully analyzed to find the fixed points of induced place fields and how these fixed points depend on system parameters such as the size and shape of presynaptic place fields, the animal's velocity during induction, and the parameters of the plasticity rule. We also make predictions about the convergence time to these fixed points, both for induced and pre-existing place fields.
Collapse
Affiliation(s)
- Ian Cone
- Department of Neurobiology and Anatomy, University of Texas Medical School, Houston, TX, United States
- Applied Physics Program, Rice University, Houston, TX, United States
| | - Harel Z. Shouval
- Department of Neurobiology and Anatomy, University of Texas Medical School, Houston, TX, United States
| |
Collapse
|
28
|
Lim DH, Yoon YJ, Her E, Huh S, Jung MW. Active maintenance of eligibility trace in rodent prefrontal cortex. Sci Rep 2020; 10:18860. [PMID: 33139778 DOI: 10.1038/s41598-020-75820-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2020] [Accepted: 09/29/2020] [Indexed: 12/05/2022] Open
Abstract
Even though persistent neural activity has been proposed as a mechanism for maintaining eligibility trace, direct empirical evidence for active maintenance of eligibility trace has been lacking. We recorded neuronal activity in the medial prefrontal cortex (mPFC) in rats performing a dynamic foraging task in which a choice must be remembered until its outcome on the timescale of seconds for correct credit assignment. We found that mPFC neurons maintain significant choice signals during the time period between action selection and choice outcome. We also found that neural signals for choice, outcome, and action value converge in the mPFC when choice outcome was revealed. Our results indicate that the mPFC maintains choice signals necessary for temporal credit assignment in the form of persistent neural activity in our task. They also suggest that the mPFC might update action value by combining actively maintained eligibility trace with action value and outcome signals.
Collapse
|
29
|
Abstract
We describe a neurobiologically informed computational model of phasic dopamine signaling to account for a wide range of findings, including many considered inconsistent with the simple reward prediction error (RPE) formalism. The central feature of this PVLV framework is a distinction between a primary value (PV) system for anticipating primary rewards (Unconditioned Stimuli [USs]), and a learned value (LV) system for learning about stimuli associated with such rewards (CSs). The LV system represents the amygdala, which drives phasic bursting in midbrain dopamine areas, while the PV system represents the ventral striatum, which drives shunting inhibition of dopamine for expected USs (via direct inhibitory projections) and phasic pausing for expected USs (via the lateral habenula). Our model accounts for data supporting the separability of these systems, including individual differences in CS-based (sign-tracking) versus US-based learning (goal-tracking). Both systems use competing opponent-processing pathways representing evidence for and against specific USs, which can explain data dissociating the processes involved in acquisition versus extinction conditioning. Further, opponent processing proved critical in accounting for the full range of conditioned inhibition phenomena, and the closely related paradigm of second-order conditioning. Finally, we show how additional separable pathways representing aversive USs, largely mirroring those for appetitive USs, also have important differences from the positive valence case, allowing the model to account for several important phenomena in aversive conditioning. Overall, accounting for all of these phenomena strongly constrains the model, thus providing a well-validated framework for understanding phasic dopamine signaling. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
Affiliation(s)
- Jessica A Mollick
- Department of Psychology and Neuroscience, University of Colorado Boulder
| | - Thomas E Hazy
- Department of Psychology and Neuroscience, University of Colorado Boulder
| | - Kai A Krueger
- Department of Psychology and Neuroscience, University of Colorado Boulder
| | - Ananta Nair
- Department of Psychology and Neuroscience, University of Colorado Boulder
| | - Prescott Mackie
- Department of Psychology and Neuroscience, University of Colorado Boulder
| | - Seth A Herd
- Department of Psychology and Neuroscience, University of Colorado Boulder
| | - Randall C O'Reilly
- Department of Psychology and Neuroscience, University of Colorado Boulder
| |
Collapse
|
30
|
Chiamulera C, Piva A, Abraham WC. Glutamate receptors and metaplasticity in addiction. Curr Opin Pharmacol 2020; 56:39-45. [PMID: 33128937 DOI: 10.1016/j.coph.2020.09.005] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2020] [Revised: 09/10/2020] [Accepted: 09/10/2020] [Indexed: 12/12/2022]
Abstract
Chronic drug use is a neuroadaptive disorder characterized by strong and persistent plasticity in the mesocorticolimbic reward system. Long-lasting effects of drugs of abuse rely on their ability to hijack glutamate receptor activity and long-term synaptic plasticity processes like long-term potentiation and depression. Importantly, metaplasticity-based modulation of synaptic plasticity contributes to durable neurotransmission changes in mesocorticolimbic pathways including the ventral tegmental area and the nucleus accumbens, causing 'maladaptive' drug memory and higher risk for drug-seeking relapse. On the other hand, drug-induced metaplasticity can make appetitive memories more malleable to modification, offering a potential target mechanism for intervention. Here we review the literature on the role of glutamate receptors in addiction-related metaplasticity phenomena.
Collapse
Affiliation(s)
- Cristiano Chiamulera
- Neuropsychopharmacology Lab, Section Pharmacology, Department Diagnostic & Public Health, University of Verona, Verona, Italy.
| | - Alessandro Piva
- Neuropsychopharmacology Lab, Section Pharmacology, Department Diagnostic & Public Health, University of Verona, Verona, Italy
| | - Wickliffe C Abraham
- Department of Psychology, Brain Health Research Centre, Brain Research New Zealand, University of Otago, Dunedin, New Zealand
| |
Collapse
|
31
|
|
32
|
Han S, Márquez-Gómez R, Woodman M, Ellender T. Histaminergic Control of Corticostriatal Synaptic Plasticity during Early Postnatal Development. J Neurosci 2020; 40:6557-6571. [PMID: 32709692 PMCID: PMC7486653 DOI: 10.1523/jneurosci.0740-20.2020] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2020] [Revised: 06/30/2020] [Accepted: 07/03/2020] [Indexed: 11/21/2022] Open
Abstract
A reduction in the synthesis of the neuromodulator histamine has been associated with Tourette's syndrome and obsessive-compulsive disorder. Symptoms of these disorders are thought to arise from a dysfunction or aberrant development ofcorticostriatal circuits. Here, we investigated how histamine affects developing corticostriatal circuits, both acutely and longer-term, during the first postnatal weeks, using patch-clamp and field recordings in mouse brain slices (C57Bl/6, male and female). Immunohistochemistry for histamine-containing axons reveals striatal histaminergic innervation by the second postnatal week, and qRT-PCR shows transcripts for H1, H2, and H3 histamine receptors in striatum from the first postnatal week onwards, with pronounced developmental increases in H3 receptor expression. Whole-cell patch-clamp recordings of striatal spiny projection neurons and histamine superfusion demonstrates expression of functional histamine receptors from the first postnatal week onwards, with histamine having diverse effects on their electrical properties, including depolarization of the membrane potential while simultaneously decreasing action potential output. Striatal field recordings and electrical stimulation of corticostriatal afferents revealed that histamine, acting at H3 receptors, negatively modulates corticostriatal synaptic transmission from the first postnatal week onwards. Last, we investigated effects of histamine on longer-term changes at developing corticostriatal synapses and show that histamine facilitates NMDA receptor-dependent LTP via H3 receptors during the second postnatal week, but inhibits synaptic plasticity at later developmental stages. Together, these results show that histamine acutely modulates developing striatal neurons and synapses and controls longer-term changes in developing corticostriatal circuits, thus providing insight into the possible etiology underlying neurodevelopmental disorders resulting from histamine dysregulation.SIGNIFICANCE STATEMENT Monogenic causes of neurologic disorders, although rare, can provide opportunities to both study and understand the brain. For example, a nonsense mutation in the coding gene for the histamine-synthesizing enzyme has been associated with Tourette's syndrome and obsessive-compulsive disorder, and dysfunction of corticostriatal circuits. Nevertheless, the etiology of these neurodevelopmental disorders and histamine's role in the development of corticostriatal circuits have remained understudied. Here we show that histamine is an active neuromodulator during the earliest periods of postnatal life and acts at developing striatal neurons and synapses. Crucially, we show that histamine permits NMDA receptor-dependent corticostriatal synaptic plasticity during an early critical period of postnatal development, which suggests that genetic or environmental perturbations of histamine levels can impact striatal development.
Collapse
Affiliation(s)
- Sungwon Han
- Department of Pharmacology, University of Oxford, OX1 3QT, Oxford, United Kingdom
| | | | - Myles Woodman
- Department of Pharmacology, University of Oxford, OX1 3QT, Oxford, United Kingdom
| | - Tommas Ellender
- Department of Pharmacology, University of Oxford, OX1 3QT, Oxford, United Kingdom
| |
Collapse
|
33
|
Urakubo H, Yagishita S, Kasai H, Ishii S. Signaling models for dopamine-dependent temporal contiguity in striatal synaptic plasticity. PLoS Comput Biol 2020; 16:e1008078. [PMID: 32701987 PMCID: PMC7402527 DOI: 10.1371/journal.pcbi.1008078] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Revised: 08/04/2020] [Accepted: 06/19/2020] [Indexed: 02/06/2023] Open
Abstract
Animals remember temporal links between their actions and subsequent rewards. We previously discovered a synaptic mechanism underlying such reward learning in D1 receptor (D1R)-expressing spiny projection neurons (D1 SPN) of the striatum. Dopamine (DA) bursts promote dendritic spine enlargement in a time window of only a few seconds after paired pre- and post-synaptic spiking (pre-post pairing), which is termed as reinforcement plasticity (RP). The previous study has also identified underlying signaling pathways; however, it still remains unclear how the signaling dynamics results in RP. In the present study, we first developed a computational model of signaling dynamics of D1 SPNs. The D1 RP model successfully reproduced experimentally observed protein kinase A (PKA) activity, including its critical time window. In this model, adenylate cyclase type 1 (AC1) in the spines/thin dendrites played a pivotal role as a coincidence detector against pre-post pairing and DA burst. In particular, pre-post pairing (Ca2+ signal) stimulated AC1 with a delay, and the Ca2+-stimulated AC1 was activated by the DA burst for the asymmetric time window. Moreover, the smallness of the spines/thin dendrites is crucial to the short time window for the PKA activity. We then developed a RP model for D2 SPNs, which also predicted the critical time window for RP that depended on the timing of pre-post pairing and phasic DA dip. AC1 worked for the coincidence detector in the D2 RP model as well. We further simulated the signaling pathway leading to Ca2+/calmodulin-dependent protein kinase II (CaMKII) activation and clarified the role of the downstream molecules of AC1 as the integrators that turn transient input signals into persistent spine enlargement. Finally, we discuss how such timing windows guide animals' reward learning.
Collapse
Affiliation(s)
- Hidetoshi Urakubo
- Integrated Systems Biology Laboratory, Department of Systems Science, Graduate School of Informatics, Kyoto University, Sakyo-ku, Kyoto, Japan
- * E-mail:
| | - Sho Yagishita
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, University of Tokyo, Bunkyo-ku, Tokyo, Japan
- International Research Center for Neurointelligence (WPI-IRCN), University of Tokyo Institutes for Advanced Study (UTIAS), Tokyo, Japan
| | - Haruo Kasai
- Laboratory of Structural Physiology, Center for Disease Biology and Integrative Medicine, Faculty of Medicine, University of Tokyo, Bunkyo-ku, Tokyo, Japan
- International Research Center for Neurointelligence (WPI-IRCN), University of Tokyo Institutes for Advanced Study (UTIAS), Tokyo, Japan
| | - Shin Ishii
- Integrated Systems Biology Laboratory, Department of Systems Science, Graduate School of Informatics, Kyoto University, Sakyo-ku, Kyoto, Japan
- International Research Center for Neurointelligence (WPI-IRCN), University of Tokyo Institutes for Advanced Study (UTIAS), Tokyo, Japan
| |
Collapse
|
34
|
Rubin JE, Vich C, Clapp M, Noneman K, Verstynen T. The credit assignment problem in cortico‐basal ganglia‐thalamic networks: A review, a problem and a possible solution. Eur J Neurosci 2020; 53:2234-2253. [DOI: 10.1111/ejn.14745] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2020] [Revised: 03/23/2020] [Accepted: 03/25/2020] [Indexed: 12/21/2022]
Affiliation(s)
- Jonathan E. Rubin
- Department of Mathematics Center for the Neural Basis of Cognition University of Pittsburgh Pittsburgh PA USA
| | - Catalina Vich
- Department de Matemàtiques i Informàtica Institute of Applied Computing and Community Code Universitat de les Illes Balears Palma Spain
| | - Matthew Clapp
- Carnegie Mellon Neuroscience Institute Carnegie Mellon University Pittsburgh PA USA
| | - Kendra Noneman
- Micron School of Materials Science and Engineering Boise State University Boise ID USA
| | - Timothy Verstynen
- Carnegie Mellon Neuroscience Institute Carnegie Mellon University Pittsburgh PA USA
- Department of Psychology Center for the Neural Basis of Cognition Carnegie Mellon University Pittsburgh PA USA
| |
Collapse
|
35
|
Abstract
Plasticity within the neuronal networks of the brain underlies the ability to learn and retain new information. The initial discovery of synaptic plasticity occurred by measuring synaptic strength in vivo, applying external stimulation and observing an increase in synaptic strength termed long-term potentiation (LTP). Many of the molecular pathways involved in LTP and other forms of synaptic plasticity were subsequently uncovered in vitro. Over the last few decades, technological advances in recording and imaging in live animals have seen many of these molecular mechanisms confirmed in vivo, including structural changes both pre- and postsynaptically, changes in synaptic strength, and changes in neuronal excitability. A well-studied aspect of neuronal plasticity is the capacity of the brain to adapt to its environment, gained by comparing the brains of deprived and experienced animals in vivo, and in direct response to sensory stimuli. Multiple in vivo studies have also strongly linked plastic changes to memory by interfering with the expression of plasticity and by manipulating memory engrams. Plasticity in vivo also occurs in the absence of any form of external stimulation, i.e., during spontaneous network activity occurring with brain development. However, there is still much to learn about how plasticity is induced during natural learning and how this is altered in neurological disorders.
Collapse
Affiliation(s)
- Juliette E Cheyne
- Department of Physiology and Centre for Brain Research, University of Auckland, Auckland, New Zealand
| | - Johanna M Montgomery
- Department of Physiology and Centre for Brain Research, University of Auckland, Auckland, New Zealand
| |
Collapse
|
36
|
Abstract
Synaptic plasticity, the activity-dependent change in neuronal connection strength, has long been considered an important component of learning and memory. Computational and engineering work corroborate the power of learning through the directed adjustment of connection weights. Here we review the fundamental elements of four broadly categorized forms of synaptic plasticity and discuss their functional capabilities and limitations. Although standard, correlation-based, Hebbian synaptic plasticity has been the primary focus of neuroscientists for decades, it is inherently limited. Three-factor plasticity rules supplement Hebbian forms with neuromodulation and eligibility traces, while true supervised types go even further by adding objectives and instructive signals. Finally, a recently discovered hippocampal form of synaptic plasticity combines the above elements, while leaving behind the primary Hebbian requirement. We suggest that the effort to determine the neural basis of adaptive behavior could benefit from renewed experimental and theoretical investigation of more powerful directed types of synaptic plasticity.
Collapse
Affiliation(s)
- Jeffrey C Magee
- Department of Neuroscience and Howard Hughes Medical Institute, Baylor College of Medicine, Houston, Texas 77030, USA;
| | - Christine Grienberger
- Department of Neuroscience and Howard Hughes Medical Institute, Baylor College of Medicine, Houston, Texas 77030, USA;
| |
Collapse
|
37
|
Mulcahy G, Atwood B, Kuznetsov A. Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states. PLoS One 2020; 15:e0228081. [PMID: 32040519 PMCID: PMC7010262 DOI: 10.1371/journal.pone.0228081] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2019] [Accepted: 01/07/2020] [Indexed: 01/06/2023] Open
Abstract
The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states.
Collapse
Affiliation(s)
- Garrett Mulcahy
- Department of Mathematics, Purdue University, West Lafayette, Indiana, United States of America
| | - Brady Atwood
- Departments of Psychiatry and Pharmacology & Toxicology, IUSM, Indianapolis, Indiana, United States of America
- Indiana Alcohol Research Center, IUSM, Indianapolis, Indiana, United States of America
| | - Alexey Kuznetsov
- Indiana Alcohol Research Center, IUSM, Indianapolis, Indiana, United States of America
- Department of Mathematical Sciences, IUPUI, Indianapolis, Indiana, United States of America
| |
Collapse
|
38
|
Morera-Herreras T, Gioanni Y, Perez S, Vignoud G, Venance L. Environmental enrichment shapes striatal spike-timing-dependent plasticity in vivo. Sci Rep 2019; 9:19451. [PMID: 31857605 PMCID: PMC6923403 DOI: 10.1038/s41598-019-55842-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2019] [Accepted: 11/27/2019] [Indexed: 01/18/2023] Open
Abstract
Behavioural experience, such as environmental enrichment (EE), induces long-term effects on learning and memory. Learning can be assessed with the Hebbian paradigm, such as spike-timing-dependent plasticity (STDP), which relies on the timing of neuronal activity on either side of the synapse. Although EE is known to control neuronal excitability and consequently spike timing, whether EE shapes STDP remains unknown. Here, using in vivo long-duration intracellular recordings at the corticostriatal synapses we show that EE promotes asymmetric anti-Hebbian STDP, i.e. spike-timing-dependent-potentiation (tLTP) for post-pre pairings and spike-timing-dependent-depression (tLTD) for pre-post pairings, whereas animals grown in standard housing show mainly tLTD and a high failure rate of plasticity. Indeed, in adult rats grown in standard conditions, we observed unidirectional plasticity (mainly symmetric anti-Hebbian tLTD) within a large temporal window (~200 ms). However, rats grown for two months in EE displayed a bidirectional STDP (tLTP and tLTD depending on spike timing) in a more restricted temporal window (~100 ms) with low failure rate of plasticity. We also found that the effects of EE on STDP characteristics are influenced by the anaesthesia status: the deeper the anaesthesia, the higher the absence of plasticity. These findings establish a central role for EE and the anaesthetic regime in shaping in vivo, a synaptic Hebbian learning rule such as STDP.
Collapse
Affiliation(s)
- Teresa Morera-Herreras
- Team Dynamic and Pathophysiology of Neuronal Networks, Center for Interdisciplinary Research in Biology, College de France, CNRS UMR7241/INSERM U1050, MemoLife Labex, Paris, France
- Department of Pharmacology, Faculty of Medicine and Nursing, University of the Basque Country (UPV/EHU), 48940, Leioa, Bizkaia, Spain
- Neurodegenerative Diseases Group, BioCruces Bizkaia Health Research Institute, 48903, Barakaldo, Bizkaia, Spain
| | - Yves Gioanni
- Team Dynamic and Pathophysiology of Neuronal Networks, Center for Interdisciplinary Research in Biology, College de France, CNRS UMR7241/INSERM U1050, MemoLife Labex, Paris, France
| | - Sylvie Perez
- Team Dynamic and Pathophysiology of Neuronal Networks, Center for Interdisciplinary Research in Biology, College de France, CNRS UMR7241/INSERM U1050, MemoLife Labex, Paris, France
| | - Gaetan Vignoud
- Team Dynamic and Pathophysiology of Neuronal Networks, Center for Interdisciplinary Research in Biology, College de France, CNRS UMR7241/INSERM U1050, MemoLife Labex, Paris, France
| | - Laurent Venance
- Team Dynamic and Pathophysiology of Neuronal Networks, Center for Interdisciplinary Research in Biology, College de France, CNRS UMR7241/INSERM U1050, MemoLife Labex, Paris, France.
| |
Collapse
|
39
|
Lehmann MP, Xu HA, Liakoni V, Herzog MH, Gerstner W, Preuschoff K. One-shot learning and behavioral eligibility traces in sequential decision making. eLife 2019; 8:e47463. [PMID: 31709980 PMCID: PMC6897511 DOI: 10.7554/elife.47463] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Accepted: 11/01/2019] [Indexed: 11/13/2022] Open
Abstract
In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.
Collapse
Affiliation(s)
- Marco P Lehmann
- Brain-Mind-Institute, School of Life SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
- School of Computer and Communication SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
| | - He A Xu
- Laboratory of Psychophysics, School of Life SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
| | - Vasiliki Liakoni
- Brain-Mind-Institute, School of Life SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
- School of Computer and Communication SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
| | - Michael H Herzog
- Laboratory of Psychophysics, School of Life SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
| | - Wulfram Gerstner
- Brain-Mind-Institute, School of Life SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
- School of Computer and Communication SciencesÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland
| | - Kerstin Preuschoff
- Swiss Center for Affective Sciences, University of GenevaGenevaSwitzerland
| |
Collapse
|
40
|
Bruce NJ, Narzi D, Trpevski D, van Keulen SC, Nair AG, Röthlisberger U, Wade RC, Carloni P, Hellgren Kotaleski J. Regulation of adenylyl cyclase 5 in striatal neurons confers the ability to detect coincident neuromodulatory signals. PLoS Comput Biol 2019; 15:e1007382. [PMID: 31665146 PMCID: PMC6821081 DOI: 10.1371/journal.pcbi.1007382] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Accepted: 09/05/2019] [Indexed: 02/04/2023] Open
Abstract
Long-term potentiation and depression of synaptic activity in response to stimuli is a key factor in reinforcement learning. Strengthening of the corticostriatal synapses depends on the second messenger cAMP, whose synthesis is catalysed by the enzyme adenylyl cyclase 5 (AC5), which is itself regulated by the stimulatory Gαolf and inhibitory Gαi proteins. AC isoforms have been suggested to act as coincidence detectors, promoting cellular responses only when convergent regulatory signals occur close in time. However, the mechanism for this is currently unclear, and seems to lie in their diverse regulation patterns. Despite attempts to isolate the ternary complex, it is not known if Gαolf and Gαi can bind to AC5 simultaneously, nor what activity the complex would have. Using protein structure-based molecular dynamics simulations, we show that this complex is stable and inactive. These simulations, along with Brownian dynamics simulations to estimate protein association rates constants, constrain a kinetic model that shows that the presence of this ternary inactive complex is crucial for AC5’s ability to detect coincident signals, producing a synergistic increase in cAMP. These results reveal some of the prerequisites for corticostriatal synaptic plasticity, and explain recent experimental data on cAMP concentrations following receptor activation. Moreover, they provide insights into the regulatory mechanisms that control signal processing by different AC isoforms. Adenylyl cyclases (ACs) are enzymes that can translate extracellular signals into the intracellular molecule cAMP, which is thus a 2nd messenger of extracellular events. The brain expresses nine membrane-bound AC variants, and AC5 is the dominant form in the striatum. The striatum is the input stage of the basal ganglia, a brain structure involved in reward learning, i.e. the learning of behaviors that lead to rewarding stimuli (such as food, water, sugar, etc). During reward learning, cAMP production is crucial for strengthening the synapses from cortical neurons onto the striatal principal neurons, and its formation is dependent on several neuromodulatory systems such as dopamine and acetylcholine. It is, however, not understood how AC5 is activated by transient (subsecond) changes in the neuromodulatory signals. Here we combine several computational tools, from molecular dynamics and Brownian dynamics simulations to bioinformatics approaches, to inform and constrain a kinetic model of the AC5-dependent signaling system. We use this model to show how the specific molecular properties of AC5 can detect particular combinations of co-occuring transient changes in the neuromodulatory signals which thus result in a supralinear/synergistic cAMP production. Our results also provide insights into the computational capabilities of the different AC isoforms.
Collapse
Affiliation(s)
- Neil J. Bruce
- Molecular and Cellular Modeling Group, Heidelberg Institute for Theoretical Studies (HITS), Schloss-Heidelberg, Germany
| | - Daniele Narzi
- Institut des Sciences et Ingénierie Chimiques, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
| | - Daniel Trpevski
- Science for Life Laboratory, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
| | - Siri C. van Keulen
- Institut des Sciences et Ingénierie Chimiques, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
- Department of Computer Science, Stanford University, Stanford, California, United States of America
| | - Anu G. Nair
- Institute of Molecular Life Sciences, University of Zurich, Zurich, Switzerland
| | - Ursula Röthlisberger
- Institut des Sciences et Ingénierie Chimiques, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
- * E-mail: (UR); (RCW); (PC); (JHK)
| | - Rebecca C. Wade
- Molecular and Cellular Modeling Group, Heidelberg Institute for Theoretical Studies (HITS), Schloss-Heidelberg, Germany
- Center for Molecular Biology (ZMBH), DKFZ-ZMBH Alliance, Heidelberg University, Heidelberg, Germany
- Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, Heidelberg, Germany
- * E-mail: (UR); (RCW); (PC); (JHK)
| | - Paolo Carloni
- Department of Physics and Department of Neurobiology, RWTH Aachen University,Aachen, Germany
- Institute for Neuroscience and Medicine (INM)-11, Forschungszentrum Jülich, Jülich, Germany
- Institute of Neuroscience and Medicine (INM-9), Forschungszentrum Jülich, Jülich, Germany
- Institute for Advanced Simulation (IAS-5), Forschungszentrum Jülich, Jülich, Germany
- * E-mail: (UR); (RCW); (PC); (JHK)
| | - Jeanette Hellgren Kotaleski
- Science for Life Laboratory, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden
- Department of Neuroscience, Karolinska Institutet, Solna, Sweden
- * E-mail: (UR); (RCW); (PC); (JHK)
| |
Collapse
|
41
|
Rusu SI, Pennartz CMA. Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems. Hippocampus 2019; 30:73-98. [PMID: 31617622 PMCID: PMC6972576 DOI: 10.1002/hipo.23167] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2018] [Revised: 09/09/2019] [Accepted: 09/11/2019] [Indexed: 01/05/2023]
Abstract
This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.
Collapse
Affiliation(s)
- Silviu I Rusu
- Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, The Netherlands.,Research Priority Program Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| | - Cyriel M A Pennartz
- Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, The Netherlands.,Research Priority Program Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
42
|
Vega-Villar M, Horvitz JC, Nicola SM. NMDA receptor-dependent plasticity in the nucleus accumbens connects reward-predictive cues to approach responses. Nat Commun 2019; 10:4429. [PMID: 31562332 PMCID: PMC6764993 DOI: 10.1038/s41467-019-12387-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2019] [Accepted: 09/09/2019] [Indexed: 12/13/2022] Open
Abstract
Learning associations between environmental cues and rewards is a fundamental adaptive function. Via such learning, reward-predictive cues come to activate approach to locations where reward is available. The nucleus accumbens (NAc) is essential for cued approach behavior in trained subjects, and cue-evoked excitations in NAc neurons are critical for the expression of this behavior. Excitatory synapses within the NAc undergo synaptic plasticity that presumably contributes to cued approach acquisition, but a direct link between synaptic plasticity within the NAc and the development of cue-evoked neural activity during learning has not been established. Here we show that, with repeated cue-reward pairings, cue-evoked excitations in the NAc emerge and grow in the trials prior to the detectable expression of cued approach behavior. We demonstrate that the growth of these signals requires NMDA receptor-dependent plasticity within the NAc, revealing a neural mechanism by which the NAc participates in learning of conditioned reward-seeking behaviors. Conditioned stimuli elicit phasic changes in nucleus accumbens (NAc) firing that invigorate approach responses to predicted rewards. Here the authors show that NAc neurons acquire cue-evoked responses during learning as a result of excitatory plasticity within the NAc.
Collapse
Affiliation(s)
- Mercedes Vega-Villar
- Department of Psychology, The Graduate Center, City University of New York, 365 Fifth Avenue, 6th Floor, New York, NY, 10016, USA.,Department of Psychology, City College of New York, City University of New York, 160 Convent Avenue, NAC 7/120, New York, NY, 10031, USA.,Department of Neuroscience, Albert Einstein College of Medicine, Jack and Pearl Resnick Campus, 1300 Morris Park Avenue, Forchheimer Building, Room-111, Bronx, NY, 10461, USA
| | - Jon C Horvitz
- Department of Psychology, City College of New York, City University of New York, 160 Convent Avenue, NAC 7/120, New York, NY, 10031, USA
| | - Saleem M Nicola
- Department of Neuroscience, Albert Einstein College of Medicine, Jack and Pearl Resnick Campus, 1300 Morris Park Avenue, Forchheimer Building, Room-111, Bronx, NY, 10461, USA. .,Department of Psychiatry, Albert Einstein College of Medicine, Bronx, NY, 10461, USA.
| |
Collapse
|
43
|
Brzosko Z, Mierau SB, Paulsen O. Neuromodulation of Spike-Timing-Dependent Plasticity: Past, Present, and Future. Neuron 2019; 103:563-581. [DOI: 10.1016/j.neuron.2019.05.041] [Citation(s) in RCA: 91] [Impact Index Per Article: 18.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Revised: 05/20/2019] [Accepted: 05/24/2019] [Indexed: 12/31/2022]
|
44
|
Hamel R, Côté K, Matte A, Lepage JF, Bernier PM. Rewards interact with repetition-dependent learning to enhance long-term retention of motor memories. Ann N Y Acad Sci 2019; 1452:34-51. [PMID: 31294872 DOI: 10.1111/nyas.14171] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2019] [Revised: 04/26/2019] [Accepted: 05/29/2019] [Indexed: 11/28/2022]
Abstract
The combination of behavioral experiences that enhance long-term retention remains largely unknown. Informed by neurophysiological lines of work, this study tested the hypothesis that performance-contingent monetary rewards potentiate repetition-dependent forms of learning, as induced by extensive practice at asymptote, to enhance long-term retention of motor memories. To this end, six groups of 14 participants (n = 84) acquired novel motor behaviors by adapting to a gradual visuomotor rotation while these factors were manipulated. Retention was assessed 24 h later. While all groups similarly acquired the novel motor behaviors, results from the retention session revealed an interaction indicating that rewards enhanced long-term retention, but only when practice was extended to asymptote. Specifically, the interaction indicated that this effect selectively occurred when rewards were intermittently available (i.e., 50%), but not when they were absent (i.e., 0%) or continuously available (i.e., 100%) during acquisition. This suggests that the influence of rewards on extensive practice and long-term retention is nonlinear, as continuous rewards did not further enhance retention as compared with intermittent rewards. One possibility is that rewards' intermittent availability allows to maintain their subjective value during acquisition, which may be key to potentiate long-term retention.
Collapse
Affiliation(s)
- Raphaël Hamel
- Département de Pédiatrie, Faculté de Médecine et des Sciences de la Santé, Université de Sherbrooke, Sherbrooke, Québec, Canada.,Département de Kinanthropologie, Faculté des Sciences de l'Activité Physique, Université de Sherbrooke, Sherbrooke, Québec, Canada
| | - Kathleen Côté
- Département de Pédiatrie, Faculté de Médecine et des Sciences de la Santé, Université de Sherbrooke, Sherbrooke, Québec, Canada
| | - Alexia Matte
- Département de Pédiatrie, Faculté de Médecine et des Sciences de la Santé, Université de Sherbrooke, Sherbrooke, Québec, Canada
| | - Jean-François Lepage
- Département de Pédiatrie, Faculté de Médecine et des Sciences de la Santé, Université de Sherbrooke, Sherbrooke, Québec, Canada
| | - Pierre-Michel Bernier
- Département de Kinanthropologie, Faculté des Sciences de l'Activité Physique, Université de Sherbrooke, Sherbrooke, Québec, Canada
| |
Collapse
|
45
|
Smith‐Dijak AI, Sepers MD, Raymond LA. Alterations in synaptic function and plasticity in Huntington disease. J Neurochem 2019; 150:346-365. [DOI: 10.1111/jnc.14723] [Citation(s) in RCA: 64] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2018] [Revised: 03/28/2019] [Accepted: 05/08/2019] [Indexed: 12/27/2022]
Affiliation(s)
- Amy I. Smith‐Dijak
- Graduate Program in Neuroscience the University of British Columbia Vancouver British Columbia Canada
- Department of Psychiatry and Djavad Mowafaghian Centre for Brain Health the University of British Columbia Vancouver British Columbia Canada
| | - Marja D. Sepers
- Department of Psychiatry and Djavad Mowafaghian Centre for Brain Health the University of British Columbia Vancouver British Columbia Canada
| | - Lynn A. Raymond
- Department of Psychiatry and Djavad Mowafaghian Centre for Brain Health the University of British Columbia Vancouver British Columbia Canada
| |
Collapse
|
46
|
Abstract
A set of sub-cortical nuclei called basal ganglia is critical for learning the values of actions. The basal ganglia include two pathways, which have been associated with approach and avoid behavior respectively and are differentially modulated by dopamine projections from the midbrain. Inspired by the influential opponent actor learning model, we demonstrate that, under certain circumstances, these pathways may represent learned estimates of the positive and negative consequences (payoffs and costs) of individual actions. In the model, the level of dopamine activity encodes the motivational state and controls to what extent payoffs and costs enter the overall evaluation of actions. We show that a set of previously proposed plasticity rules is suitable to extract payoffs and costs from a prediction error signal if they occur at different moments in time. For those plasticity rules, successful learning requires differential effects of positive and negative outcome prediction errors on the two pathways and a weak decay of synaptic weights over trials. We also confirm through simulations that the model reproduces drug-induced changes of willingness to work, as observed in classical experiments with the D2-antagonist haloperidol. The basal ganglia are structures underneath the surface of the vertebrate brain, associated with error-driven learning. Much is known about the anatomical and biological features of the basal ganglia; scientists now try to understand the algorithms implemented by these structures. Numerous models aspire to capture the learning functionality, but many of them only cover some specific aspect of the algorithm. Instead of further adding to that pool of partial models, we unify two existing ones—one which captures what the basal ganglia learn, and one that describes the learning mechanism itself. The first model suggests that the basal ganglia weigh positive against negative consequences of actions according to the motivational state. It hints how payoff and cost might be represented, but does not explain how those representations arise. The other model consists of biologically plausible plasticity rules, which describe how learning takes place, but not how the brain makes use of what is learned. We show that the two theories are compatible. Together, they form a model of learning and decision making that integrates the motivational state as well as the learned payoffs and costs of opportunities.
Collapse
Affiliation(s)
- Moritz Möller
- MRC Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Rafal Bogacz
- MRC Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
- * E-mail:
| |
Collapse
|
47
|
Deperrois N, Moiseeva V, Gutkin B. Minimal Circuit Model of Reward Prediction Error Computations and Effects of Nicotinic Modulations. Front Neural Circuits 2019; 12:116. [PMID: 30687021 PMCID: PMC6336136 DOI: 10.3389/fncir.2018.00116] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2018] [Accepted: 12/14/2018] [Indexed: 11/29/2022] Open
Abstract
Dopamine (DA) neurons in the ventral tegmental area (VTA) are thought to encode reward prediction errors (RPE) by comparing actual and expected rewards. In recent years, much work has been done to identify how the brain uses and computes this signal. While several lines of evidence suggest the interplay of the DA and the inhibitory interneurons in the VTA implements the RPE computation, it still remains unclear how the DA neurons learn key quantities, for example the amplitude and the timing of primary rewards during conditioning tasks. Furthermore, endogenous acetylcholine and exogenous nicotine, also likely affect these computations by acting on both VTA DA and GABA (γ -aminobutyric acid) neurons via nicotinic-acetylcholine receptors (nAChRs). To explore the potential circuit-level mechanisms for RPE computations during classical-conditioning tasks, we developed a minimal computational model of the VTA circuitry. The model was designed to account for several reward-related properties of VTA afferents and recent findings on VTA GABA neuron dynamics during conditioning. With our minimal model, we showed that the RPE can be learned by a two-speed process computing reward timing and magnitude. By including models of nAChR-mediated currents in the VTA DA-GABA circuit, we showed that nicotine should reduce the acetylcholine action on the VTA GABA neurons by receptor desensitization and potentially boost DA responses to reward-related signals in a non-trivial manner. Together, our results delineate the mechanisms by which RPE are computed in the brain, and suggest a hypothesis on nicotine-mediated effects on reward-related perception and decision-making.
Collapse
Affiliation(s)
- Nicolas Deperrois
- Group for Neural Theory, LNC2 INSERM U960, DEC, École Normale Supérieure PSL University, Paris, France
| | - Victoria Moiseeva
- Center for Cognition and Decision Making, Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
| | - Boris Gutkin
- Group for Neural Theory, LNC2 INSERM U960, DEC, École Normale Supérieure PSL University, Paris, France.,Center for Cognition and Decision Making, Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
| |
Collapse
|
48
|
Morita K, Kawaguchi Y. A Dual Role Hypothesis of the Cortico-Basal-Ganglia Pathways: Opponency and Temporal Difference Through Dopamine and Adenosine. Front Neural Circuits 2019; 12:111. [PMID: 30687019 PMCID: PMC6338031 DOI: 10.3389/fncir.2018.00111] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2018] [Accepted: 11/29/2018] [Indexed: 01/07/2023] Open
Abstract
The hypothesis that the basal-ganglia direct and indirect pathways represent goodness (or benefit) and badness (or cost) of options, respectively, explains a wide range of phenomena. However, this hypothesis, named the Opponent Actor Learning (OpAL), still has limitations. Structurally, the OpAL model does not incorporate differentiation of the two types of cortical inputs to the basal-ganglia pathways received from intratelencephalic (IT) and pyramidal-tract (PT) neurons. Functionally, the OpAL model does not describe the temporal-difference (TD)-type reward-prediction-error (RPE), nor explains how RPE is calculated in the circuitry connecting to the DA neurons. In fact, there is a different hypothesis on the basal-ganglia pathways and DA, named the Cortico-Striatal-Temporal-Difference (CS-TD) model. The CS-TD model differentiates the IT and PT inputs, describes the TD-type RPE, and explains how TD-RPE is calculated. However, a critical difficulty in this model lies in its assumption that DA induces the same direction of plasticity in both direct and indirect pathways, which apparently contradicts the experimentally observed opposite effects of DA on these pathways. Here, we propose a new hypothesis that integrates the OpAL and CS-TD models. Specifically, we propose that the IT-basal-ganglia pathways represent goodness/badness of current options while the PT-indirect pathway represents the overall value of the previously chosen option, and both of these have influence on the DA neurons, through the basal-ganglia output, so that a variant of TD-RPE is calculated. A key assumption is that opposite directions of plasticity are induced upon phasic activation of DA neurons in the IT-indirect pathway and PT-indirect pathway because of different profiles of IT and PT inputs. Specifically, at PT→indirect-pathway-medium-spiny-neuron (iMSN) synapses, sustained glutamatergic inputs generate rich adenosine, which allosterically prevents DA-D2 receptor signaling and instead favors adenosine-A2A receptor signaling. Then, phasic DA-induced phasic adenosine, which reflects TD-RPE, causes long-term synaptic potentiation. In contrast, at IT→iMSN synapses where adenosine is scarce, phasic DA causes long-term synaptic depression via D2 receptor signaling. This new Opponency and Temporal-Difference (OTD) model provides unique predictions, part of which is potentially in line with recently reported activity patterns of neurons in the globus pallidus externus on the indirect pathway.
Collapse
Affiliation(s)
- Kenji Morita
- Physical and Health Education, Graduate School of Education, The University of Tokyo, Tokyo, Japan.,International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo Institutes for Advanced Study, Tokyo, Japan
| | - Yasuo Kawaguchi
- Division of Cerebral Circuitry, National Institute for Physiological Sciences, Okazaki, Japan.,Department of Physiological Sciences, Graduate University for Advanced Studies, Okazaki, Japan
| |
Collapse
|
49
|
Perrin E, Venance L. Bridging the gap between striatal plasticity and learning. Curr Opin Neurobiol 2018; 54:104-112. [PMID: 30321866 DOI: 10.1016/j.conb.2018.09.007] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2018] [Revised: 09/19/2018] [Accepted: 09/25/2018] [Indexed: 12/28/2022]
Abstract
The striatum, the main input nucleus of the basal ganglia, controls goal-directed behavior and procedural learning. Striatal projection neurons integrate glutamatergic inputs from cortex and thalamus together with neuromodulatory systems, and are subjected to plasticity. Striatal projection neurons exhibit bidirectional plasticity (LTP and LTD) when exposed to Hebbian paradigms. Importantly, correlative and even causal links between procedural learning and striatal plasticity have recently been shown. This short review summarizes the current view on striatal plasticity (with a focus on spike-timing-dependent plasticity), recent studies aiming at bridging in vivo skill acquisition and striatal plasticity, the temporal credit-assignment problem, and the gaps that remain to be filled.
Collapse
Affiliation(s)
- Elodie Perrin
- Center for Interdisciplinary Research in Biology, Collège de France, INSERM U1050, CNRS UMR7241, Labex Memolife, 75005 Paris, France; Université Pierre et Marie Curie, ED 158, Paris, France
| | - Laurent Venance
- Center for Interdisciplinary Research in Biology, Collège de France, INSERM U1050, CNRS UMR7241, Labex Memolife, 75005 Paris, France; Université Pierre et Marie Curie, ED 158, Paris, France.
| |
Collapse
|
50
|
Xu H, Perez S, Cornil A, Detraux B, Prokin I, Cui Y, Degos B, Berry H, de Kerchove d'Exaerde A, Venance L. Dopamine-endocannabinoid interactions mediate spike-timing-dependent potentiation in the striatum. Nat Commun 2018; 9:4118. [PMID: 30297767 DOI: 10.1038/s41467-018-06409-5] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2016] [Accepted: 08/30/2018] [Indexed: 01/01/2023] Open
Abstract
Dopamine modulates striatal synaptic plasticity, a key substrate for action selection and procedural learning. Thus, characterizing the repertoire of activity-dependent plasticity in striatum and its dependence on dopamine is of crucial importance. We recently unraveled a striatal spike-timing-dependent long-term potentiation (tLTP) mediated by endocannabinoids (eCBs) and induced with few spikes (~5–15). Whether this eCB-tLTP interacts with the dopaminergic system remains to be investigated. Here, we report that eCB-tLTP is impaired in a rodent model of Parkinson’s disease and rescued by L-DOPA. Dopamine controls eCB-tLTP via dopamine type-2 receptors (D2R) located presynaptically in cortical terminals. Dopamine–endocannabinoid interactions via D2R are required for the emergence of tLTP in response to few coincident pre- and post-synaptic spikes and control eCB-plasticity by modulating the long-term potentiation (LTP)/depression (LTD) thresholds. While usually considered as a depressing synaptic function, our results show that eCBs in the presence of dopamine constitute a versatile system underlying bidirectional plasticity implicated in basal ganglia pathophysiology. Dopamine tightly regulates plasticity at corticostriatal synapses. Here, the authors report that endocannabinoid dependent LTP induced with few spikes in the striatum is impaired in a rodent model of Parkinson’s disease, requires dopamine through presynaptic D2 receptors located on corticostriatal inputs.
Collapse
|