A critical time window for dopamine actions on the structural plasticity of dendritic spines

Cellular bases for reward-related dopamine actions

Dopamine neurons exhibit transient increases and decreases in their firing rate upon reward and punishment for learning. This bidirectional modulation of dopamine dynamics occurs on the order of hundreds of milliseconds, and it is sensitively detected to reinforce the preceding sensorimotor events. These observations indicate that the mechanisms of dopamine detection at the projection sites are of remarkable precision, both in time and concentration. A major target of dopamine projection is the striatum, including the ventral region of the nucleus accumbens, which mainly comprises dopamine D1 and D2 receptor (D1R and D2R)-expressing spiny projection neurons. Although the involvement of D1R and D2R in dopamine-dependent learning has been suggested, the exact cellular bases for detecting transient dopamine signaling remain unclear. This review discusses recent cellular studies on the novel synaptic mechanisms for detecting dopamine transient signals associated with learning. Analyses of behavior based on these mechanisms have further revealed new behavioral aspects that are closely associated with these synaptic mechanisms. Thus, it is gradually possible to mechanistically explain behavioral learning via synaptic and cellular bases in rodents.

Unraveling the dynamics of dopamine release and its actions on target cells

The neuromodulator dopamine (DA) is essential for regulating learning, motivation, and movement. Despite its importance, however, the mechanisms by which DA influences the activity of target cells to alter behavior remain poorly understood. In this review, we describe recent methodological advances that are helping to overcome challenges that have historically hindered the field. We discuss how the employment of these methods is shedding light on the complex dynamics of extracellular DA in the brain, as well as how DA signaling alters the electrical, biochemical, and population activity of target neurons in vivo. These developments are generating novel hypotheses about the mechanisms through which DA release modifies behavior.

Transient cAMP production drives rapid and sustained spiking in brainstem parabrachial neurons to suppress feeding

Brief stimuli can trigger longer lasting brain states. G protein-coupled receptors (GPCRs) could help sustain such states by coupling slow-timescale molecular signals to neuronal excitability. Brainstem parabrachial nucleus glutamatergic neurons (PBN Glut ) regulate sustained brain states such as pain, and express G s -coupled GPCRs that increase cAMP signaling. We asked whether cAMP directly influences PBN Glut excitability and behavior. Both brief tail shocks and brief optogenetic stimulation of cAMP production in PBN Glut neurons drove minutes-long suppression of feeding. This suppression matched the duration of prolonged elevations in cAMP, Protein Kinase A (PKA), and calcium activity in vivo and in vitro. Shortening this elevation in cAMP reduced the duration of feeding suppression following tail shocks. cAMP elevations in PBN Glut neurons rapidly lead to sustained increases in action potential firing via PKA-dependent mechanisms. Thus, molecular signaling in PBN Glut neurons helps prolong neural activity and behavioral states evoked by brief, salient bodily stimuli.

Striatal dopamine supports reward expectation and learning: A simultaneous PET/fMRI study

Converging evidence from both human neuroimaging and animal studies has supported a model of mesolimbic processing underlying reward learning behaviors, based on the computation of reward prediction errors. However, competing evidence supports human dopamine signaling in the basal ganglia as also contributing to the generation of higher order learning heuristics. Here, we present data from a large (N = 81, 18-30yo), multi-modal neuroimaging study using simultaneously acquired task fMRI, affording temporal resolution of reward system function, and PET imaging with [11C]Raclopride (RAC), assessing striatal dopamine (DA) D2/3 receptor binding, during performance of a probabilistic reward learning task. Both fMRI activation and PET DA measures showed ventral striatum involvement for signaling rewards. However, greater DA release was uniquely associated with learning strategies (i.e., learning rates) that were more task-optimal within the best fitting reinforcement learning model. This DA response was associated with BOLD activation of a network of regions including anterior cingulate cortex, medial prefrontal cortex, thalamus and posterior parietal cortex, primarily during expectation, rather than prediction error, task epochs. Together, these data provide novel, human in vivo evidence that striatal dopaminergic signaling interacts with a network of cortical regions to generate task-optimal learning strategies, rather than representing reward outcomes in isolation.

A computational model of learning flexible navigation in a maze by layout-conforming replay of place cells

Recent experimental observations have shown that the reactivation of hippocampal place cells (PC) during sleep or wakeful immobility depicts trajectories that can go around barriers and can flexibly adapt to a changing maze layout. However, existing computational models of replay fall short of generating such layout-conforming replay, restricting their usage to simple environments, like linear tracks or open fields. In this paper, we propose a computational model that generates layout-conforming replay and explains how such replay drives the learning of flexible navigation in a maze. First, we propose a Hebbian-like rule to learn the inter-PC synaptic strength during exploration. Then we use a continuous attractor network (CAN) with feedback inhibition to model the interaction among place cells and hippocampal interneurons. The activity bump of place cells drifts along paths in the maze, which models layout-conforming replay. During replay in sleep, the synaptic strengths from place cells to striatal medium spiny neurons (MSN) are learned by a novel dopamine-modulated three-factor rule to store place-reward associations. During goal-directed navigation, the CAN periodically generates replay trajectories from the animal's location for path planning, and the trajectory leading to a maximal MSN activity is followed by the animal. We have implemented our model into a high-fidelity virtual rat in the MuJoCo physics simulator. Extensive experiments have demonstrated that its superior flexibility during navigation in a maze is due to a continuous re-learning of inter-PC and PC-MSN synaptic strength.

Spontaneous behaviour is structured by reinforcement without explicit reward

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.

Crosstalk between biochemical signalling network architecture and trafficking governs AMPAR dynamics in synaptic plasticity

Synaptic plasticity involves modification of both biochemical and structural components of neurons. Many studies have revealed that the change in the number density of the glutamatergic receptor AMPAR at the synapse is proportional to synaptic weight update; an increase in AMPAR corresponds to strengthening of synapses while a decrease in AMPAR density weakens synaptic connections. The dynamics of AMPAR are thought to be regulated by upstream signalling, primarily the calcium-CaMKII pathway, trafficking to and from the synapse, and influx from extrasynaptic sources. Previous work in the field of deterministic modelling of CaMKII dynamics has assumed bistable kinetics, while experiments and rule-based modelling have revealed that CaMKII dynamics can be either monostable or ultrasensitive. This raises the following question: how does the choice of model assumptions involving CaMKII dynamics influence AMPAR dynamics at the synapse? To answer this question, we have developed a set of models using compartmental ordinary differential equations to systematically investigate contributions of different signalling and trafficking variations, along with their coupled effects, on AMPAR dynamics at the synaptic site. We find that the properties of the model including network architecture describing different stability features of CaMKII and parameters that capture the endocytosis and exocytosis of AMPAR significantly affect the integration of fast upstream species by slower downstream species. Furthermore, we predict that the model outcome, as determined by bound AMPAR at the synaptic site, depends on (1) the choice of signalling model (bistable CaMKII or monostable CaMKII dynamics), (2) trafficking versus influx contributions and (3) frequency of stimulus. KEY POINTS: The density of AMPA receptors (AMPARs) at the postsynaptic density of the synapse provides a readout of synaptic plasticity, which involves crosstalk between complex biochemical signalling networks including CaMKII dynamics and trafficking pathways including exocytosis and endocytosis. Here we build a model that integrates CaMKII dynamics and AMPAR trafficking to explore this crosstalk. We compare different models of CaMKII that result in monostable or bistable kinetics and their impact on AMPAR dynamics. Our results show that AMPAR density depends on the coupling between aspects of biochemical signalling and trafficking. Specifically, assumptions regarding CaMKII dynamics and its stability features can alter AMPAR density at the synapse. Our model also predicts that the kinetics of trafficking versus influx of AMPAR from the extrasynaptic space can further impact AMPAR density. Thus, the contributions of both signalling and trafficking should be considered in computational models.

Unraveling the mysteries of dendritic spine dynamics: Five key principles shaping memory and cognition

Recent research extends our understanding of brain processes beyond just action potentials and chemical transmissions within neural circuits, emphasizing the mechanical forces generated by excitatory synapses on dendritic spines to modulate presynaptic function. From in vivo and in vitro studies, we outline five central principles of synaptic mechanics in brain function: P1: Stability - Underpinning the integral relationship between the structure and function of the spine synapses. P2: Extrinsic dynamics - Highlighting synapse-selective structural plasticity which plays a crucial role in Hebbian associative learning, distinct from pathway-selective long-term potentiation (LTP) and depression (LTD). P3: Neuromodulation - Analyzing the role of G-protein-coupled receptors, particularly dopamine receptors, in time-sensitive modulation of associative learning frameworks such as Pavlovian classical conditioning and Thorndike's reinforcement learning (RL). P4: Instability - Addressing the intrinsic dynamics crucial to memory management during continual learning, spotlighting their role in "spine dysgenesis" associated with mental disorders. P5: Mechanics - Exploring how synaptic mechanics influence both sides of synapses to establish structural traces of short- and long-term memory, thereby aiding the integration of mental functions. We also delve into the historical background and foresee impending challenges.

Plasticity of synapses and reward circuit function in the genesis and treatment of depression

What changes in brain function cause the debilitating symptoms of depression? Can we use the answers to this question to invent more effective, faster acting antidepressant drug therapies? This review provides an overview and update of the converging human and preclinical evidence supporting the hypothesis that changes in the function of excitatory synapses impair the function of the circuits they are embedded in to give rise to the pathological changes in mood, hedonic state, and thought processes that characterize depression. The review also highlights complementary human and preclinical findings that classical and novel antidepressant drugs relieve the symptoms of depression by restoring the functions of these same synapses and circuits. These findings offer a useful path forward for designing better antidepressant compounds.

Mesolimbic dopamine adapts the rate of learning from action

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions^1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction⁴; however, so far there has been little consideration of how direct policy learning might inform our understanding⁵. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning⁶.

Dendritic Spines in Learning and Memory: From First Discoveries to Current Insights

The central nervous system is composed of neural ensembles, and their activity patterns are neural correlates of cognitive functions. Those ensembles are networks of neurons connected to each other by synapses. Most neurons integrate synaptic signal through a remarkable subcellular structure called spine. Dendritic spines are protrusions whose diverse shapes make them appear as a specific neuronal compartment, and they have been the focus of studies for more than a century. Soon after their first description by Ramón y Cajal, it has been hypothesized that spine morphological changes could modify neuronal connectivity and sustain cognitive abilities. Later studies demonstrated that changes in spine density and morphology occurred in experience-dependent plasticity during development, and in clinical cases of mental retardation. This gave ground for the assumption that dendritic spines are the particular locus of cerebral plasticity. With the discovery of synaptic long-term potentiation, a research program emerged with the aim to establish whether dendritic spine plasticity could explain learning and memory. The development of live imaging methods revealed on the one hand that dendritic spine remodeling is compatible with learning process and, on the other hand, that their long-term stability is compatible with lifelong memories. Furthermore, the study of the mechanisms of spine growth and maintenance shed new light on the rules of plasticity. In behavioral paradigms of memory, spine formation or elimination and morphological changes were found to correlate with learning. In a last critical step, recent experiments have provided evidence that dendritic spines play a causal role in learning and memory.

Psychosis spectrum illnesses as disorders of prefrontal critical period plasticity

Emerging research on neuroplasticity processes in psychosis spectrum illnesses-from the synaptic to the macrocircuit levels-fill key gaps in our models of pathophysiology and open up important treatment considerations. In this selective narrative review, we focus on three themes, emphasizing alterations in spike-timing dependent and Hebbian plasticity that occur during adolescence, the critical period for prefrontal system development: (1) Experience-dependent dysplasticity in psychosis emerges from activity decorrelation within neuronal ensembles. (2) Plasticity processes operate bidirectionally: deleterious environmental and experiential inputs shape microcircuits. (3) Dysregulated plasticity processes interact across levels of scale and time and include compensatory mechanisms that have pathogenic importance. We present evidence that-given the centrality of progressive dysplastic changes, especially in prefrontal cortex-pharmacologic or neuromodulatory interventions will need to be supplemented by corrective learning experiences for the brain if we are to help people living with these illnesses to fully thrive.

Adaptive control of synaptic plasticity integrates micro- and macroscopic network function

Synaptic plasticity configures interactions between neurons and is therefore likely to be a primary driver of behavioral learning and development. How this microscopic-macroscopic interaction occurs is poorly understood, as researchers frequently examine models within particular ranges of abstraction and scale. Computational neuroscience and machine learning models offer theoretically powerful analyses of plasticity in neural networks, but results are often siloed and only coarsely linked to biology. In this review, we examine connections between these areas, asking how network computations change as a function of diverse features of plasticity and vice versa. We review how plasticity can be controlled at synapses by calcium dynamics and neuromodulatory signals, the manifestation of these changes in networks, and their impacts in specialized circuits. We conclude that metaplasticity-defined broadly as the adaptive control of plasticity-forges connections across scales by governing what groups of synapses can and can't learn about, when, and to what ends. The metaplasticity we discuss acts by co-opting Hebbian mechanisms, shifting network properties, and routing activity within and across brain systems. Asking how these operations can go awry should also be useful for understanding pathology, which we address in the context of autism, schizophrenia and Parkinson's disease.

Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues

The ability to learn Pavlovian associations from environmental cues predicting positive outcomes is critical for survival, motivating adaptive behaviours. This cued-motivated behaviour depends on the nucleus accumbens (NAc). NAc output activity mediated by spiny projecting neurons (SPNs) is regulated by dopamine, but also by cholinergic interneurons (CINs), which can release acetylcholine and glutamate via the activity of the vesicular acetylcholine transporter (VAChT) or the vesicular glutamate transporter (VGLUT3), respectively. Here we investigated behavioural and neurochemical changes in mice performing a touchscreen Pavlovian approach task by recording dopamine, acetylcholine, and calcium dynamics from D1- and D2-SPNs using fibre photometry in control, VAChT or VGLUT3 mutant mice to understand how these signals cooperate in the service of approach behaviours toward reward-predicting cues. We reveal that NAc acetylcholine-dopaminergic signalling is continuously updated to regulate striatal output underlying the acquisition of Pavlovian approach learning toward reward-predicting cues.

Lateral axonal modulation is required for stimulus-specific olfactory conditioning in Drosophila

Effective and stimulus-specific learning is essential for animals' survival. Two major mechanisms are known to aid stimulus specificity of associative learning. One is accurate stimulus-specific representations in neurons. The second is a limited effective temporal window for the reinforcing signals to induce neuromodulation after sensory stimuli. However, these mechanisms are often imperfect in preventing unspecific associations; different sensory stimuli can be represented by overlapping populations of neurons, and more importantly, the reinforcing signals alone can induce neuromodulation even without coincident sensory-evoked neuronal activity. Here, we report a crucial neuromodulatory mechanism that counteracts both limitations and is thereby essential for stimulus specificity of learning. In Drosophila, olfactory signals are sparsely represented by cholinergic Kenyon cells (KCs), which receive dopaminergic reinforcing input. We find that KCs have numerous axo-axonic connections mediated by the muscarinic type-B receptor (mAChR-B). By using functional imaging and optogenetic approaches, we show that these axo-axonic connections suppress both odor-evoked calcium responses and dopamine-evoked cAMP signals in neighboring KCs. Strikingly, behavior experiments demonstrate that mAChR-B knockdown in KCs impairs olfactory learning by inducing undesired changes to the valence of an odor that was not associated with the reinforcer. Thus, this local neuromodulation acts in concert with sparse sensory representations and global dopaminergic modulation to achieve effective and accurate memory formation.

Dendritic involvement in inhibition and disinhibition of vulnerable dopaminergic neurons in healthy and pathological conditions

Dopaminergic neurons in the substantia nigra pars compacta (SNc) differentially degenerate in Parkinson's Disease, with the ventral region degenerating more severely than the dorsal region. Compared with the dorsal neurons, the ventral neurons in the SNc have distinct dendritic morphology, electrophysiological characteristics, and circuit connections with the basal ganglia. These characteristics shape information processing in the ventral SNc and structure the balance of inhibition and disinhibition in the striatonigral circuitry. In this paper, I review foundational studies and recent work comparing the circuitry of the ventral and dorsal SNc neurons and discuss how loss of the ventral neurons early in Parkinson's Disease could affect the overall balance of inhibition and disinhibition of dopamine signals.

Synaptic Plasticity Is Predicted by Spatiotemporal Firing Rate Patterns and Robust to In Vivo-like Variability

Synaptic plasticity, the experience-induced change in connections between neurons, underlies learning and memory in the brain. Most of our understanding of synaptic plasticity derives from in vitro experiments with precisely repeated stimulus patterns; however, neurons exhibit significant variability in vivo during repeated experiences. Further, the spatial pattern of synaptic inputs to the dendritic tree influences synaptic plasticity, yet is not considered in most synaptic plasticity rules. Here, we investigate how spatiotemporal synaptic input patterns produce plasticity with in vivo-like conditions using a data-driven computational model with a plasticity rule based on calcium dynamics. Using in vivo spike train recordings as inputs to different size clusters of spines, we show that plasticity is strongly robust to trial-to-trial variability of spike timing. In addition, we derive general synaptic plasticity rules describing how spatiotemporal patterns of synaptic inputs control the magnitude and direction of plasticity. Synapses that strongly potentiated have greater firing rates and calcium concentration later in the trial, whereas strongly depressing synapses have hiring firing rates early in the trial. The neighboring synaptic activity influences the direction and magnitude of synaptic plasticity, with small clusters of spines producing the greatest increase in synaptic strength. Together, our results reveal that calcium dynamics can unify diverse plasticity rules and reveal how spatiotemporal firing rate patterns control synaptic plasticity.

Dopamine drives neuronal excitability via KCNQ channel phosphorylation for reward behavior

Dysfunctional dopamine signaling is implicated in various neuropsychological disorders. Previously, we reported that dopamine increases D1 receptor (D1R)-expressing medium spiny neuron (MSN) excitability and firing rates in the nucleus accumbens (NAc) via the PKA/Rap1/ERK pathway to promote reward behavior. Here, the results show that the D1R agonist, SKF81297, inhibits KCNQ-mediated currents and increases D1R-MSN firing rates in murine NAc slices, which is abolished by ERK inhibition. In vitro ERK phosphorylates KCNQ2 at Ser414 and Ser476; in vivo, KCNQ2 is phosphorylated downstream of dopamine signaling in NAc slices. Conditional deletion of Kcnq2 in D1R-MSNs reduces the inhibitory effect of SKF81297 on KCNQ channel activity, while enhancing neuronal excitability and cocaine-induced reward behavior. These effects are restored by wild-type, but not phospho-deficient KCNQ2. Hence, D1R-ERK signaling controls MSN excitability via KCNQ2 phosphorylation to regulate reward behavior, making KCNQ2 a potential therapeutical target for psychiatric diseases with a dysfunctional reward circuit.

Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior

We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes.

Contingent Amygdala Inputs Trigger Heterosynaptic LTP at Hippocampus-To-Accumbens Synapses

The nucleus accumbens shell (NAcSh) is a key brain region where environmental cues acquire incentive salience to reinforce motivated behaviors. Principal medium spiny neurons (MSNs) in the NAcSh receive extensive glutamatergic projections from limbic regions, among which, the ventral hippocampus (vH) transmits information enriched in contextual cues, and the basolateral amygdala (BLA) encodes real-time arousing states. The vH and BLA project convergently to NAcSh MSNs, both activated in a time-locked manner on a cue-conditioned motivational action. In brain slices prepared from male and female mice, we show that co-activation of the two projections induces long-term potentiation (LTP) at vH-to-NAcSh synapses without affecting BLA-to-NAcSh synapses, revealing a heterosynaptic mechanism through which BLA signals persistently increase the temporally contingent vH-to-NAcSh transmission. Furthermore, this LTP is more prominent in dopamine D1 receptor-expressing (D1) MSNs than D2 MSNs and can be prevented by inhibition of either D1 receptors or dopaminergic terminals in NAcSh. This heterosynaptic LTP may provide a dopamine-guided mechanism through which vH-encoded cue inputs that are contingent to BLA activation acquire increased circuit representation to reinforce behavior.SIGNIFICANCE STATEMENT In motivated behaviors, environmental cues associated with arousing stimuli acquire increased incentive salience, processes mediated in part by the nucleus accumbens (NAc). NAc principal neurons receive glutamatergic projections from the ventral hippocampus (vH) and basolateral amygdala (BLA), which transmit information encoding contextual cues and affective states, respectively. Our results show that co-activation of the two projections induces long-term potentiation (LTP) at vH-to-NAc synapses without affecting BLA-to-NAc synapses, revealing a heterosynaptic mechanism through which BLA signals potentiate the temporally contingent vH-to-NAc transmission. Furthermore, this LTP is prevented by inhibition of either D1 receptors or dopaminergic axons. This heterosynaptic LTP may provide a dopamine-guided mechanism through which vH-encoded cue inputs that are contingent to BLA activation acquire increased circuit representation to reinforce behavior.

Ramping activity in the striatum

Control of the timing of behavior is thought to require the basal ganglia (BG) and BG pathologies impair performance in timing tasks. Temporal interval discrimination depends on the ramping activity of medium spiny neurons (MSN) in the main BG input structure, the striatum, but the underlying mechanisms driving this activity are unclear. Here, we combine an MSN dynamical network model with an action selection system applied to an interval discrimination task. We find that when network parameters are appropriate for the striatum so that slowly fluctuating marginally stable dynamics are intrinsically generated, up and down ramping populations naturally emerge which enable significantly above chance task performance. We show that emergent population activity is in very good agreement with empirical studies and discuss how MSN network dysfunction in disease may alter temporal perception.

Green fluorescent cAMP indicator of high speed and specificity suitable for neuronal live-cell imaging

Cyclic adenosine monophosphate (cAMP) is a canonical intracellular messenger playing diverse roles in cell functions. In neurons, cAMP promotes axonal growth during early development, and mediates sensory transduction and synaptic plasticity after maturation. The molecular cascades of cAMP are well documented, but its spatiotemporal profiles associated with neuronal functions remain hidden. Hence, we developed a genetically encoded cAMP indicator based on a bacterial cAMP-binding protein. This indicator "gCarvi" monitors [cAMP]i at 0.2 to 20 µM with a subsecond time resolution and a high specificity over cyclic guanosine monophosphate (cGMP). gCarvi can be converted to a ratiometric probe for [cAMP]i quantification and its expression can be specifically targeted to various subcellular compartments. Monomeric gCarvi also enables simultaneous multisignal monitoring in combination with other indicators. As a proof of concept, simultaneous cAMP/Ca2+ imaging in hippocampal neurons revealed a tight linkage of cAMP to Ca2+ signals. In cerebellar presynaptic boutons, forskolin induced nonuniform cAMP elevations among boutons, which positively correlated with subsequent increases in the size of the recycling pool of synaptic vesicles assayed using FM dye. Thus, the cAMP domain in presynaptic boutons is an important determinant of the synaptic strength.

Integrated regulation of PKA by fast and slow neurotransmission in the nucleus accumbens controls plasticity and stress responses

Cortical glutamate and midbrain dopamine neurotransmission converge to mediate striatum-dependent behaviors, while maladaptations in striatal circuitry contribute to mental disorders. However, the crosstalk between glutamate and dopamine signaling has not been entirely elucidated. Here we uncover a molecular mechanism by which glutamatergic and dopaminergic signaling integrate to regulate cAMP-dependent protein kinase (PKA) via phosphorylation of the PKA regulatory subunit, RIIβ. Using a combination of biochemical, pharmacological, neurophysiological, and behavioral approaches, we find that glutamate-dependent reduction in cyclin-dependent kinase 5 (Cdk5)-dependent RIIβ phosphorylation alters the PKA holoenzyme autoinhibitory state to increase PKA signaling in response to dopamine. Furthermore, we show that disruption of RIIβ phosphorylation by Cdk5 enhances cortico-ventral striatal synaptic plasticity. In addition, we demonstrate that acute and chronic stress in rats inversely modulate RIIβ phosphorylation and ventral striatal infusion of a small interfering peptide that selectively targets RIIβ regulation by Cdk5 improves behavioral response to stress. We propose this new signaling mechanism integrating ventral striatal glutamate and dopamine neurotransmission is important to brain function, may contribute to neuropsychiatric conditions, and serves as a possible target for the development of novel therapeutics for stress-related disorders.

Deep Brain Stimulation for Addictive Disorders-Where Are We Now?

In the face of a global epidemic of drug addiction, neglecting to develop new effective therapies will perpetuate the staggering human and economic costs of substance use. This review aims to summarize and evaluate the preclinical and clinical studies of deep brain stimulation (DBS) as a novel therapy for refractory addiction, in hopes to engage and inform future research in this promising novel treatment avenue. An electronic database search (MEDLINE, EMBASE, Cochrane library) was performed using keywords and predefined inclusion criteria between 1974 and 6/18/2021 (registered on Open Science Registry). Selected articles were reviewed in full text and key details were summarized and analyzed to understand DBS' therapeutic potential and possible mechanisms of action. The search yielded 25 animal and 22 human studies. Animal studies showed that DBS of targets such as nucleus accumbens (NAc), insula, and subthalamic nucleus reduces drug use and seeking. All human studies were case series/reports (level 4/5 evidence), mostly targeting the NAc with generally positive outcomes. From the limited evidence in the literature, DBS, particularly of the NAc, appears to be a reasonable last resort option for refractory addictive disorders. We propose that future research in objective electrophysiological (e.g., local field potentials) and neurochemical (e.g., extracellular dopamine levels) biomarkers would assist monitoring the progress of treatment and developing a closed-loop DBS system. Preclinical literature also highlighted the prefrontal cortex as a promising DBS target, which should be explored in human research.

Dopamine-related oxidative stress and mitochondrial dysfunction in dopaminergic neurons differentiated from deciduous teeth-derived stem cells of children with Down syndrome

Down syndrome (DS) is one of the common genetic disorders caused by the trisomy of human chromosome 21 (HSA21). Mitochondrial dysfunction and redox imbalance play important roles in DS pathology, and altered dopaminergic regulation has been demonstrated in the brain of individuals with DS. However, the pathological association of these elements is not yet fully understood. In this study, we analyzed dopaminergic neurons (DNs) differentiated from deciduous teeth-derived stem cells of children with DS or healthy control children. As previously observed in the analysis of a single case of DS, compared to controls, patient-derived DNs (DS-DNs) displayed shorter neurite outgrowth and fewer branches, as well as downregulated vesicular monoamine transporter 2 and upregulated dopamine transporter 1, both of which are key regulators of dopamine homeostasis in DNs. In agreement with these expression profiles, DS-DNs accumulated dopamine intracellularly and had increased levels of cellular and mitochondrial reactive oxygen species (ROS). DS-DNs showed downregulation of non-canonical Notch ligand, delta-like 1, which may contribute to dopamine accumulation and increased ROS levels through DAT1 upregulation. Furthermore, DS-DNs showed mitochondrial dysfunction in consistent with lower expression of peroxisome proliferator-activated receptor-gamma coactivator 1 alpha (PGC-1α) and upregulation of a HSA21-encoded negative regulator of PGC-1α, nuclear receptor-interacting protein 1. These results suggest that dysregulated dopamine homeostasis may participate in oxidative stress and mitochondrial dysfunction of the dopaminergic system in DS.

Distinct representation of cue-outcome association by D1 and D2 neurons in the ventral striatum's olfactory tubercle

Positive and negative associations acquired through olfactory experience are thought to be especially strong and long-lasting. The conserved direct olfactory sensory input to the ventral striatal olfactory tubercle (OT) and its convergence with dense dopaminergic input to the OT could underlie this privileged form of associative memory, but how this process occurs is not well understood. We imaged the activity of the two canonical types of striatal neurons, expressing D1- or D2-type dopamine receptors, in the OT at cellular resolution while mice learned odor-outcome associations ranging from aversive to rewarding. D1 and D2 neurons both responded to rewarding and aversive odors. D1 neurons in the OT robustly and bidirectionally represented odor valence, responding similarly to odors predicting similar outcomes regardless of odor identity. This valence representation persisted even in the absence of a licking response to the odors and in the absence of the outcomes, indicating a true transformation of odor sensory information by D1 OT neurons. In contrast, D2 neuronal representation of the odor-outcome associations was weaker, contingent on a licking response by the mouse, and D2 neurons were more selective for odor identity than valence. Stimulus valence coding in the OT was modality-sensitive, with separate sets of D1 neurons responding to odors and sounds predicting the same outcomes, suggesting that integration of multimodal valence information happens downstream of the OT. Our results point to distinct representation of identity and valence of odor stimuli by D1 and D2 neurons in the OT.

Norepinephrine potentiates and serotonin depresses visual cortical responses by transforming eligibility traces

Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in mouse V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.

Persimmon leaf extract alleviates chronic social defeat stress-induced depressive-like behaviors by preventing dendritic spine loss via inhibition of serotonin reuptake in mice

Background

Fresh or dried Persimmon leaves (Diospyros kaki Thunb.) exhibit preventive effects on cardiovascular and cerebrovascular diseases. However, their antidepressant effects and underlying mechanisms are unclear. Thus, we investigated mechanisms responsible for Persimmon leaf extract (PLE) activity on chronic social defeat stress (CSDS)-induced depressive-like behaviors in mice.

Methods

CSDS was used as a mouse model of depression. We performed the sucrose preference test (SPT), forced swim test (FST), and tail suspension test (TST) to identify depressive-like behavior. Spine density and dendritic morphology were assessed using Golgi staining. Neurochemicals were quantified by microdialysis, doublecortin by immunofluorescence, and cAMP using an ELISA kit. Finally, the levels of cortical proteins of phosphorylated cAMP-response element binding protein (CREB), brain-derived neurotrophic factor (BDNF), postsynaptic density synapsin-1 and protein 95 (PSD95) were quantified by western blot. 16S rRNA gene sequencing was used to detect fecal microbiota.

Results

Treatment of CSDS-subjected mice with PLE (30.0–60.0 mg/kg, i.g.) enhanced sucrose preference, decreased immobility times in the TST and FST but did not affect locomotor activity. Furthermore, persistent social defeat stress decreased dendritic spine density and dendritic length in the brain, as well as decreased PSD95 and synapsin-1 expression. PLE, interestingly, inhibited dendritic spine loss and increased synaptic protein levels. PLE also increased brain levels of 5-HT, cAMP, phosphorylated (p)-CREB, BDNF, PSD95, and synapsin-1 in mice subjected to CSDS. Furthermore, PLE increased their doublecortin-positive cell count in the hippocampal dentate gyrus. CSDS mice represented a distinct fecal microbiota cluster which differed compared with normal C57BL/6J mice, and the phenotype was rescued by PLE.

Conclusions

PLE alleviated CSDS-induced depressive behaviors and spinal damage by suppressing serotonin reuptake and activating the cAMP/CREB/BDNF signaling pathway. Simultaneously, PLE influenced the composition of the fecal microbiota in CSDS-subjected mice.

Supplementary information

The online version contains supplementary material available at 10.1186/s13020-022-00609-4.

A synaptomic analysis reveals dopamine hub synapses in the mouse striatum

Dopamine transmission is involved in reward processing and motor control, and its impairment plays a central role in numerous neurological disorders. Despite its strong pathophysiological relevance, the molecular and structural organization of the dopaminergic synapse remains to be established. Here, we used targeted labelling and fluorescence activated sorting to purify striatal dopaminergic synaptosomes. We provide the proteome of dopaminergic synapses with 57 proteins specifically enriched. Beyond canonical markers of dopamine neurotransmission such as dopamine biosynthetic enzymes and cognate receptors, we validated 6 proteins not previously described as enriched. Moreover, our data reveal the adhesion of dopaminergic synapses to glutamatergic, GABAergic or cholinergic synapses in structures we named “dopamine hub synapses”. At glutamatergic synapses, pre- and postsynaptic markers are significantly increased upon association with dopamine synapses. Dopamine hub synapses may thus support local dopaminergic signalling, complementing volume transmission thought to be the major mechanism by which monoamines modulate network activity.

The neurotransmitter dopamine is an important regulator of brain function. Here the authors describe “dopamine hub synapses”, where dopamine transmission may act in synergy with other neurotransmitters.

Choice-selective sequences dominate in cortical relative to thalamic inputs to NAc to support reinforcement learning

How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens, which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex and midline regions of the thalamus. However, little is known about whether and how representations differ across these input pathways. By comparing these inputs during a reinforcement learning task in mice, we discovered that prelimbic cortical inputs preferentially represent actions and choices, whereas midline thalamic inputs preferentially represent cues. Choice-selective activity in the prelimbic cortical inputs is organized in sequences that persist beyond the outcome. Through computational modeling, we demonstrate that these sequences can support the neural implementation of reinforcement-learning algorithms, in both a circuit model based on synaptic plasticity and one based on neural dynamics. Finally, we test and confirm a prediction of our circuit models by direct manipulation of nucleus accumbens input neurons.

Training a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning

Recent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.

Serotonin receptors contribute to dopamine depression of lateral inhibition in the nucleus accumbens

Dopamine modulation of nucleus accumbens (NAc) circuitry is central to theories of reward seeking and reinforcement learning. Despite decades of effort, the acute dopamine actions on the NAc microcircuitry remain puzzling. Here, we dissect out the direct actions of dopamine on lateral inhibition between medium spiny neurons (MSNs) in mouse brain slices and find that they are pathway specific. Dopamine potently depresses GABAergic transmission from presynaptic dopamine D2 receptor-expressing MSNs (D2-MSNs), whereas it potentiates transmission from presynaptic dopamine D1 receptor-expressing MSNs (D1-MSNs) onto other D1-MSNs. To our surprise, presynaptic D2 receptors mediate only half of the depression induced by endogenous and exogenous dopamine. Presynaptic serotonin 5-HT1B receptors are responsible for a significant component of dopamine-induced synaptic depression. This study clarifies the mechanistic understanding of dopamine actions in the NAc by showing pathway-specific modulation of lateral inhibition and involvement of D2 and 5-HT1B receptors in dopamine depression of D2-MSN synapses.

Adaptive changes in striatal projection neurons explain the long duration response and the emergence of dyskinesias in patients with Parkinson's disease

Neuronal activity in the brain is tightly regulated. During operation in real time, for instance, feedback and feedforward loops limit excessive excitation. In addition, cell autonomous processes ensure that neurons’ average activity is restored to a setpoint in response to chronic perturbations. These processes are summarized as homeostatic plasticity (Turrigiano in Cold Spring Harb Perspect Biol 4:a005736–a005736, 2012). In the basal ganglia, information is mainly transmitted through disinhibition, which already constraints the possible range of neuronal activity. When this tightly adjusted system is challenged by the chronic decline in dopaminergic neurotransmission in Parkinson’s disease (PD), homeostatic plasticity aims to compensate for this perturbation. We here summarize recent experimental work from animals demonstrating that striatal projection neurons adapt excitability and morphology in response to chronic dopamine depletion and substitution. We relate these cellular processes to clinical observations in patients with PD that cannot be explained by the classical model of basal ganglia function. These include the long duration response to dopaminergic medication that takes weeks to develop and days to wear off. Moreover, dyskinesias are considered signs of excessive dopaminergic neurotransmission in Parkinson’s disease, but they are typically more severe on the body side that is more strongly affected by dopamine depletion. We hypothesize that these clinical observations can be explained by homeostatic plasticity in the basal ganglia, suggesting that plastic changes in response to chronic dopamine depletion and substitution need to be incorporated into models of basal ganglia function. In addition, better understanding the molecular mechanism of homeostatic plasticity might offer new treatment options to avoid motor complications in patients with PD.

Use of Levodopa After a Stroke: A Systematic Review

Stroke is a leading cause of death and disability, especially in certain ethnic groups. Impaired consciousness is a common outcome in stroke patients, serving as a predictor of prognosis and mortality. Lately, there has been increased interest in drugs such as Levodopa (LD), which have been found to promote wakefulness. To further appreciate this association, we gathered updated evidence of this novel therapeutic approach and compared it, evaluating its clinical use in an acute stroke setting. We carried out a systematic review of clinical trials conducted exclusively on stroke patients who received levodopa. Four clinical trials were reviewed and analyzed after applying the inclusion/exclusion criteria. The use of levodopa showed positive results in four of the clinical trials, and statistically significant results in 3/4 of the studies; however, more studies need to be conducted to corroborate these results.

A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots

Building a human-like integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole. This approach is based on two ideas: (1) brain-inspired AI, learning human brain architecture to build human-level intelligence, and (2) a probabilistic generative model (PGM)-based cognitive architecture to develop a cognitive system for developmental robots by integrating PGMs. The proposed development framework is called a whole brain PGM (WB-PGM), which differs fundamentally from existing cognitive architectures in that it can learn continuously through a system based on sensory-motor information. In this paper, we describe the rationale for WB-PGM, the current status of PGM-based elemental cognitive modules, their relationship with the human brain, the approach to the integration of the cognitive modules, and future challenges. Our findings can serve as a reference for brain studies. As PGMs describe explicit informational relationships between variables, WB-PGM provides interpretable guidance from computational sciences to brain science. By providing such information, researchers in neuroscience can provide feedback to researchers in AI and robotics on what the current models lack with reference to the brain. Further, it can facilitate collaboration among researchers in neuro-cognitive sciences as well as AI and robotics.

Striatal synaptic adaptations in Parkinson's disease

The striatum is densely innervated by mesencephalic dopaminergic neurons that modulate acquisition and vigor of goal-directed actions and habits. This innervation is progressively lost in Parkinson's disease (PD), contributing to the defining movement deficits of the disease. Although boosting dopaminergic signaling with levodopa early in the course of the disease alleviates these deficits, later this strategy leads to the emergence of debilitating dyskinesia. Here, recent advances in our understanding of how striatal cells and circuits adapt to this progressive de-innervation and to levodopa therapy are discussed. First, we discuss how dopamine (DA) depletion triggers cell type-specific, homeostatic changes in spiny projection neurons (SPNs) that tend to normalize striatal activity but also lead to disruption of the synaptic architecture sculpted by experience. Second, we discuss the roles played by cholinergic and nitric oxide-releasing interneurons in these adaptations. Third, we examine recent work in freely moving mice suggesting that alterations in the spatiotemporal dynamics of striatal ensembles contributes to PD movement deficits. Lastly, we discuss recently published evidence from a progressive model of PD suggesting that contrary to the classical model, striatal pathway imbalance is necessary but not sufficient to produce frank parkinsonism.

Selective activation of BK channels in small-headed dendritic spines suppresses excitatory postsynaptic potentials

Dendritic spines are the main receptacles of excitatory information in the brain. Their particular morphology, with a small head connected to the dendrite by a slender neck, has inspired theoretical and experimental work to understand how these structural features affect the processing, storage and integration of synaptic inputs in pyramidal neurons (PNs). The activation of glutamate receptors in spines triggers a large voltage change as well as calcium signals at the spine head. Thus, voltage-gated and calcium-activated potassium channels located in the spine head likely play a key role in synaptic transmission. Here we study the presence and function of large conductance calcium-activated potassium (BK) channels in spines from layer 5 PNs. We found that BK channels are localized to dendrites and spines regardless of their size, but their activity can only be detected in spines with small head volumes (≤0.09 μm³ ), which reduces the amplitude of two-photon uncaging excitatory postsynaptic potentials recorded at the soma. In addition, we found that calcium signals in spines with small head volumes are significantly larger than those observed in spines with larger head volumes. In accordance with our experimental data, numerical simulations predict that synaptic inputs impinging onto spines with small head volumes generate voltage responses and calcium signals within the spine head itself that are significantly larger than those observed in spines with larger head volumes, which are sufficient to activate spine BK channels. These results show that BK channels are selectively activated in small-headed spines, suggesting a new level of dendritic spine-mediated regulation of synaptic processing, integration and plasticity in cortical PNs. KEY POINTS: BK channels are expressed in the visual cortex and layer 5 pyramidal neuron somata, dendrites and spines regardless of their size. BK channels are selectively activated in small-headed spines (≤0.09 μm³ ), which reduces the amplitude of two-photon (2P) uncaging excitatory postsynaptic potentials (EPSPs) recorded at the soma. Two-photon imaging revealed that intracellular calcium responses in the head of 2P-activated spines are significantly larger in small-headed spines (≤0.09 μm³ ) than in spines with larger head volumes. In accordance with our experimental data, numerical simulations showed that synaptic inputs impinging onto spines with small head volumes (≤0.09 μm³ ) generate voltage responses and calcium signals within the spine head itself that are significantly larger than those observed in spines with larger head volumes, sufficient to activate spine BK channels and suppress EPSPs.

A multisynaptic pathway from the ventral midbrain toward spinal motoneurons in monkeys

Abstract

Motivation boosts motor performance. Activity of the ventral midbrain (VM), consisting of the ventral tegmental area (VTA), the substantia nigra pars compacta (SNc) and the retrorubral field (RRF), plays an important role in processing motivation. However, little is known about the neural substrate bridging the VM and the spinal motor output. We hypothesized that the VM might exert a modulatory influence over the descending motor pathways. By retrograde transneuronal labelling with rabies virus, we demonstrated the existence of multisynaptic projections from the VM to the cervical enlargement in monkeys. The distribution pattern of spinal projection neurons in the VM exhibited a caudorostral gradient, in that the RRF and the caudal part of the SNc contained more retrogradely labelled neurons than the VTA and the rostral part of the SNc. Electrical stimulation of the VM induced muscle responses in the contralateral forelimb with a delay of a few milliseconds following the responses of the ipsilateral primary motor cortex (M1). The magnitude and number of evoked muscle responses were associated with the stimulus intensity and number of pulses. The muscle responses were diminished during M1 inactivation. Thus, the present study has identified a multisynaptic VM–spinal pathway that is mediated, at least in part, by the M1 and might play a pivotal role in modulatory control of the spinal motor output.

Key points

Motivation to obtain reward is thought to boost motor performance, and activity in the ventral midbrain is important to the motivational process.

Little is known about a neural substrate bridging the ventral midbrain and the spinal motor output.

Retrograde trans‐synaptic experiments revealed that the ventral midbrain projects multisynaptically to the spinal cord in macaque monkeys.

Ventral midbrain activation by electrical stimulation generated cortical activity in the motor cortex and forelimb muscle activity.

A multisynaptic ventral midbrain–spinal pathway most probably plays a pivotal role in modulatory control of the spinal motor output.

A behavioural correlate of the synaptic eligibility trace in the nucleus accumbens

Reward reinforces the association between a preceding sensorimotor event and its outcome. Reinforcement learning (RL) theory and recent brain slice studies explain the delayed reward action such that synaptic activities triggered by sensorimotor events leave a synaptic eligibility trace for 1 s. The trace produces a sensitive period for reward-related dopamine to induce synaptic plasticity in the nucleus accumbens (NAc). However, the contribution of the synaptic eligibility trace to behaviour remains unclear. Here we examined a reward-sensitive period to brief pure tones with an accurate measurement of an effective timing of water reward in head-fixed Pavlovian conditioning, which depended on the plasticity-related signaling in the NAc. We found that the reward-sensitive period was within 1 s after the pure tone presentation and optogenetically-induced presynaptic activities at the NAc, showing that the short reward-sensitive period was in conformity with the synaptic eligibility trace in the NAc. These findings support the application of the synaptic eligibility trace to construct biologically plausible RL models.

History-dependent dopamine release increases cAMP levels in most basal amygdala glutamatergic neurons to control learning

Dopaminergic inputs to basal amygdala (BA) instruct learning of motivational salience. This learning depends on intracellular plasticity signals such as cyclic adenosine monophosphate (cAMP), which is regulated by activation of dopamine receptors. We examine the dynamics of dopamine release and downstream signaling during multiple salient events occurring within tens of seconds. We perform real-time tracking and manipulation of cAMP in BA neurons in vitro and in vivo. Optogenetically evoked release of dopamine drives proportional increases in cAMP in almost all BA glutamatergic neurons, suggesting widespread actions of dopamine across neurons preferring positive or negative valence. This cAMP response decreases across trials with short intertrial intervals owing to depression of dopamine release. No such depression is evident when photostimulating cAMP production directly. cAMP and protein kinase A responses to repeated appetitive or aversive stimuli also exhibit pronounced depression. Thus, history-dependent dynamics of dopamine and cAMP may regulate learning of temporally clustered, salient stimuli.

Molecular and functional architecture of striatal dopamine release sites

Despite the importance of dopamine for striatal circuit function, mechanistic understanding of dopamine transmission remains incomplete. We recently showed that dopamine secretion relies on the presynaptic scaffolding protein RIM, indicating that it occurs at active zone-like sites similar to classical synaptic vesicle exocytosis. Here, we establish using a systematic gene knockout approach that Munc13 and Liprin-α, active zone proteins for vesicle priming and release site organization, are important for dopamine secretion. Furthermore, RIM zinc finger and C₂B domains, which bind to Munc13 and Liprin-α, respectively, are needed to restore dopamine release after RIM ablation. In contrast, and different from typical synapses, the active zone scaffolds RIM-BP and ELKS, and RIM domains that bind to them, are expendable. Hence, dopamine release necessitates priming and release site scaffolding by RIM, Munc13, and Liprin-α, but other active zone proteins are dispensable. Our work establishes that efficient release site architecture mediates fast dopamine exocytosis.

A Nonlinear Hidden Layer Enables Actor-Critic Agents to Learn Multiple Paired Association Navigation

Navigation to multiple cued reward locations has been increasingly used to study rodent learning. Though deep reinforcement learning agents have been shown to be able to learn the task, they are not biologically plausible. Biologically plausible classic actor-critic agents have been shown to learn to navigate to single reward locations, but which biologically plausible agents are able to learn multiple cue-reward location tasks has remained unclear. In this computational study, we show versions of classic agents that learn to navigate to a single reward location, and adapt to reward location displacement, but are not able to learn multiple paired association navigation. The limitation is overcome by an agent in which place cell and cue information are first processed by a feedforward nonlinear hidden layer with synapses to the actor and critic subject to temporal difference error-modulated plasticity. Faster learning is obtained when the feedforward layer is replaced by a recurrent reservoir network.

Canonical neural networks perform active inference

This work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function-and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity-accompanied with adaptation of firing thresholds-is a sufficient neuronal substrate to attain Bayes optimal inference and control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

Cell-type- and region-specific modulation of cocaine seeking by micro-RNA-1 in striatal projection neurons

The persistent and experience-dependent nature of drug addiction may result in part from epigenetic alterations, including non-coding micro-RNAs (miRNAs), which are both critical for neuronal function and modulated by cocaine in the striatum. Two major striatal cell populations, the striato-nigral and striato-pallidal projection neurons, express, respectively, the D1 (D1-SPNs) and D2 (D2-SPNs) dopamine receptor, and display distinct but complementary functions in drug-evoked responses. However, a cell-type-specific role for miRNAs action has yet to be clarified. Here, we evaluated the expression of a subset of miRNAs proposed to modulate cocaine effects in the nucleus accumbens (NAc) and dorsal striatum (DS) upon sustained cocaine exposure in mice and showed that these selected miRNAs were preferentially upregulated in the NAc. We focused on miR-1 considering the important role of some of its predicted mRNA targets, Fosb and Npas4, in the effects of cocaine. We validated these targets in vitro and in vivo. We explored the potential of miR-1 to regulate cocaine-induced behavior by overexpressing it in specific striatal cell populations. In DS D1-SPNs miR-1 overexpression downregulated Fosb and Npas4 and reduced cocaine-induced CPP reinstatement, but increased cue-induced cocaine seeking. In DS D2-SPNs miR-1 overexpression reduced the motivation to self-administer cocaine. Our results indicate a role of miR1 and its target genes, Fosb and Npas4, in these behaviors and highlight a precise cell-type- and region-specific modulatory role of miR-1, illustrating the importance of cell-specific investigations.

Cell-Type-Specific Adaptions in Striatal Medium-Sized Spiny Neurons and Their Roles in Behavioral Responses to Drugs of Abuse

Drug addiction is defined as a compulsive pattern of drug-seeking- and taking- behavior, with recurrent episodes of abstinence and relapse, and a loss of control despite negative consequences. Addictive drugs promote reinforcement by increasing dopamine in the mesocorticolimbic system, which alters excitatory glutamate transmission within the reward circuitry, thereby hijacking reward processing. Within the reward circuitry, the striatum is a key target structure of drugs of abuse since it is at the crossroad of converging glutamate inputs from limbic, thalamic and cortical regions, encoding components of drug-associated stimuli and environment, and dopamine that mediates reward prediction error and incentive values. These signals are integrated by medium-sized spiny neurons (MSN), which receive glutamate and dopamine axons converging onto their dendritic spines. MSN primarily form two mostly distinct populations based on the expression of either DA-D1 (D1R) or DA-D2 (D2R) receptors. While a classical view is that the two MSN populations act in parallel, playing antagonistic functional roles, the picture seems much more complex. Herein, we review recent studies, based on the use of cell-type-specific manipulations, demonstrating that dopamine differentially modulates dendritic spine density and synapse formation, as well as glutamate transmission, at specific inputs projecting onto D1R-MSN and D2R-MSN to shape persistent pathological behavioral in response to drugs of abuse. We also discuss the identification of distinct molecular events underlying the detrimental interplay between dopamine and glutamate signaling in D1R-MSN and D2R-MSN and highlight the relevance of such cell-type-specific molecular studies for the development of innovative strategies with potential therapeutic value for addiction. Because drug addiction is highly prevalent in patients with other psychiatric disorders when compared to the general population, we last discuss the hypothesis that shared cellular and molecular adaptations within common circuits could explain the co-occurrence of addiction and depression. We will therefore conclude this review by examining how the nucleus accumbens (NAc) could constitute a key interface between addiction and depression.

Spike-Timing-Dependent Plasticity Mediated by Dopamine and its Role in Parkinson's Disease Pathophysiology

Parkinson's disease (PD) is a multi-systemic neurodegenerative brain disorder. Motor symptoms of PD are linked to the significant dopamine (DA) loss in substantia nigra pars compacta (SNc) followed by basal ganglia (BG) circuit dysfunction. Increasing experimental and computational evidence indicates that (synaptic) plasticity plays a key role in the emergence of PD-related pathological changes following DA loss. Spike-timing-dependent plasticity (STDP) mediated by DA provides a mechanistic model for synaptic plasticity to modify synaptic connections within the BG according to the neuronal activity. To shed light on how DA-mediated STDP can shape neuronal activity and synaptic connectivity in the PD condition, we reviewed experimental and computational findings addressing the modulatory effect of DA on STDP as well as other plasticity mechanisms and discussed their potential role in PD pathophysiology and related network dynamics and connectivity. In particular, reshaping of STDP profiles together with other plasticity-mediated processes following DA loss may abnormally modify synaptic connections in competing pathways of the BG. The cascade of plasticity-induced maladaptive or compensatory changes can impair the excitation-inhibition balance towards the BG output nuclei, leading to the emergence of pathological activity-connectivity patterns in PD. Pre-clinical, clinical as well as computational studies reviewed here provide an understanding of the impact of synaptic plasticity and other plasticity mechanisms on PD pathophysiology, especially PD-related network activity and connectivity, after DA loss. This review may provide further insights into the abnormal structure-function relationship within the BG contributing to the emergence of pathological states in PD. Specifically, this review is intended to provide detailed information for the development of computational network models for PD, serving as testbeds for the development and optimization of invasive and non-invasive brain stimulation techniques. Computationally derived hypotheses may accelerate the development of therapeutic stimulation techniques and potentially reduce the number of related animal experiments.

A Modular Workflow for Model Building, Analysis, and Parameter Estimation in Systems Biology and Neuroscience

Neuroscience incorporates knowledge from a range of scales, from single molecules to brain wide neural networks. Modeling is a valuable tool in understanding processes at a single scale or the interactions between two adjacent scales and researchers use a variety of different software tools in the model building and analysis process. Here we focus on the scale of biochemical pathways, which is one of the main objects of study in systems biology. While systems biology is among the more standardized fields, conversion between different model formats and interoperability between various tools is still somewhat problematic. To offer our take on tackling these shortcomings and by keeping in mind the FAIR (findability, accessibility, interoperability, reusability) data principles, we have developed a workflow for building and analyzing biochemical pathway models, using pre-existing tools that could be utilized for the storage and refinement of models in all phases of development. We have chosen the SBtab format which allows the storage of biochemical models and associated data in a single file and provides a human readable set of syntax rules. Next, we implemented custom-made MATLAB® scripts to perform parameter estimation and global sensitivity analysis used in model refinement. Additionally, we have developed a web-based application for biochemical models that allows simulations with either a network free solver or stochastic solvers and incorporating geometry. Finally, we illustrate convertibility and use of a biochemical model in a biophysically detailed single neuron model by running multiscale simulations in NEURON. Using this workflow, we can simulate the same model in three different simulators, with a smooth conversion between the different model formats, enhancing the characterization of different aspects of the model.

Active inference leads to Bayesian neurophysiology

The neuronal substrates that implement the free-energy principle and ensuing active inference at the neuron and synapse level have not been fully elucidated. This Review considers possible neuronal substrates underlying the principle. First, the foundations of the free-energy principle are introduced, and then its ability to empirically explain various brain functions and psychological and biological phenomena in terms of Bayesian inference is described. Mathematically, the dynamics of neural activity and plasticity that minimise a cost function can be cast as performing Bayesian inference that minimises variational free energy. This equivalence licenses the adoption of the free-energy principle as a universal characterisation of neural networks. Further, the neural network structure itself represents a generative model under which an agent operates. A virtue of this perspective is that it enables the formal association of neural network properties with prior beliefs that regulate inference and learning. The possible neuronal substrates that implement prior beliefs and how to empirically examine the theory are discussed. This perspective renders brain activity explainable, leading to a deeper understanding of the neuronal mechanisms underlying basic psychology and psychiatric disorders in terms of an implicit generative model.