Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment

Microinvasive Probes for Monitoring Electrical and Chemical Neural Activity in Nonhuman Primates

We leveraged carbon fiber materials for creating sensors that provide dual neurochemical and electrical neural activity recording at microinvasive (10 μm) spatial footprints proximal to recording sites, and enabling these measurements from deep brain targets of primates with conventional cranial chambers. These shaft-assisted microinvasive probes (s-μIPs) are approximately 10 μm in diameter along the distal length (1-15 mm) immediately surrounding the targeted recording site. This microinvasive portion ensures that the recording site is isolated from tissue damage induced by the wider shaft portion of the device. The shaft (150-165 μm in diameter) within the device stiffens the remaining length of the probe (>100 mm), and provides compatibility with standard intracranial insertion protocols (e.g., guide tubes and chamber setups) that require a sufficiently rigid and long shaft for deep brain insertion in monkeys. The s-μIP was further expanded to provide dual-channel chemical and electrical neural activity recording with micrometer spatial resolution. Measurements of reward- and movement- related dopamine, spikes, and local field potentials were made from single and dual-channel s-μIPs implanted in task-performing monkeys. Recordings from chronically implanted s-μIPs display the capability of functional multimodal (chemical and electrical) neural activity measurements over 1-year postimplantation from microinvasive devices.

Striatal Dopamine Actions and Movement: Inferences from Parkinson Disease

The nature of motor deficits in Parkinson disease (PD) and aspects of their improvements with ʟ-DOPA replacement therapy (LDRT) offer potential insights into striatal dopamine actions. The defining and most LDRT responsive feature of PD, bradykinesia, is a complex phenomenon exhibiting impairments of both simple and complex limb movements. LDRT significantly remediates the former but not the latter. LDRT pharmacodynamics has two major components, the Short Duration Response (SDR), with a time course of seconds to minutes, and the Long Duration Response (LDR), with a time course of days to weeks. LDRT pharmacodynamics suggests different striatal dopamine actions on different time scales. While many studies used PD subjects to investigate striatal dopamine actions, few take LDRT pharmacodynamics into account. Correlating bradykinesia features and LDRT pharmacodynamics with our present understanding of striatal dopamine actions suggests that LDRT failure to improve complex movement performance reflects loss of phasic dopaminergic signaling. The SDR is likely a result of partially restored tonic/volume striatal dopamine neurotransmission. There is no existing explanation for the LDR. Recent experiments isolating the LDR cast doubt on prior accounts of how LDRT remediates bradykinesia. Interactions between the SDR and LDR may give rise to novel properties. Longer duration effects of striatal dopaminergic signaling need to be incorporated into studies of striatal dopamine actions in both preclinical and clinical experiments. Careful studies of PD bradykinesia and LDRT pharmacodynamics offer potential avenues for future explorations of striatal dopamine actions. Systematic preclinical experiments are needed to optimize design of clinical experiments.

A neural mechanism for learning from delayed postingestive feedback

Animals learn the value of foods on the basis of their postingestive effects and thereby develop aversions to foods that are toxic1-10 and preferences to those that are nutritious11-13. However, it remains unclear how the brain is able to assign credit to flavours experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here we reveal an unexpected role for the postingestive reactivation of neural flavour representations in this temporal credit-assignment process. To begin, we leverage the fact that mice learn to associate novel14,15, but not familiar, flavours with delayed gastrointestinal malaise signals to investigate how the brain represents flavours that support aversive postingestive learning. Analyses of brain-wide activation patterns reveal that a network of amygdala regions is unique in being preferentially activated by novel flavours across every stage of learning (consumption, delayed malaise and memory retrieval). By combining high-density recordings in the amygdala with optogenetic stimulation of malaise-coding hindbrain neurons, we show that delayed malaise signals selectively reactivate flavour representations in the amygdala from a recent meal. The degree of malaise-driven reactivation of individual neurons predicts the strengthening of flavour responses upon memory retrieval, which in turn leads to stabilization of the population-level representation of the recently consumed flavour. By contrast, flavour representations in the amygdala degrade in the absence of unexpected postingestive consequences. Thus, we demonstrate that postingestive reactivation and plasticity of neural flavour representations may support learning from delayed feedback.

Genetically encoded sensors illuminate in vivo detection for neurotransmission: Development, application, and optimization strategies

Limitations in existing tools have hindered neuroscientists from achieving a deeper understanding of complex behaviors and diseases. The recent development and optimization of genetically encoded sensors offer a powerful solution for investigating intricate dynamics such as calcium influx, membrane potential, and the release of neurotransmitters and neuromodulators. In contrast, traditional methods are constrained by insufficient spatial and/or temporal resolution, low sensitivity, and stringent application conditions. Genetically encoded sensors have gained widespread popularity due to their advantageous features, which stem from their genetic encoding and optical imaging capabilities. These include broad applicability, tissue specificity, and non-invasive operation. When combined with advanced microscopic techniques, optogenetics, and machine learning approaches, these sensors have become versatile tools for studying neuronal circuits in intact living systems, providing millisecond-scale temporal resolution and spatial resolution ranging from nanometers to micrometers. In this review, we highlight the advantages of genetically encoded sensors over traditional methods in the study of neurotransmission. We also discuss their recent advancements, diverse applications, and optimization strategies.

Metaplasticity and continual learning: mechanisms subserving brain computer interface proficiency

Objective.Brain computer interfaces (BCIs) require substantial cognitive flexibility to optimize control performance. Remarkably, learning this control is rapid, suggesting it might be mediated by neuroplasticity mechanisms operating on very short time scales. Here, we propose a meta plasticity model of BCI learning and skill consolidation at the single cell and population levels comprised of three elements: (a) behavioral time scale synaptic plasticity (BTSP), (b) intrinsic plasticity (IP) and (c) synaptic scaling (SS) operating at time scales from seconds to minutes to hours and days. Notably, the model is able to explainrepresentational drift-a frequent and widespread phenomenon that adversely affects BCI control and continued use.Approach.We developed an all-optical approach to characterize IP, BTSP and SS with single cell resolution in awake mice using fluorescent two photon (2P) GCaMP7s imaging and optogenetic stimulation of the soma targeted ChRmineKv2.1. We further trained mice on a one-dimensional BCI control task and systematically characterized within session (seconds to minutes) learning as well as across sessions (days and weeks) with different neural ensembles.Main results.On the time scale of seconds, substantial BTSP could be induced and was followed by significant IP over minutes. Over the time scale of days and weeks, these changes could predict BCI control proficiency, suggesting that BTSP and IP might be complemented by SS to stabilize and consolidate BCI control.Significance.Our results provide early experimental support for a meta plasticity model of continual BCI learning and skill consolidation. The model predictions may be used to design and calibrate neural decoders with complete autonomy while considering the temporal and spatial scales of plasticity mechanisms. With the power of modern-day machine learning and artificial Intelligence, fully autonomous neural decoding and adaptation in BCIs might be achieved with minimal to no human intervention.

Developing algorithmic psychiatry via multi-level spanning computational models

Modern psychiatry faces challenges in translating neurobiological insights into treatments for severe illnesses. The mid-20th century witnessed the rise of molecular mechanisms as pathophysiological and treatment models, with recent holistic proposals keeping this focus unaltered. In this perspective, we explore how psychiatry can utilize systems neuroscience to develop a vertically integrated understanding of brain function to inform treatment. Using schizophrenia as a case study, we discuss scale-related challenges faced by researchers studying molecules, circuits, networks, and cognition and clinicians operating within existing frameworks. We emphasize computation as a bridging language, with algorithmic models like hierarchical predictive processing offering explanatory potential for targeted interventions. Developing such models will not only facilitate new interventions but also optimize combining existing treatments by predicting their multi-level effects. We conclude with the prognosis that the future is bright, but that continued investment in research closely driven by clinical realities will be critical.

From avoidance to new action: the multifaceted role of the striatal indirect pathway

A hallmark of optimal reinforcement learning is that an agent learns to avoid actions that lead to negative outcomes while still exploring alternative actions that could lead to better outcomes. Although the basal ganglia have been hypothesized to contribute to this computation, the mechanisms by which they do so are still unclear. Here, we focus on the function of the striatal indirect pathway and propose that it is regulated by a synaptic plasticity rule that allows an animal to avoid actions that lead to suboptimal outcomes. We consider current theories of striatal indirect pathway function in light of recent experimental findings and discuss studies that suggest that indirect pathway activity is potentiated by the suppression of dopamine release in the striatum. Furthermore, we highlight recent studies showing that activation of the indirect pathway can trigger an action, allowing animals to explore new actions while suppressing suboptimal actions. We show how our framework can reconcile previously conflicting results regarding the indirect pathway and suggest experiments for future investigation.

Bioengineered nanomaterials for dynamic diagnostics in vivo

In vivo diagnostics obtains real-time physiological information directly from the site of interest in a patient's body, providing more accurate disease diagnosis compared with ex vivo diagnostics. Particularly, in vivo dynamic diagnostics allows the continuous monitoring of physiological signals over a period of time, offering deeper insights into disease pathogenesis and progression. However, achieving in situ dynamic diagnostics in deep tissues presents challenges related to energy and signal penetration as well as dynamic monitoring. Bioengineered nanomaterials serve as an ideal platform for in vivo dynamic diagnostics, leveraging energy conversion and biofunctionalization to enable continuous acquisition of physiological information across temporal and spatial scales. In this review, with reference to the studies from the last five years, we summarize the fundamental components that are essential for dynamic diagnosis in vivo. Firstly, an input energy source with high tissue penetration is needed, such as near-infrared (NIR) light, X-rays, magnetic field and ultrasound. Secondly, a nanomaterial class that is responsive to such an energy source to provide a readable output signal is chosen. Thirdly, bioengineered nanoprobes are designed to exhibit spatial, temporal or spatiotemporal changes in the output signal. Finally, different methods are used to analyse the output signal of nanoprobes, such as detecting changes in optical, radiation, magnetic and ultrasound signals. This review also discusses the obstacles and potential solutions for advancing these bioengineered nanomaterials toward clinical translational applications.

An axonal brake on striatal dopamine output by cholinergic interneurons

Depolarization of axons is necessary for somatic action potentials to trigger axonal neurotransmitter release. Here we show that striatal cholinergic interneurons (ChIs) and nicotinic receptors (nAChRs) on mouse dopamine axons interrupt this relationship. After nAChR-mediated depolarization, dopamine release by subsequent depolarization events was suppressed for ~100 ms. This suppression was not due to depletion of dopamine or acetylcholine, but to a limited reactivation of dopamine axons after nAChR-mediated depolarization, and is more prominent in dorsal than in ventral striatum. In vivo, nAChRs predominantly depressed dopamine release, as nAChR antagonism in dorsal striatum elevated dopamine detected with optic-fiber photometry of dopamine sensor GRABDA2m and promoted conditioned place preference. Our findings reveal that ChIs acting via nAChRs transiently limit the reactivation of dopamine axons for subsequent action potentials in dopamine neurons and therefore generate a dynamic inverse scaling of dopamine release according to ChI activity.

Temporal fMRI Dynamics Map Dopamine Physiology

Spatial variations in dopamine function are linked to cognition and substance use disorders but are challenging to characterize with current methods. Because dopamine influences blood vessel dilation, we hypothesized that hemodynamic latency, which reflects BOLD signal timing, could serve as an indirect marker of dopamine physiology. Across four datasets, we found a topography of hemodynamic latencies that precisely distinguished the nucleus accumbens, a dopaminergic region implicated in motivation and substance abuse, from other striatal regions. Using PET, genetics, and pharmacology, we found that hemodynamic latencies are robustly related to dopamine function and dopamine-linked behavior. In individuals with cocaine use disorder, we observed a spatial gradient of altered hemodynamic latencies in the striatum. This pattern independently predicted nicotine use, revealing a conserved physiological profile associated with addictive substance use. Hemodynamic latencies map regional, individual, and pathological differences linked to dopamine, opening new avenues for indirectly assessing the role of dopamine in healthy cognition and disease.

Dopamine D1 receptor activation in the striatum is sufficient to drive reinforcement of anteceding cortical patterns

Timed dopamine signals underlie reinforcement learning, favoring neural activity patterns that drive behaviors with positive outcomes. In the striatum, dopamine activates five dopamine receptors (D1R-D5R), which are differentially expressed in striatal neurons. However, the role of specific dopamine receptors in reinforcement is poorly understood. Using our cell-specific D1R photo-agonist, we find that D1R activation in D1-expressing neurons in the dorsomedial striatum is sufficient to reinforce preceding neural firing patterns in defined ensembles of layer 5 cortico-striatal neurons of the mouse motor cortex. The reinforcement is cumulative and time dependent, with an optimal effect when D1R activation follows the selected neural pattern after a short interval. Our results show that D1R activation in striatal neurons can selectively reinforce cortical activity patterns, independent of a behavioral outcome or a reward, crucially contributing to the fundamental mechanisms that support cognitive functions like learning, memory, and decision-making.

Micro-invasive probes for monitoring electrical and chemical neural activity in nonhuman primates

We leveraged carbon fiber materials for creating sensors that provide dual neurochemical and electrical neural activity recording at micro-invasive (10 μm) spatial footprints proximal to recording sites, and enabling these measurements from deep brain targets of primates with conventional cranial chambers. These shaft-assisted micro-invasive probes (s-μIPs) are approximately 10 μm in diameter along the distal length (1 - 15 mm) immediately surrounding the targeted recording site. This micro-invasive portion ensures that the recording site is isolated from tissue damage induced by the wider shaft portion of the device. The shaft (150 - 165 μm in diameter) within the device stiffens the remaining length of the probe (> 100 mm), and provides compatibility with standard intracranial insertion protocols (e.g., guide tubes and chamber setups) that require a sufficiently rigid and long shaft for deep brain insertion in monkeys. The s-μIP was further expanded to provide dual-channel chemical and electrical neural activity recording with micrometer spatial resolution. Measurements of reward- and movement- related dopamine, spikes, and local field potentials were made from single and dual-channel s-μIPs implanted in task-performing monkeys. Recordings from chronically implanted s-μIPs display the capability of functional multi-modal (chemical and electrical) neural activity measurements over 1-year post-implantation from micro-invasive devices.

Neuronal encoding of behaviors and instrumental learning in the dorsal striatum

The dorsal striatum is instrumental in regulating motor control and goal-directed behaviors. The classical description of the two output pathways of the dorsal striatum highlights their antagonistic control over actions. However, recent experimental evidence implicates both pathways and their coordinated activities during actions. In this review, we examine the different models proposed for striatal encoding of actions during self-paced behaviors and how they can account for evidence harvested during goal-directed behaviors. We also discuss how the activation of striatal ensembles can be reshaped and reorganized to support the formation of instrumental learning and behavioral flexibility. Future work integrating these considerations may resolve controversies regarding the control of actions by striatal networks.

Wideband ratiometric measurement of tonic and phasic dopamine release in the striatum

Reward learning, cognition, and motivation are supported by changes in neurotransmitter levels across multiple timescales. Current measurement technologies for various neuromodulators (such as dopamine and serotonin) do not bridge timescales of fluctuations, limiting the ability to define the behavioral significance, regulation, and relationship between fast (phasic) and slow (tonic) dynamics. To help resolve longstanding debates about the behavioral significance of dopamine across timescales, we developed a novel quantification strategy, augmenting extensively used carbon-fiber Fast Scan Cyclic Voltammetry (FSCV). We iteratively engineered the FSCV scan sequence to rapidly modify electrode sensitivity within a sampling window and applied ratiometric analysis for wideband dopamine measurement. This allowed us to selectively eliminate artifacts unrelated to electrochemical detection (i.e., baseline drift), overcoming previous limitations that precluded wideband dopamine detection from milliseconds to hours. We extensively characterize this approach in vitro, validate performance in vivo with simultaneous microdialysis, and deploy this technique to measure wideband dopamine changes across striatal regions under pharmacological, optogenetic, and behavioral manipulations. We demonstrate that our approach can extend to additional analytes, including serotonin and pH, providing a robust platform to assess the contributions of multi-timescale neuromodulator fluctuations to cognition, learning, and motivation.

Dopamine release plateau and outcome signals in dorsal striatum contrast with classic reinforcement learning formulations

We recorded dopamine release signals in centromedial and centrolateral sectors of the striatum as mice learned consecutive versions of visual cue-outcome conditioning tasks. Dopamine release responses differed for the centromedial and centrolateral sites. In neither sector could these be accounted for by classic reinforcement learning alone as classically applied to the activity of nigral dopamine-containing neurons. Medially, cue responses ranged from initial sharp peaks to modulated plateau responses; outcome (reward) responses during cue conditioning were minimal or, initially, negative. At centrolateral sites, by contrast, strong, transient dopamine release responses occurred at both cue and outcome. Prolonged, plateau release responses to cues emerged in both regions when discriminative behavioral responses became required. At most sites, we found no evidence for a transition from outcome signaling to cue signaling, a hallmark of temporal difference reinforcement learning as applied to midbrain dopaminergic neuronal activity. These findings delineate a reshaping of striatal dopamine release activity during learning and suggest that current views of reward prediction error encoding need review to accommodate distinct learning-related spatial and temporal patterns of striatal dopamine release in the dorsal striatum.

Transformers and cortical waves: encoders for pulling in context across time

The capabilities of transformer networks such as ChatGPT and other large language models (LLMs) have captured the world's attention. The crucial computational mechanism underlying their performance relies on transforming a complete input sequence - for example, all the words in a sentence - into a long 'encoding vector' that allows transformers to learn long-range temporal dependencies in naturalistic sequences. Specifically, 'self-attention' applied to this encoding vector enhances temporal context in transformers by computing associations between pairs of words in the input sequence. We suggest that waves of neural activity traveling across single cortical areas, or multiple regions on the whole-brain scale, could implement a similar encoding principle. By encapsulating recent input history into a single spatial pattern at each moment in time, cortical waves may enable a temporal context to be extracted from sequences of sensory inputs, the same computational principle as that used in transformers.

Multiplexed neurochemical sensing with sub-nM sensitivity across 2.25 mm2 area

Multichannel arrays capable of real-time sensing of neuromodulators in the brain are crucial for gaining insights into new aspects of neural communication. However, measuring neurochemicals, such as dopamine, at low concentrations over large areas has proven challenging. In this research, we demonstrate a novel approach that leverages the scalability and processing power offered by microelectrode array devices integrated with a functionalized, high-density microwire bundle, enabling electrochemical sensing at an unprecedented scale and spatial resolution. The sensors demonstrate outstanding selective molecular recognition by incorporating a selective polymeric membrane. By combining cutting-edge commercial multiplexing, digitization, and data acquisition hardware with a bio-compatible and highly sensitive neurochemical interface array, we establish a powerful platform for neurochemical analysis. This multichannel array has been successfully utilized in vitro and ex vivo systems. Notably, our results show a sensing area of 2.25 mm2 with an impressive detection limit of 820 pM for dopamine. This new approach paves the way for investigating complex neurochemical processes and holds promise for advancing our understanding of brain function and neurological disorders.

Learning temporal relationships between symbols with Laplace Neural Manifolds

Firing across populations of neurons in many regions of the mammalian brain maintains a temporal memory, a neural timeline of the recent past. Behavioral results demonstrate that people can both remember the past and anticipate the future over an analogous internal timeline. This paper presents a mathematical framework for building this timeline of the future. We assume that the input to the system is a time series of symbols-sparse tokenized representations of the present-in continuous time. The goal is to record pairwise temporal relationships between symbols over a wide range of time scales. We assume that the brain has access to a temporal memory in the form of the real Laplace transform. Hebbian associations with a diversity of synaptic time scales are formed between the past timeline and the present symbol. The associative memory stores the convolution between the past and the present. Knowing the temporal relationship between the past and the present allows one to infer relationships between the present and the future. With appropriate normalization, this Hebbian associative matrix can store a Laplace successor representation and a Laplace predecessor representation from which measures of temporal contingency can be evaluated. The diversity of synaptic time constants allows for learning of non-stationary statistics as well as joint statistics between triplets of symbols. This framework synthesizes a number of recent neuroscientific findings including results from dopamine neurons in the mesolimbic forebrain.

A neural mechanism for learning from delayed postingestive feedback

Animals learn the value of foods based on their postingestive effects and thereby develop aversions to foods that are toxic1-6 and preferences to those that are nutritious7-14. However, it remains unclear how the brain is able to assign credit to flavors experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here, we reveal an unexpected role for postingestive reactivation of neural flavor representations in this temporal credit assignment process. To begin, we leverage the fact that mice learn to associate novel15-18, but not familiar, flavors with delayed gastric malaise signals to investigate how the brain represents flavors that support aversive postingestive learning. Surveying cellular resolution brainwide activation patterns reveals that a network of amygdala regions is unique in being preferentially activated by novel flavors across every stage of the learning process: the initial meal, delayed malaise, and memory retrieval. By combining high-density recordings in the amygdala with optogenetic stimulation of genetically defined hindbrain malaise cells, we find that postingestive malaise signals potently and specifically reactivate amygdalar novel flavor representations from a recent meal. The degree of malaise-driven reactivation of individual neurons predicts strengthening of flavor responses upon memory retrieval, leading to stabilization of the population-level representation of the recently consumed flavor. In contrast, meals without postingestive consequences degrade neural flavor representations as flavors become familiar and safe. Thus, our findings demonstrate that interoceptive reactivation of amygdalar flavor representations provides a neural mechanism to resolve the temporal credit assignment problem inherent to postingestive learning.

Reward timescale controls the rate of behavioural and dopaminergic learning

Learning the causes of rewards is necessary for survival. Thus, it is critical to understand the mechanisms of such a vital biological process. Cue-reward learning is controlled by mesolimbic dopamine and improves with spacing of cue-reward pairings. However, whether a mathematical rule governs such improvements in learning rate, and if so, whether a unifying mechanism captures this rule and dopamine dynamics during learning remain unknown. Here, we investigate the behavioral, algorithmic, and dopaminergic mechanisms governing cue-reward learning rate. Across a range of conditions in mice, we show a strong, mathematically proportional relationship between both behavioral and dopaminergic learning rates and the duration between rewards. Due to this relationship, removing up to 19 out of 20 cue-reward pairings over a fixed duration has no influence on overall learning. These findings are explained by a dopamine-based model of retrospective learning, thereby providing a unified account of the biological mechanisms of learning.

Explaining dopamine through prediction errors and beyond

The most influential account of phasic dopamine holds that it reports reward prediction errors (RPEs). The RPE-based interpretation of dopamine signaling is, in its original form, probably too simple and fails to explain all the properties of phasic dopamine observed in behaving animals. This Perspective helps to resolve some of the conflicting interpretations of dopamine that currently exist in the literature. We focus on the following three empirical challenges to the RPE theory of dopamine: why does dopamine (1) ramp up as animals approach rewards, (2) respond to sensory and motor features and (3) influence action selection? We argue that the prediction error concept, once it has been suitably modified and generalized based on an analysis of each computational problem, answers each challenge. Nonetheless, there are a number of additional empirical findings that appear to demand fundamentally different theoretical explanations beyond encoding RPE. Therefore, looking forward, we discuss the prospects for a unifying theory that respects the diversity of dopamine signaling and function as well as the complex circuitry that both underlies and responds to dopaminergic transmission.

Distinct cholinergic circuits underlie discrete effects of reward on attention

Attention and reward are functions that are critical for the control of behavior, and massive multi-region neural systems have evolved to support the discrete computations associated with each. Previous research has also identified that attention and reward interact, though our understanding of the neural mechanisms that mediate this interplay is incomplete. Here, we review the basic neuroanatomy of attention, reward, and cholinergic systems. We then examine specific contexts in which attention and reward computations interact. Building on this work, we propose two discrete neural circuits whereby acetylcholine, released from cell groups located in different parts of the brain, mediates the impact of stimulus-reward associations as well as motivation on attentional control. We conclude by examining these circuits as a potential shared loci of dysfunction across diseases states associated with deficits in attention and reward.

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases

"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.

A feature-specific prediction error model explains dopaminergic heterogeneity

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

The challenge of learning adaptive mental behavior

Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

PET brain imaging in neurological disorders

Brain disorders are a series of conditions with damage or loss of neurons, such as Parkinson's disease (PD), Alzheimer's disease (AD), or drug dependence. These individuals have gradual deterioration of cognitive, motor, and other central nervous system functions affected. This degenerative trajectory is intricately associated with dysregulations in neurotransmitter systems. Positron Emission Tomography (PET) imaging, employing radiopharmaceuticals and molecular imaging techniques, emerges as a crucial tool for detecting brain biomarkers. It offers invaluable insights for early diagnosis and distinguishing brain disorders. This article comprehensively reviews the application and progress of conventional and novel PET imaging agents in diagnosing brain disorders. Furthermore, it conducts a thorough analysis on merits and limitations. The article also provides a forward-looking perspective in the future development directions of PET imaging agents for diagnosing brain disorders and proposes potential innovative strategies. It aims to furnish clinicians and researchers with an all-encompassing overview of the latest advancements and forthcoming trends in the utilization of PET imaging for diagnosing brain disorders.

Rethinking dopamine-guided action sequence learning

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.

Synchronous Measurements of Extracellular Action Potentials and Neurochemical Activity with Carbon Fiber Electrodes in Nonhuman Primates

Measuring the dynamic relationship between neuromodulators, such as dopamine, and neuronal action potentials is imperative to understand how these fundamental modes of neural signaling interact to mediate behavior. We developed methods to measure concurrently dopamine and extracellular action potentials (i.e., spikes) in monkeys. Standard fast-scan cyclic voltammetric (FSCV) electrochemical (EChem) and electrophysiological (EPhys) recording systems are combined and used to collect spike and dopamine signals, respectively, from an array of carbon fiber (CF) sensors implanted in the monkey striatum. FSCV requires the application of small voltages at the implanted sensors to measure redox currents generated from target molecules, such as dopamine. These applied voltages create artifacts at neighboring EPhys measurement sensors which may lead to misclassification of these signals as physiological spikes. Therefore, simple automated temporal interpolation algorithms were designed to remove these artifacts and enable accurate spike extraction. We validated these methods using simulated artifacts and demonstrated an average spike recovery rate of 84.5%. We identified and discriminated cell type-specific units in the monkey striatum that were shown to correlate to specific behavioral task parameters related to reward size and eye movement direction. Synchronously recorded spike and dopamine signals displayed contrasting relations to the task variables, suggesting a complex relationship between these two modes of neural signaling. Future application of our methods will help advance our understanding of the interactions between neuromodulator signaling and neuronal activity, to elucidate more detailed mechanisms of neural circuitry and plasticity mediating behaviors in health and in disease.

Brain states as wave-like motifs

There is ample evidence of wave-like activity in the brain at multiple scales and levels. This emerging literature supports the broader adoption of a wave perspective of brain activity. Specifically, a brain state can be described as a set of recurring, sequential patterns of propagating brain activity, namely a wave. We examine a collective body of experimental work investigating wave-like properties. Based on these works, we consider brain states as waves using a scale-agnostic framework across time and space. Emphasis is placed on the sequentiality and periodicity associated with brain activity. We conclude by discussing the implications, prospects, and experimental opportunities of this framework.

The direction of theta and alpha travelling waves modulates human memory processing

To support a range of behaviours, the brain must flexibly coordinate neural activity across widespread brain regions. One potential mechanism for this coordination is a travelling wave, in which a neural oscillation propagates across the brain while organizing the order and timing of activity across regions. Although travelling waves are present across the brain in various species, their potential functional relevance has remained unknown. Here, using rare direct human brain recordings, we demonstrate a distinct functional role for travelling waves of theta- and alpha-band (2-13 Hz) oscillations in the cortex. Travelling waves propagate in different directions during separate cognitive processes. In episodic memory, travelling waves tended to propagate in a posterior-to-anterior direction during successful memory encoding and in an anterior-to-posterior direction during recall. Because travelling waves of oscillations correspond to local neuronal spiking, these patterns indicate that rhythmic pulses of activity move across the brain in different directions for separate behaviours. More broadly, our results suggest a fundamental role for travelling waves and oscillations in dynamically coordinating neural connectivity, by flexibly organizing the timing and directionality of network interactions across the cortex to support cognition and behaviour.

What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Acetylcholine demixes heterogeneous dopamine signals for learning and moving

Midbrain dopamine neurons promote reinforcement learning and movement vigor. A major outstanding question is how dopamine-recipient neurons in the striatum parse these heterogeneous signals. Here we characterized dopamine and acetylcholine release in the dorsomedial striatum (DMS) of rats performing a decision-making task. We found that dopamine acted as a reward prediction error (RPE), modulating behavior and DMS spiking on subsequent trials when coincident with pauses in cholinergic release. In contrast, at task events that elicited coincident bursts of acetylcholine and dopamine, dopamine preceded contralateral movements and predicted movement vigor without inducing plastic changes in DMS firing rates. Our findings provide a circuit-level mechanism by which cholinergic modulation allows the same dopamine signals to be used for either movement or learning depending on instantaneous behavioral context.

Spatiotemporal Organization of Prefrontal Norepinephrine Influences Neuronal Activity

Norepinephrine (NE), a neuromodulator released by locus ceruleus (LC) neurons throughout the cortex, influences arousal and learning through extrasynaptic vesicle exocytosis. While NE within cortical regions has been viewed as a homogenous field, recent studies have demonstrated heterogeneous axonal dynamics and advances in GPCR-based fluorescent sensors permit direct observation of the local dynamics of NE at cellular scale. To investigate how the spatiotemporal dynamics of NE release in the prefrontal cortex (PFC) affect neuronal firing, we employed in vivo two-photon imaging of layer 2/3 of the PFC in order to observe fine-scale neuronal calcium and NE dynamics concurrently. In this proof of principle study, we found that local and global NE fields can decouple from one another, providing a substrate for local NE spatiotemporal activity patterns. Optic flow analysis revealed putative release and reuptake events which can occur at the same location, albeit at different times, indicating the potential to create a heterogeneous NE field. Utilizing generalized linear models, we demonstrated that cellular Ca2+ fluctuations are influenced by both the local and global NE field. However, during periods of local/global NE field decoupling, the local field drives cell firing dynamics rather than the global field. These findings underscore the significance of localized, phasic NE fluctuations for structuring cell firing, which may provide local neuromodulatory control of cortical activity.

Mesolimbic dopamine ramps reflect environmental timescales

Mesolimbic dopamine activity occasionally exhibits ramping dynamics, reigniting debate on theories of dopamine signaling. This debate is ongoing partly because the experimental conditions under which dopamine ramps emerge remain poorly understood. Here, we show that during Pavlovian and instrumental conditioning, mesolimbic dopamine ramps are only observed when the inter-trial interval is short relative to the trial period. These results constrain theories of dopamine signaling and identify a critical variable determining the emergence of dopamine ramps.

Distributional coding of associative learning in discrete populations of midbrain dopamine neurons

Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.

Toward a brighter constellation: multiorgan neuroimaging of neural and vascular dynamics in the spinal cord and brain

Significance

Pain comprises a complex interaction between motor action and somatosensation that is dependent on dynamic interactions between the brain and spinal cord. This makes understanding pain particularly challenging as it involves rich interactions between many circuits (e.g., neural and vascular) and signaling cascades throughout the body. As such, experimentation on a single region may lead to an incomplete and potentially incorrect understanding of crucial underlying mechanisms.

Aim

We aimed to develop and validate tools to enable detailed and extended observation of neural and vascular activity in the brain and spinal cord. The first key set of innovations was targeted to developing novel imaging hardware that addresses the many challenges of multisite imaging. The second key set of innovations was targeted to enabling bioluminescent (BL) imaging, as this approach can address limitations of fluorescent microscopy including photobleaching, phototoxicity, and decreased resolution due to scattering of excitation signals.

Approach

We designed 3D-printed brain and spinal cord implants to enable effective surgical implantations and optical access with wearable miniscopes or an open window (e.g., for one- or two-photon microscopy or optogenetic stimulation). We also tested the viability for BL imaging and developed a novel modified miniscope optimized for these signals (BLmini).

Results

We describe "universal" implants for acute and chronic simultaneous brain-spinal cord imaging and optical stimulation. We further describe successful imaging of BL signals in both foci and a new miniscope, the "BLmini," which has reduced weight, cost, and form-factor relative to standard wearable miniscopes.

Conclusions

The combination of 3D-printed implants, advanced imaging tools, and bioluminescence imaging techniques offers a coalition of methods for understanding spinal cord-brain interactions. Our work has the potential for use in future research into neuropathic pain and other sensory disorders and motor behavior.

BioLuminescent OptoGenetics in the choroid plexus: integrated opto- and chemogenetic control in vivo

Significance

The choroid plexus (ChP) epithelial network displays diverse dynamics, including propagating calcium waves and individuated fluctuations in single cells. These rapid events underscore the possibility that ChP dynamics may reflect behaviorally relevant and clinically important changes in information processing and signaling. Optogenetic and chemogenetic tools provide spatiotemporally precise and sustained approaches for testing such dynamics in vivo. Here, we describe the feasibility of a novel combined opto- and chemogenetic tool, BioLuminescent-OptoGenetics (BL-OG), for the ChP in vivo. In the "LuMinOpsin" (LMO) BL-OG strategy, a luciferase is tethered to an adjacent optogenetic element. This molecule allows chemogenetic activation when the opsin is driven by light produced through luciferase binding a small molecule (luciferin) or by conventional optogenetic light sources and BL-OG report of activation through light production.

Aim

To test the viability of BL-OG/LMO for ChP control.

Approach

Using transgenic and Cre-directed targeting to the ChP, we expressed LMO3 (a Gaussia luciferase-VChR1 fusion), a highly effective construct in neural systems. In mice expressing LMO3 in ChP, we directly imaged BL light production following multiple routes of coelenterazine (CTZ: luciferin) administration using an implanted cannula system. We also used home-cage videography with Deep LabCut analysis to test for any impact of repeated CTZ administration on basic health and behavioral indices.

Results

Multiple routes of CTZ administration drove BL photon production, including intracerebroventricular, intravenous, and intraperitoneal injection. Intravenous administration resulted in fast "flash" kinetics that diminished in seconds to minutes, and intraperitoneal administration resulted in slow rising activity that sustained hours. Mice showed no consistent impact of 1 week of intraperitoneal CTZ administration on weight, drinking, motor behavior, or sleep/wake cycles.

Conclusions

BL-OG/LMO provides unique advantages for testing the role of ChP dynamics in biological processes.

Improved green and red GRAB sensors for monitoring dopaminergic activity in vivo

Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a large network of dopaminergic projections. To dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent G-protein-coupled receptor activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity and signal-to-noise ratio with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.

Striatal dopamine release tracks the relationship between actions and their consequences

The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.

Targeted micro-fiber arrays for measuring and manipulating localized multi-scale neural dynamics over large, deep brain volumes during behavior

Neural population dynamics relevant to behavior vary over multiple spatial and temporal scales across three-dimensional volumes. Current optical approaches lack the spatial coverage and resolution necessary to measure and manipulate naturally occurring patterns of large-scale, distributed dynamics within and across deep brain regions such as the striatum. We designed a new micro-fiber array approach capable of chronically measuring and optogenetically manipulating local dynamics across over 100 targeted locations simultaneously in head-fixed and freely moving mice, enabling the investigation of cell-type- and neurotransmitter-specific signals over arbitrary 3D volumes at a spatial resolution and coverage previously inaccessible. We applied this method to resolve rapid dopamine release dynamics across the striatum, revealing distinct, modality-specific spatiotemporal patterns in response to salient sensory stimuli extending over millimeters of tissue. Targeted optogenetics enabled flexible control of neural signaling on multiple spatial scales, better matching endogenous signaling patterns, and the spatial localization of behavioral function across large circuits.

Parkinson's Disease: Cells Succumbing to Lifelong Dopamine-Related Oxidative Stress and Other Bioenergetic Challenges

The core pathological event in Parkinson's disease (PD) is the specific dying of dopamine (DA) neurons of the substantia nigra pars compacta (SNc). The reasons why SNc DA neurons are especially vulnerable and why idiopathic PD has only been found in humans are still puzzling. The two main underlying factors of SNc DA neuron vulnerability appear related to high DA production, namely (i) the toxic effects of cytoplasmic DA metabolism and (ii) continuous cytosolic Ca2+ oscillations in the absence of the Ca2+-buffer protein calbindin. Both factors cause oxidative stress by producing highly reactive quinones and increasing intra-mitochondrial Ca2+ concentrations, respectively. High DA expression in human SNc DA neuron cell bodies is suggested by the abundant presence of the DA-derived pigment neuromelanin, which is not found in such abundance in other species and has been associated with toxicity at higher levels. The oxidative stress created by their DA production system, despite the fact that the SN does not use unusually high amounts of energy, explains why SNc DA neurons are sensitive to various genetic and environmental factors that create mitochondrial damage and thereby promote PD. Aging increases multiple risk factors for PD, and, to a large extent, PD is accelerated aging. To prevent PD neurodegeneration, possible approaches that are discussed here are (1) reducing cytoplasmic DA accumulation, (2) blocking cytoplasmic Ca2+ oscillations, and (3) providing bioenergetic support.

Apathy and Motivation: Biological Basis and Drug Treatment

Apathy is a disabling syndrome associated with poor functional outcomes that is common across a broad range of neurological and psychiatric conditions. Currently, there are no established therapies specifically for the condition, and safe and effective treatments are urgently needed. Advances in the understanding of motivation and goal-directed behavior in humans and animals have shed light on the cognitive and neurobiological mechanisms contributing to apathy, providing an important foundation for the development of new treatments. Here, we review the cognitive components, neural circuitry, and pharmacology of apathy and motivation, highlighting converging evidence of shared transdiagnostic mechanisms. Though no pharmacological treatments have yet been licensed, we summarize trials of existing and novel compounds to date, identifying several promising candidates for clinical use and avenues of future drug development.

Dopamine release in human associative striatum during reversal learning

The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned.

Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

Towards a Brighter Constellation: Multi-Organ Neuroimaging of Neural and Vascular Dynamics in the Spinal Cord and Brain

Significance

Pain is comprised of a complex interaction between motor action and somatosensation that is dependent on dynamic interactions between the brain and spinal cord. This makes understanding pain particularly challenging as it involves rich interactions between many circuits (e.g., neural and vascular) and signaling cascades throughout the body. As such, experimentation on a single region may lead to an incomplete and potentially incorrect understanding of crucial underlying mechanisms.

Aim

Here, we aimed to develop and validate new tools to enable detailed and extended observation of neural and vascular activity in the brain and spinal cord. The first key set of innovations were targeted to developing novel imaging hardware that addresses the many challenges of multi-site imaging. The second key set of innovations were targeted to enabling bioluminescent imaging, as this approach can address limitations of fluorescent microscopy including photobleaching, phototoxicity and decreased resolution due to scattering of excitation signals.

Approach

We designed 3D-printed brain and spinal cord implants to enable effective surgical implantations and optical access with wearable miniscopes or an open window (e.g., for one- or two-photon microscopy or optogenetic stimulation). We also tested the viability for bioluminescent imaging, and developed a novel modified miniscope optimized for these signals (BLmini).

Results

Here, we describe novel 'universal' implants for acute and chronic simultaneous brain-spinal cord imaging and optical stimulation. We further describe successful imaging of bioluminescent signals in both foci, and a new miniscope, the 'BLmini,' which has reduced weight, cost and form-factor relative to standard wearable miniscopes.

Conclusions

The combination of 3D printed implants, advanced imaging tools, and bioluminescence imaging techniques offers a new coalition of methods for understanding spinal cord-brain interactions. This work has the potential for use in future research into neuropathic pain and other sensory disorders and motor behavior.

Synchronous Measurements of Extracellular Action Potentials and Neurochemical Activity with Carbon Fiber Electrodes in Nonhuman Primates

Measuring the dynamic relationship between neuromodulators, such as dopamine, and neuronal action potentials is imperative to understand how these fundamental modes of neural signaling interact to mediate behavior. Here, we developed methods to measure concurrently dopamine and extracellular action potentials (i.e., spikes) and applied these in a monkey performing a behavioral task. Standard fast-scan cyclic voltammetric (FSCV) electrochemical (EChem) and electrophysiological (EPhys) recording systems are combined and used to collect spike and dopamine signals, respectively, from an array of carbon fiber (CF) sensors implanted in the monkey striatum. FSCV requires the application of small voltages at the implanted sensors to measure redox currents generated from target molecules, such as dopamine. These applied voltages create artifacts at neighboring EPhys-measurement sensors, producing signals that may falsely be classified as physiological spikes. Therefore, simple automated temporal interpolation algorithms were designed to remove these artifacts and enable accurate spike extraction. We validated these methods using simulated artifacts and demonstrated an average spike recovery rate of 84.5%. This spike extraction was performed on data collected from concurrent EChem and EPhys recordings made in a task-performing monkey to discriminate cell-type specific striatal units. These identified units were shown to correlate to specific behavioral task parameters related to reward size and eye-movement direction. Synchronous measures of spike and dopamine signals displayed contrasting relations to the behavioral task parameters, as taken from our small set of representative data, suggesting a complex relationship between these two modes of neural signaling. Future application of our methods will help advance our understanding of the interactions between neuromodulator signaling and neuronal activity, to elucidate more detailed mechanisms of neural circuitry and plasticity mediating behaviors in health and in disease.

Scalable, flexible carbon fiber electrode thread arrays for three-dimensional probing of neurochemical activity in deep brain structures of rodents

We developed a flexible "electrode-thread" array for recording dopamine neurochemicals from a lateral distribution of subcortical targets (up to 16) transverse to the axis of insertion. Ultrathin (∼10 μm diameter) carbon fiber (CF) electrode-threads (CFETs) are clustered into a tight bundle to introduce them into the brain from a single-entry point. The individual CFETs splay laterally in deep brain tissue during insertion due to their innate flexibility. This spatial redistribution allows navigation of the CFETs towards deep brain targets spreading horizontally from the axis of insertion. Commercial "linear" arrays provide single-entry insertion but only allow measurements along the axis of insertion. Horizontally configured arrays inflict separate penetrations for each individual channel. We tested functional performance of our CFET arrays in vivo for recording dopamine and for providing lateral spread to multiple distributed sites in the rat striatum. Spatial spread was further characterized in agar brain phantoms as a function of insertion depth. We also developed protocols to slice the embedded CFETs within fixed brain tissue using standard histology. This method allowed extraction of the precise spatial coordinates of the implanted CFETs and their recording sites as integrated with immunohistochemical staining for surrounding anatomical, cytological, and protein expression labels. Our CFET array has the potential to unlock a wide range of applications, from uncovering the role of neuromodulators in synaptic plasticity, to addressing critical safety barriers in clinical translation towards diagnostic and adaptive treatment in Parkinson's disease and major mood disorders.