Dopamine neurons encode performance error in singing birds

Neuromodulatory processing in the bi-pathway brain architecture

The brain is inherently a complex and parallel system that processes both external and internal sensory cues to generate adaptive responses. Sensory cues encapsulate not only objective information about their physical and chemical properties but also subjective information related to their ecological significance. Objective information is processed and conveyed through relatively stereotyped sensorimotor pathways to drive behaviors, while subjective information is received and transmitted through relatively flexible neuromodulatory systems. These neuromodulatory pathways influence signal processing of the sensorimotor pathways at multiple stages by modulating neuronal excitability and the efficiency of synaptic transmission, thereby endowing animals with flexibility. This sophisticated neuromodulatory processing is finely tuned by the spatiotemporal dynamics of various neuromodulators released from specialized neuromodulatory neurons that encode sensory, motor as well as cognitive variables. Dysfunctions in neuromodulatory pathways disrupt spatiotemporal patterns of neuromodulators, which in turn compromise sensorimotor transformation and cognitive functions. This review aims to delineate the mechanisms and roles of neuromodulatory processing within the bi-pathway brain architecture and propose prospective research topics along with innovative experimental paradigms.

Dual neuromodulatory dynamics underlie birdsong learning

Although learning in response to extrinsic reinforcement is theorized to be driven by dopamine signals that encode the difference between expected and experienced rewards1,2, skills that enable verbal or musical expression can be learned without extrinsic reinforcement. Instead, spontaneous execution of these skills is thought to be intrinsically reinforcing3,4. Whether dopamine signals similarly guide learning of these intrinsically reinforced behaviours is unknown. In juvenile zebra finches learning from an adult tutor, dopamine signalling in a song-specialized basal ganglia region is required for successful song copying, a spontaneous, intrinsically reinforced process5. Here we show that dopamine dynamics in the song basal ganglia faithfully track the learned quality of juvenile song performance on a rendition-by-rendition basis. Furthermore, dopamine release in the basal ganglia is driven not only by inputs from midbrain dopamine neurons classically associated with reinforcement learning but also by song premotor inputs, which act by means of local cholinergic signalling to elevate dopamine during singing. Although both cholinergic and dopaminergic signalling are necessary for juvenile song learning, only dopamine tracks the learned quality of song performance. Therefore, dopamine dynamics in the basal ganglia encode performance quality during self-directed, long-term learning of natural behaviours.

Natural behaviour is learned through dopamine-mediated reinforcement

Many natural motor skills, such as speaking or locomotion, are acquired through a process of trial-and-error learning over the course of development. It has long been hypothesized, motivated by observations in artificial learning experiments, that dopamine has a crucial role in this process. Dopamine in the basal ganglia is thought to guide reward-based trial-and-error learning by encoding reward prediction errors1, decreasing after worse-than-predicted reward outcomes and increasing after better-than-predicted ones. Our previous work in adult zebra finches-in which we changed the perceived song quality with distorted auditory feedback-showed that dopamine in Area X, the singing-related basal ganglia, encodes performance prediction error: dopamine is suppressed after worse-than-predicted (distorted syllables) and activated after better-than-predicted (undistorted syllables) performance2. However, it remains unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Here we tracked song learning trajectories in juvenile zebra finches and used fibre photometry3 to monitor concurrent dopamine activity in Area X. We found that dopamine was activated after syllable renditions that were closer to the eventual adult version of the song, compared with recent renditions, and suppressed after renditions that were further away. Furthermore, the relationship between dopamine and song fluctuations revealed that dopamine predicted the future evolution of song, suggesting that dopamine drives behaviour. Finally, dopamine activity was explained by the contrast between the quality of the current rendition and the recent history of renditions-consistent with dopamine's hypothesized role in encoding prediction errors in an actor-critic reinforcement-learning model4,5. Reinforcement-learning algorithms6 have emerged as a powerful class of model to explain learning in reward-based laboratory tasks, as well as for driving autonomous learning in artificial intelligence7. Our results suggest that complex natural behaviours in biological systems can also be acquired through dopamine-mediated reinforcement learning.

The devilish details affecting TDRL models in dopamine research

Over recent decades, temporal difference reinforcement learning (TDRL) models have successfully explained much dopamine (DA) activity. This success has invited heightened scrutiny of late, with many studies challenging the validity of TDRL models of DA function. Yet, when evaluating the validity of these models, the devil is truly in the details. TDRL is a broad class of algorithms sharing core ideas but differing greatly in implementation and predictions. Thus, it is important to identify the defining aspects of the TDRL framework being tested and to use state spaces and model architectures that capture the known complexity of the behavioral representations and neural systems involved. Here, we discuss several examples that illustrate the importance of these considerations.

A mechanism linking dopamine's roles in reinforcement, movement and motivation

Dopamine neurons (DANs) play seemingly distinct roles in reinforcement, 1-3 motivation, 4,5 and movement, 6,7 and DA-modulating therapies relieve symptoms across a puzzling spectrum of neurologic and psychiatric symptoms. 8 Yet, the mechanistic relationship among these roles is unknown. Here, we show DA's tripartite roles are causally linked by a process in which phasic striatal DA rapidly and persistently recalibrates the propensity to move, a measure of vigor. Using a self-timed movement task, we found that single exposures to reward-related DA transients (both endogenous and exogenously-induced) exerted one-shot updates to movement timing-but in a surprising fashion. Rather than reinforce specific movement times, DA transients quantitatively changed movement timing on the next trial, with larger transients leading to earlier movements (and smaller to later), consistent with a stochastic search process that calibrates the frequency of movement. Both abrupt and gradual changes in external and internal contingencies-such as timing criterion, reward content, and satiety state-caused changes to the amplitude of DA transients that causally altered movement timing. The rapidity and bidirectionality of the one-shot effects are difficult to reconcile with gradual synaptic plasticity, and instead point to more flexible cellular mechanisms, such as DA-dependent modulation of neuronal excitability. Our findings shed light on how natural reinforcement, as well as DA-related disorders such as Parkinson's disease, could affect behavioral vigor.

Chaotic recurrent neural networks for brain modelling: A review

Even in the absence of external stimuli, the brain is spontaneously active. Indeed, most cortical activity is internally generated by recurrence. Both theoretical and experimental studies suggest that chaotic dynamics characterize this spontaneous activity. While the precise function of brain chaotic activity is still puzzling, we know that chaos confers many advantages. From a computational perspective, chaos enhances the complexity of network dynamics. From a behavioural point of view, chaotic activity could generate the variability required for exploration. Furthermore, information storage and transfer are maximized at the critical border between order and chaos. Despite these benefits, many computational brain models avoid incorporating spontaneous chaotic activity due to the challenges it poses for learning algorithms. In recent years, however, multiple approaches have been proposed to overcome this limitation. As a result, many different algorithms have been developed, initially within the reservoir computing paradigm. Over time, the field has evolved to increase the biological plausibility and performance of the algorithms, sometimes going beyond the reservoir computing framework. In this review article, we examine the computational benefits of chaos and the unique properties of chaotic recurrent neural networks, with a particular focus on those typically utilized in reservoir computing. We also provide a detailed analysis of the algorithms designed to train chaotic RNNs, tracing their historical evolution and highlighting key milestones in their development. Finally, we explore the applications and limitations of chaotic RNNs for brain modelling, consider their potential broader impacts beyond neuroscience, and outline promising directions for future research.

The barn owl in systems and behavioral neuroscience: Progress and promise

Though well-adapted to their evolutionary niches, animals exhibit a repertoire of behavioral functions that are common across species. Neuroscientific research that promotes the study of similar functions in multiple species, can illuminate shared versus specialized design features of the nervous system, revealing potentially profound insights into the neural basis of behavior and cognition. Here, we advance the idea that the barn owl is an excellent animal model in which to investigate such common functions. We do so by drawing attention to the range of exciting questions that can be asked in the owl beyond those deriving from its evolutionary specializations, by underscoring the variety of complex yet experimentally tractable behaviors it exhibits naturally, by emphasizing its complex network of brain systems, and by highlighting emerging opportunities for the application of modern neural technologies. Our goal is to motivate broader adoption of the powerful barn owl model for behavioral and systems neuroscience.

Context-dependent modulations in zebra finch distance calls revealed by a novel goal-directed vocalization paradigm

Songbirds are renowned for their complex vocal communication abilities; among them, zebra finches (Taeniopygia guttata) are a key species for studying vocal learning and communication. Zebra finches use various calls with different meanings, including the distance call, which is used for long-distance contact. Whether these calls are static with fixed meanings or flexible remains an open question. In this study we aimed to answer this question by designing a novel behavioral paradigm, in which we trained food-restricted zebra finches to use distance calls for food request. Nine out of ten birds learned this association and used their distance calls to obtain food when they were hungry. We then introduced a visually-separated audience and compared the distance calls used for food requests with those used for communication between birds. Our analyses revealed significant acoustic differences in power, pitch, and other spectral characteristics between the distance calls uttered in these two contexts, with calls directed at conspecifics exhibiting higher amplitude. Our findings suggest that zebra finches can use their distance call for different goals and also acoustically modulate it based on the context. Therefore, it demonstrates a level of vocal control thought to be exclusive to songs. This study enhances our understanding of vocal flexibility and its role in vocal communication.

Visually-guided compensation of deafening-induced song deterioration

Human language learning and maintenance depend primarily on auditory feedback but are also shaped by other sensory modalities. Individuals who become deaf after learning to speak (post-lingual deafness) experience a gradual decline in their language abilities. A similar process occurs in songbirds, where deafness leads to progressive song deterioration. However, songbirds can modify their songs using non-auditory cues, challenging the prevailing assumption that auditory feedback is essential for vocal control. In this study, we investigated whether deafened birds could use visual cues to prevent or limit song deterioration. We developed a new metric for assessing syllable deterioration called the spectrogram divergence score. We then trained deafened birds in a behavioral task where the spectrogram divergence score of a target syllable was computed in real-time, triggering a contingent visual stimulus based on the score. Birds exposed to the contingent visual stimulus-a brief light extinction-showed more stable song syllables than birds that received either no light extinction or randomly triggered light extinction. Notably, this effect was specific to the targeted syllable and did not influence other syllables. This study demonstrates that deafness-induced song deterioration in birds can be partially mitigated with visual cues.

Social context affects sequence modification learning in birdsong

Social interactions are crucial for imitative vocal learning such as human speech learning or song learning in songbirds. Recently, introducing specific learned modifications into adult song by experimenter-controlled reinforcement learning has emerged as a key protocol to study aspects of vocal learning in songbirds. This form of adult plasticity does not require conspecifics as a model for imitation or to provide social feedback on song performance. We therefore hypothesized that social interactions are irrelevant to, or even inhibit, song modification learning. We tested whether social context affects song sequence learning in adult male Bengalese finches (Lonchura striata domestica). We targeted specific syllable sequences in adult birds' songs with negative auditory feedback, which led the birds to reduce the targeted syllable sequence in favor of alternate sequences. Changes were apparent in catch trials without feedback, indicating a learning process. Each experiment was repeated within subjects with three different social contexts (male-male, MM; male-female, MF; and male alone, MA) in randomized order. We found robust learning in all three social contexts, with a nonsignificant trend toward facilitated learning with social company (MF, MM) compared to the single-housed (MA) condition. This effect could not be explained by the order of social contexts, nor by different singing rates across contexts. Our results demonstrate that social context can influence degree of learning in adult birds even in experimenter-controlled reinforcement learning tasks, and therefore suggest that social interactions might facilitate song plasticity beyond their known role for imitation and social feedback.

Bridging the fields of cognition and birdsong with corvids

Corvids, readily adaptable across social and ecological contexts, successfully inhabit almost the entire world. They are seen as highly intelligent birds, and current research examines their cognitive abilities. Despite being songbirds with a complete 'song system', corvids have historically received less attention in studies of song production, learning, and perception compared to non-corvid songbirds. However, recent neurobiological studies have demonstrated that songbird vocal production and its neuronal representations are regularly influenced by environmental and cognitive factors. This opinion article discusses the literature on 'corvid song' before introducing other flexible vocal behaviors of corvids in both the wild and controlled laboratory studies. We suggest corvids with their flexible vocal control as promising model species to study the links between brain networks for cognition and vocalization. Studying corvid vocal flexibility and associated cognitive processes in both ecological and lab settings offers complementary insights, crucial for bridging the fields of cognition and birdsong.

Exploring Anatomical Links Between the Crow's Nidopallium Caudolaterale and Its Song System

Crows are corvid songbirds that exhibit remarkable cognitive control, including their ability to vocalize on command. The activity of single neurons from the crow's associative telencephalic structure nidopallium caudolaterale (NCL) is correlated with the execution of this vocal and many non-vocal behaviors. However, whether anatomical connections directly link the crow NCL to its "song system" remains unclear. To address this, we used fluorescent tracers along with histological staining methods to characterize the connectivity of the crow's NCL in relation to its song system. Consistent with previous findings in other songbirds, we found that the NCL sends dense projections into the dorsal intermediate arcopallium (AID) directly adjacent to the song system's telencephalic motor output, the robust nucleus of the arcopallium (RA). Similarly, we demonstrate dense NCL projections into the striatum engulfing the basal ganglia song nucleus "area X." Both of these descending projections mirror the projections of the nidopallial song nucleus HVC (proper name) into RA and area X, with extremely sparse NCL fibers extending into area X. Furthermore, we characterized the distribution of cells projecting from the lateral part of the magnocellular nucleus of the anterior nidopallium (MAN) to NCL. Notably, a separate medial population of MAN cells projects to HVC. These two sets of connections-MAN to NCL and MAN to HVC-run in parallel but do not overlap. Taken together, our findings support the hypothesis that the NCL is part of a "general motor system" that parallels the song system but exhibits only minimal monosynaptic interconnections with it.

Delta opioid receptors affect acoustic features of song during vocal learning in zebra finches

Delta-opioid receptors (δ-ORs) are known to be involved in associative learning and modulating motivational states. We wanted to study if they were also involved in naturally-occurring reinforcement learning behaviors such as vocal learning, using the zebra finch model system. Zebra finches learn to vocalize early in development and song learning in males is affected by factors such as the social environment and internal reward, both of which are modulated by endogenous opioids. Pairs of juvenile male siblings (35-day-old) were systemically administered a δ-OR-selective antagonist naltrindole or vehicle (controls) for a period of 10 days. The acoustic structure of songs differed across treated and control groups at adulthood (120 days). Naltrindole-treated birds had a significantly lower pitch, mean frequency, and frequency modulation than controls, whereas there was no difference in the number of songs in naltrindole-treated and control siblings. Since the opioid and dopaminergic systems interact, we decided to study whether blocking δ-ORs during the sensitive period led to changes in dopaminoceptive neurons in Area X, a song control nucleus in the basal ganglia. Interestingly, compared with controls, naltrindole-treated birds had higher numbers of DARPP-32-positive medium spiny neurons and potentially excitatory synapses in Area X. We show that manipulating δ-OR signaling during the learning phase resulted in alterations in the acoustic features of song and had long term effects on dopaminergic targets within the basal ganglia in adulthood. Our results suggest that endogenous opioids regulate the development of cognitive processes and the underlying neural circuitry during the sensitive period for learning.

Weight Transfer in the Reinforcement Learning Model of Songbird Acquisition

Song acquisition behavior observed in the songbird system provides a notable example of learning through trial- and-error which parallels human speech acquisition. Studying songbird vocal learning can offer insights into mechanisms underlying human language. We present a computational model of song learning that integrates reinforcement learning (RL) and Hebbian learning and agrees with known songbird circuitry. The song circuit outputs activity from nucleus RA, which receives two primary inputs: timing information from area HVC and stochastic activity from nucleus LMAN. Additionally, song learning relies on Area X, a basal ganglia area that receives dopaminergic inputs from VTA. In our model, song is first acquired in the HVC-to-Area X connectivity, employing an RL mechanism that involves node perturbation. This information is then consolidated into HVC-to-RA synapses through a Hebbian mechanism. The transfer of weights from Area X to RA takes place via the thalamus, utilizing a specific form of spike-timing-dependent plasticity (STDP). Thus, we present a computational model grounded in songbird circuitry in which the optimal policy is initially guided by RL and subsequently transferred to another circuit through Hebbian plasticity.

The songbird connectome (OSCINE-NET.ORG): structure-function organization beyond the canonical vocal control network

BACKGROUND

Understanding the neural basis of behavior requires insight into how different brain systems coordinate with each other. Existing connectomes for various species have highlighted brain systems essential to various aspects of behavior, yet their application to complex learned behaviors remains limited. Research on vocal learning in songbirds has extensively focused on the vocal control network, though recent work implicates a variety of circuits in contributing to important aspects of vocal behavior. Thus, a more comprehensive understanding of brain-wide connectivity is essential to further assess the totality of circuitry underlying this complex learned behavior.

RESULTS

We present the Oscine Structural Connectome for Investigating NEural NETwork ORGanization (OSCINE-NET.ORG), the first interactive mesoscale connectome for any vocal learner. This comprehensive digital map includes all known connectivity data, covering major brain superstructures and functional networks. Our analysis reveals that the songbird brain exhibits small-world properties, with highly connected communities functionally designated as motor, visual, associative, vocal, social, and auditory. Moreover, there is a small set of significant connections across these communities, including from social and auditory sub-communities to vocal sub-communities, which highlight ethologically relevant facets of vocal learning and production. Notably, the vocal community contains the majority of the canonical vocal control network, as well as a variety of other nodes that are highly interconnected with it, meriting further evaluation for their inclusion in this network. A subset of nodes forms a "rich broker club," highly connected across the brain and forming a small circuit amongst themselves, indicating they may play a key role in information transfer broadly. Collectively, their bidirectional connectivity with multiple communities indicates they may act as liaisons across multiple functional circuits for a variety of complex behaviors.

CONCLUSIONS

OSCINE-NET.ORG offers unprecedented access to detailed songbird connectivity data, promoting insight into the neural circuits underlying complex behaviors. This data emphasizes the importance of brain-wide integration in vocal learning, facilitating a potential reevaluation of the canonical vocal control network. Furthermore, we computationally identify a small, previously unidentified circuit-one which may play an impactful role in brain-wide coordination of multiple complex behaviors.

Tapping into the vocal learning and rhythmic synchronization hypothesis

In this article, I present three main points that could benefit the "vocal learning and rhythmic synchronization hypothesis", encompassing neurogenetic mechanisms of gene expression transmission and single motor neuron function, classification of different behavioral motor phenotypes (e.g., spontaneous vs. voluntary), and other evolutionary considerations (i.e., the involvement of reward mechanisms).

Central nervous system disturbances by thiamethoxam in Japanese quail (Coturnix japonica): In vivo, ex vivo, and in silico study

The neurotoxic effects of neonicotinoids (NEOs) have been widely reported in relation to the poisoning of wild birds, yet the underlying molecular mechanism has remained elusive. This study employed Japanese quails (Coturnix japonica) and primary quail embryonic neurons as in vivo and ex vivo models, respectively, to investigate the neurotoxic effects and mechanism of thiamethoxam (TMX), a representative neonicotinoid insecticide, at environmentally relevant concentrations. Following a 28-day exposure to TMX, metabolomic analysis of quail brain revealed TMX-induced changes in glutamatergic, GABA-ergic, and dopaminergic function. Subsequent ex vivo and in silico experimentation revealed that the activation of nicotinic acetylcholine receptors and calcium signaling, induced by clothianidin (CLO), the primary metabolite of TMX, served as upstream events for the alterations in neurotransmitter synthesis, metabolism, release, and uptake. Our findings propose that the disruption of the central nervous system, caused by environmentally significant concentrations of NEOs, may account for the avian poisoning events induced by NEOs.

Motor cortex is responsible for motoric dynamics in striatum and the execution of both skilled and unskilled actions

Striatum and its predominant input, motor cortex, are responsible for the selection and performance of purposive movement, but how their interaction guides these processes is not understood. To establish its neural and behavioral contributions, we bilaterally lesioned motor cortex and recorded striatal activity and reaching performance daily, capturing the lesion's direct ramifications within hours of the intervention. We observed reaching impairment and an absence of striatal motoric activity following lesion of motor cortex, but not parietal cortex control lesions. Although some aspects of performance began to recover after 8-10 days, striatal projection and interneuronal dynamics did not-eventually entering a non-motor encoding state that aligned with persisting kinematic control deficits. Lesioned mice also exhibited a profound inability to switch motor plans while locomoting, reminiscent of clinical freezing of gait (FOG). Our results demonstrate the necessity of motor cortex in generating trained and untrained actions as well as striatal motoric dynamics.

Spatially organized striatal neuromodulator release encodes trajectory errors

Goal-directed navigation requires animals to continuously evaluate their current direction and speed of travel relative to landmarks to discern whether they are approaching or deviating from their goal. Striatal dopamine and acetylcholine are powerful modulators of goal-directed behavior, but it is unclear whether and how neuromodulator dynamics at landmarks incorporate relative motion for effective behavioral guidance. Using optical measurements in mice, we demonstrate that cue-evoked striatal dopamine release encodes bi-directional 'trajectory errors' reflecting relationships between ongoing speed and direction of locomotion and visual flow relative to optimal goal trajectories. Striatum-wide micro-fiber array recordings resolved an anatomical gradient of trajectory error signaling across the anterior-posterior axis, distinct from trajectory error independent cue signals. Dynamic regression modeling revealed that positive and negative trajectory error encoding emerges early and late respectively during learning and over different time courses in the medial and lateral striatum, enabling region specific contributions to learning. Striatal acetylcholine release also encodes trajectory errors, but encoding is more spatially restricted, opposite polarity, and delayed relative to dopamine, supporting distinct roles in modulating striatal output and behavior. Dopamine trajectory error signaling and task performance were reproduced in a reinforcement learning model incorporating a conjunctive state space representation, suggesting a potential neural substrate for trajectory error generation. Our results establish region specific neuromodulator signals positioned to guide the speed and direction of locomotion to reach goals based on environmental landmarks during navigation.

Tutor auditory memory for guiding sensorimotor learning in birdsong

Memory-guided motor shaping is necessary for sensorimotor learning. Vocal learning, such as speech development in human babies and song learning in bird juveniles, begins with the formation of an auditory template by hearing adult voices followed by vocally matching to the memorized template using auditory feedback. In zebra finches, the widely used songbird model system, only males develop individually unique stereotyped songs. The production of normal songs relies on auditory experience of tutor's songs (commonly their father's songs) during a critical period in development that consists of orchestrated auditory and sensorimotor phases. "Auditory templates" of tutor songs are thought to form in the brain to guide later vocal learning, while formation of "motor templates" of own song has been suggested to be necessary for the maintenance of stereotyped adult songs. Where these templates are formed in the brain and how they interact with other brain areas to guide song learning, presumably with template-matching error correction, remains to be clarified. Here, we review and discuss studies on auditory and motor templates in the avian brain. We suggest that distinct auditory and motor template systems exist that switch their functions during development.

Reinforcement Learning May Demystify the Limited Human Motor Learning Efficacy Due to Visual-Proprioceptive Mismatch

Vision and proprioception have fundamental sensory mismatches in delivering locational information, and such mismatches are critical factors limiting the efficacy of motor learning. However, it is still not clear how and to what extent this mismatch limits motor learning outcomes. To further the understanding of the effect of sensory mismatch on motor learning outcomes, a reinforcement learning algorithm and the simplified biomechanical elbow joint model were employed to mimic the motor learning process in a computational environment. By applying a reinforcement learning algorithm to the motor learning of elbow joint flexion task, simulation results successfully explained how visual-proprioceptive mismatch limits motor learning outcomes in terms of motor control accuracy and task completion speed. The larger the perceived angular offset between the two sensory modalities, the lower the motor control accuracy. Also, the more similar the peak reward amplitude of the two sensory modalities, the lower the motor control accuracy. In addition, simulation results suggest that insufficient exploration rate limits task completion speed, and excessive exploration rate limits motor control accuracy. Such a speed-accuracy trade-off shows that a moderate exploration rate could serve as another important factor in motor learning.

Striatal Dopamine Contributions to Skilled Motor Learning

Coordinated multijoint limb and digit movements-"manual dexterity"-underlie both specialized skills (e.g., playing the piano) and more mundane tasks (e.g., tying shoelaces). Impairments in dexterous skill cause significant disability, as occurs with motor cortical injury, Parkinson's disease, and a range of other pathologies. Clinical observations, as well as basic investigations, suggest that corticostriatal circuits play a critical role in learning and performing dexterous skills. Furthermore, dopaminergic signaling in these regions is implicated in synaptic plasticity and motor learning. Nonetheless, the role of striatal dopamine signaling in skilled motor learning remains poorly understood. Here, we use fiber photometry paired with a genetically encoded dopamine sensor to investigate striatal dopamine release in both male and female mice as they learn and perform a skilled reaching task. Dopamine rapidly increases during a skilled reach and peaks near pellet consumption. In the dorsolateral striatum, dopamine dynamics are faster than in the dorsomedial and ventral striatum. Across training, as reaching performance improves, dopamine signaling shifts from pellet consumption to cues that predict pellet availability, particularly in medial and ventral areas of the striatum. Furthermore, performance prediction errors are present across the striatum, with reduced dopamine release after an unsuccessful reach. These findings show that dopamine dynamics during skilled motor behaviors change with learning and are differentially regulated across striatal subregions.

A neuroethological view of the multifaceted sensory influences on birdsong

Learning and execution of complex motor skills are often modulated by sensory feedback and contextual cues arriving across multiple sensory modalities. Vocal motor behaviors, in particular, are primarily influenced by auditory inputs, both during learning and mature vocal production. The importance of auditory input in shaping vocal output has been investigated in several songbird species that acquire their adult song based on auditory exposure to a tutor during development. Recent studies have highlighted the influences of stimuli arriving through other sensory channels in juvenile song learning and in adult song production. Here, we review changes induced by diverse sensory stimuli during the song learning process and the production of adult song, considering the neuroethological significance of sensory channels in different species of songbirds. Additionally, we highlight advances, open questions, and possible future approaches for understanding the neural circuits that enable the multimodal shaping of singing behavior.

Bursts from the past: Intrinsic properties link a network model to zebra finch song

Neuronal intrinsic excitability is a mechanism implicated in learning and memory that is distinct from synaptic plasticity. Prior work in songbirds established that intrinsic properties (IPs) of premotor basal-ganglia-projecting neurons (HVC X ) relate to learned song. Here we find that temporal song structure is related to specific HVC X IPs: HVC X from birds who sang longer songs including longer invariant vocalizations (harmonic stacks) had IPs that reflected increased post-inhibitory rebound. This suggests a rebound excitation mechanism underlying the ability of HVC X neurons to integrate over long periods of time and represent sequence information. To explore this, we constructed a network model of realistic neurons showing how in-vivo HVC bursting properties link rebound excitation to network structure and behavior. These results demonstrate an explicit link between neuronal IPs and learned behavior. We propose that sequential behaviors exhibiting temporal regularity require IPs to be included in realistic network-level descriptions.

Dopamine transients follow a striatal gradient of reward time horizons

Animals make predictions to guide their behavior and update those predictions through experience. Transient increases in dopamine (DA) are thought to be critical signals for updating predictions. However, it is unclear how this mechanism handles a wide range of behavioral timescales-from seconds or less (for example, if singing a song) to potentially hours or more (for example, if hunting for food). Here we report that DA transients in distinct rat striatal subregions convey prediction errors based on distinct time horizons. DA dynamics systematically accelerated from ventral to dorsomedial to dorsolateral striatum, in the tempo of spontaneous fluctuations, the temporal integration of prior rewards and the discounting of future rewards. This spectrum of timescales for evaluative computations can help achieve efficient learning and adaptive motivation for a broad range of behaviors.

A neural circuit for vocal production responds to viscerosensory input in the songbird

Motor performance is monitored continuously by specialized brain circuits and used adaptively to modify behavior on a moment-to-moment basis and over longer time periods. During vocal behaviors, such as singing in songbirds, internal evaluation of motor performance relies on sensory input from the auditory and vocal-respiratory systems. Sensory input from the auditory system to the motor system, often referred to as auditory feedback, has been well studied in singing zebra finches (Taeniopygia guttata), but little is known about how and where nonauditory sensory feedback is evaluated. Here we show that brief perturbations in air sac pressure cause short-latency neural responses in the higher-order song control nucleus HVC (used as proper name), an area necessary for song learning and song production. Air sacs were briefly pressurized through a cannula in anesthetized or sedated adult male zebra finches, and neural responses were recorded in both nucleus parambigualis (PAm), a brainstem inspiratory center, and HVC, a cortical premotor nucleus. These findings show that song control nuclei in the avian song system are sensitive to perturbations directly targeted to vocal-respiratory, or viscerosensory, afferents and support a role for multimodal sensory feedback integration in modifying and controlling vocal control circuits.NEW & NOTEWORTHY This study presents the first evidence of sensory input from the vocal-respiratory periphery directly activating neurons in a motor circuit for vocal production in songbirds. It was previously thought that this circuit relies exclusively on sensory input from the auditory system, but we provide groundbreaking evidence for nonauditory sensory input reaching the higher-order premotor nucleus HVC, expanding our understanding of what sensory feedback may be available for vocal control.

Songbird mesostriatal dopamine pathways are spatially segregated before the onset of vocal learning

Diverse dopamine (DA) pathways send distinct reinforcement signals to different striatal regions. In adult songbirds, a DA pathway from the ventral tegmental area (VTA) to Area X, the striatal nucleus of the song system, carries singing-related performance error signals important for learning. Meanwhile, a parallel DA pathway to a medial striatal area (MST) arises from a distinct group of neighboring DA neurons that lack connectivity to song circuits and do not encode song error. To test if the structural and functional segregation of these two pathways depends on singing experience, we carried out anatomical studies early in development before the onset of song learning. We find that distinct VTA neurons project to either Area X or MST in juvenile birds before the onset of substantial vocal practice. Quantitative comparisons of early juveniles (30-35 days post hatch), late juveniles (60-65 dph), and adult (>90 dph) brains revealed an outsized expansion of Area X-projecting neurons relative to MST-projecting neurons in VTA over development. These results show that a mesostriatal DA system dedicated to social communication can exist and be spatially segregated before the onset of vocal practice and associated sensorimotor experience.

Topography of inputs into the hippocampal formation of a food-caching bird

The mammalian hippocampal formation (HF) is organized into domains associated with different functions. These differences are driven in part by the pattern of input along the hippocampal long axis, such as visual input to the septal hippocampus and amygdalar input to the temporal hippocampus. HF is also organized along the transverse axis, with different patterns of neural activity in the hippocampus and the entorhinal cortex. In some birds, a similar organization has been observed along both of these axes. However, it is not known what role inputs play in this organization. We used retrograde tracing to map inputs into HF of a food-caching bird, the black-capped chickadee. We first compared two locations along the transverse axis: the hippocampus and the dorsolateral hippocampal area (DL), which is analogous to the entorhinal cortex. We found that pallial regions predominantly targeted DL, while some subcortical regions like the lateral hypothalamus (LHy) preferentially targeted the hippocampus. We then examined the hippocampal long axis and found that almost all inputs were topographic along this direction. For example, the anterior hippocampus was preferentially innervated by thalamic regions, while the posterior hippocampus received more amygdalar input. Some of the topographies we found bear a resemblance to those described in the mammalian brain, revealing a remarkable anatomical similarity of phylogenetically distant animals. More generally, our work establishes the pattern of inputs to HF in chickadees. Some of these patterns may be unique to chickadees, laying the groundwork for studying the anatomical basis of these birds' exceptional hippocampal memory.

Dual credit assignment processes underlie dopamine signals in a complex spatial environment

Animals frequently make decisions based on expectations of future reward ("values"). Values are updated by ongoing experience: places and choices that result in reward are assigned greater value. Yet, the specific algorithms used by the brain for such credit assignment remain unclear. We monitored accumbens dopamine as rats foraged for rewards in a complex, changing environment. We observed brief dopamine pulses both at reward receipt (scaling with prediction error) and at novel path opportunities. Dopamine also ramped up as rats ran toward reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation of value along taken paths, as in temporal difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.

Dopaminergic error signals retune to social feedback during courtship

Hunger, thirst, loneliness and ambition determine the reward value of food, water, social interaction and performance outcome1. Dopamine neurons respond to rewards meeting these diverse needs2-8, but it remains unclear how behaviour and dopamine signals change as priorities change with new opportunities in the environment. One possibility is that dopamine signals for distinct drives are routed to distinct dopamine pathways9,10. Another possibility is that dopamine signals in a given pathway are dynamically tuned to rewards set by the current priority. Here we used electrophysiology and fibre photometry to test how dopamine signals associated with quenching thirst, singing a good song and courting a mate change as male zebra finches (Taeniopygia guttata) were provided with opportunities to retrieve water, evaluate song performance or court a female. When alone, water reward signals were observed in two mesostriatal pathways but singing-related performance error signals were routed to Area X, a striatal nucleus specialized for singing. When courting a female, water seeking was reduced and dopamine responses to both water and song performance outcomes diminished. Instead, dopamine signals in Area X were driven by female calls timed with the courtship song. Thus the dopamine system handled coexisting drives by routing vocal performance and social feedback signals to a striatal area for communication and by flexibly re-tuning to rewards set by the prioritized drive.

Dynamic top-down biasing implements rapid adaptive changes to individual movements

Complex behaviors depend on the coordinated activity of neural ensembles in interconnected brain areas. The behavioral function of such coordination, often measured as co-fluctuations in neural activity across areas, is poorly understood. One hypothesis is that rapidly varying co-fluctuations may be a signature of moment-by-moment task-relevant influences of one area on another. We tested this possibility for error-corrective adaptation of birdsong, a form of motor learning which has been hypothesized to depend on the top-down influence of a higher-order area, LMAN (lateral magnocellular nucleus of the anterior nidopallium), in shaping moment-by-moment output from a primary motor area, RA (robust nucleus of the arcopallium). In paired recordings of LMAN and RA in singing birds, we discovered a neural signature of a top-down influence of LMAN on RA, quantified as an LMAN-leading co-fluctuation in activity between these areas. During learning, this co-fluctuation strengthened in a premotor temporal window linked to the specific movement, sequential context, and acoustic modification associated with learning. Moreover, transient perturbation of LMAN activity specifically within this premotor window caused rapid occlusion of pitch modifications, consistent with LMAN conveying a temporally localized motor-biasing signal. Combined, our results reveal a dynamic top-down influence of LMAN on RA that varies on the rapid timescale of individual movements and is flexibly linked to contexts associated with learning. This finding indicates that inter-area co-fluctuations can be a signature of dynamic top-down influences that support complex behavior and its adaptation.

Context-dependent activation of a social behavior brain network during learned vocal production

Neural activation in brain regions for vocal control is social context dependent. This context-dependent brain activation reflects social context-appropriate vocal behavior but has unresolved mechanisms. Studies of non-vocal social behaviors in multiple organisms suggest a functional role for several evolutionarily conserved and highly interconnected brain regions. Here, we use neural activity-dependent gene expression to evaluate the functional connectivity of this social behavior network within zebra finches in non-social and social singing contexts. We found that activity in one social behavior network region, the medial preoptic area (POM), was strongly associated with the amount of non-social undirected singing in zebra finches. In addition, in all regions of the social behavior network and the paraventricular nucleus (PVN), a higher percentage of EGR1 expression was observed during a social female-directed singing context compared to a non-social undirected singing context. Furthermore, we observed distinct patterns of significantly correlated activity between regions of the social behavior network during non-social undirected and social female-directed singing. Our results suggest that non-social vs. social contexts differentially activate this social behavior network and PVN. Moreover, neuronal activity within this social behavior network, PVN, and POM may alter context-appropriate vocal production.

Does the syrinx, a peripheral structure, constrain effects of sex steroids on behavioral sex reversal in adult canaries?

We previously confirmed that effects of testosterone (T) on singing activity and on the volume of brain song control nuclei are sexually differentiated in adult canaries: females are limited in their ability to respond to T as males do. Here we expand on these results by focusing on sex differences in the production and performance of trills, i.e., rapid repetitions of song elements. We analyzed >42,000 trills recorded over a period of 6 weeks from 3 groups of castrated males and 3 groups of photoregressed females that received Silastic™ implants filled with T, T plus estradiol or left empty as control. Effects of T on the number of trills, trill duration and percent of time spent trilling were all stronger in males than females. Irrespective of endocrine treatment, trill performance assessed by vocal deviations from the trill rate versus trill bandwidth trade-off was also higher in males than in females. Finally, inter-individual differences in syrinx mass were positively correlated with specific features of trills in males but not in females. Given that T increases syrinx mass and syrinx fiber diameter in males but not in females, these data indicate that sex differences in trilling behavior are related to sex differences in syrinx mass and syrinx muscle fiber diameter that cannot be fully suppressed by sex steroids in adulthood. Sexual differentiation of behavior thus reflects organization not only of the brain but also of peripheral structures.

Dopamine injections to the midbrain periaqueductal gray inhibit vocal-motor production in a teleost fish

Across vertebrates, the midbrain periaqueductal gray (PAG) plays a critical role in social and vocal behavior. Dopaminergic neurotransmission also modulates these behaviors, and dopaminergic innervation of the PAG has been well documented. Nonetheless, the potential role of dopamine in shaping vocal production at the level of the PAG is not well understood. Here, we tested the hypothesis that dopamine modulates vocal production in the PAG, using a well-characterized vertebrate model system for the study of vocal communication, the plainfin midshipman fish, Porichthys notatus. We found that focal dopamine injections to the midshipman PAG rapidly and reversibly inhibited vocal production triggered by stimulation of known vocal-motor structures in the preoptic area / anterior hypothalamus. While dopamine inhibited vocal-motor output, it did not alter behaviorally-relevant parameters of this output, such as vocalization duration and frequency. Dopamine-induced inhibition of vocal production was prevented by the combined blockade of D1- and D2-like receptors but was unaffected by isolated blockade of either D1-receptors or D2-receptors. Our results suggest dopamine neuromodulation in the midshipman PAG may inhibit natural vocal behavior, in courtship and/or agonistic social contexts.

Learning critically drives parkinsonian motor deficits through imbalanced striatal pathway recruitment

Dopamine (DA) loss in Parkinson's disease (PD) causes debilitating motor deficits. However, dopamine is also widely linked to reward prediction and learning, and the contribution of dopamine-dependent learning to movements that are impaired in PD-which often do not lead to explicit rewards-is unclear. Here, we used two distinct motor tasks to dissociate dopamine's acute motoric effects vs. its long-lasting, learning-mediated effects. In dopamine-depleted mice, motor task performance gradually worsened with task exposure. Task experience was critical, as mice that remained in the home cage during the same period were relatively unimpaired when subsequently probed on the task. Repeated dopamine replacement treatments acutely rescued deficits and gradually induced long-term rescue that persisted despite treatment withdrawal. Surprisingly, both long-term rescue and parkinsonian performance decline were task specific, implicating dopamine-dependent learning. D1R activation potently induced acute rescue that gradually consolidated into long-term rescue. Conversely, reduced D2R activation potently induced parkinsonian decline. In dopamine-depleted mice, either D1R activation or D2R activation prevented parkinsonian decline, and both restored balanced activation of direct vs. indirect striatal pathways. These findings suggest that reinforcement and maintenance of movements-even movements not leading to explicit rewards-are fundamental functions of dopamine and provide potential mechanisms for the hitherto unexplained "long-duration response" by dopaminergic therapies in PD.

Dual credit assignment processes underlie dopamine signals in a complex spatial environment

Dopamine in the nucleus accumbens helps motivate behavior based on expectations of future reward ("values"). These values need to be updated by experience: after receiving reward, the choices that led to reward should be assigned greater value. There are multiple theoretical proposals for how this credit assignment could be achieved, but the specific algorithms that generate updated dopamine signals remain uncertain. We monitored accumbens dopamine as freely behaving rats foraged for rewards in a complex, changing environment. We observed brief pulses of dopamine both when rats received reward (scaling with prediction error), and when they encountered novel path opportunities. Furthermore, dopamine ramped up as rats ran towards reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation along taken paths, as in temporal-difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.

Topography of inputs into the hippocampal formation of a food-caching bird

The mammalian hippocampal formation (HF) is organized into domains associated with different functions. These differences are driven in part by the pattern of input along the hippocampal long axis, such as visual input to the septal hippocampus and amygdalar input to temporal hippocampus. HF is also organized along the transverse axis, with different patterns of neural activity in the hippocampus and the entorhinal cortex. In some birds, a similar organization has been observed along both of these axes. However, it is not known what role inputs play in this organization. We used retrograde tracing to map inputs into HF of a food-caching bird, the black-capped chickadee. We first compared two locations along the transverse axis: the hippocampus and the dorsolateral hippocampal area (DL), which is analogous to the entorhinal cortex. We found that pallial regions predominantly targeted DL, while some subcortical regions like the lateral hypothalamus (LHy) preferentially targeted the hippocampus. We then examined the hippocampal long axis and found that almost all inputs were topographic along this direction. For example, the anterior hippocampus was preferentially innervated by thalamic regions, while posterior hippocampus received more amygdalar input. Some of the topographies we found bear resemblance to those described in the mammalian brain, revealing a remarkable anatomical similarity of phylogenetically distant animals. More generally, our work establishes the pattern of inputs to HF in chickadees. Some of these patterns may be unique to chickadees, laying the groundwork for studying the anatomical basis of these birds ’ exceptional hippocampal memory.

Unraveling the dynamics of dopamine release and its actions on target cells

The neuromodulator dopamine (DA) is essential for regulating learning, motivation, and movement. Despite its importance, however, the mechanisms by which DA influences the activity of target cells to alter behavior remain poorly understood. In this review, we describe recent methodological advances that are helping to overcome challenges that have historically hindered the field. We discuss how the employment of these methods is shedding light on the complex dynamics of extracellular DA in the brain, as well as how DA signaling alters the electrical, biochemical, and population activity of target neurons in vivo. These developments are generating novel hypotheses about the mechanisms through which DA release modifies behavior.

Context-dependent activation of a social behavior brain network associates with learned vocal production

In zebra finches, an avian brain network for vocal control undergoes context-dependent patterning of song-dependent activation. Previous studies in zebra finches also implicate the importance of dopaminergic input in producing context-appropriate singing behavior. In mice, it has been shown that oxytocinergic neurons originated in the paraventricular nucleus of the hypothalamus (PVN) synapse directly onto dopamine neurons in the ventral tegmental area (VTA), implicating the necessity of oxytocin signaling from the PVN for producing a context-appropriate song. Both avian and non-avian axonal tract-tracing studies indicate high levels of PVN innervation by the social behavior network. Here, we hypothesize that the motivation for PVN oxytocin neurons to trigger dopamine release originates in the social behavior network, a highly conserved and interconnected collection of six regions implicated in various social and homeostatic behaviors. We found that expression of the neuronal activity marker EGR1 was not strongly correlated with song production in any of the regions of the social behavior network. However, when EGR1 expression levels were normalized to the singing rate, we found significantly higher levels of expression in the social behavior network regions except the medial preoptic area during a social female-directed singing context compared to a non-social undirected singing context. Our results suggest neuronal activity within the male zebra finch social behavior network influences the synaptic release of oxytocin from PVN onto dopaminergic projection neurons in the VTA, which in turn signals to the vocal control network to allow for context-appropriate song production.

A dopaminergic reward prediction error signal shapes maternal behavior in mice

How social contact is perceived as rewarding and subsequently modifies interactions is unclear. Dopamine (DA) from the ventral tegmental area (VTA) regulates sociality, but the ongoing, unstructured nature of free behavior makes it difficult to ascertain how. Here, we tracked the emergence of a repetitive stereotyped parental retrieval behavior and conclude that VTA DA neurons incrementally refine it by reinforcement learning (RL). Trial-by-trial performance was correlated with the history of DA neuron activity, but DA signals were inconsistent with VTA directly influencing the current trial. We manipulated the subject's expectation of imminent pup contact and show that DA signals convey reward prediction error, a fundamental component of RL. Finally, closed-loop optogenetic inactivation of DA neurons at the onset of pup contact dramatically slowed emergence of parental care. We conclude that this component of maternal behavior is shaped by an RL mechanism in which social contact itself is the primary reward.

Developmental treatments with Δ9- tetrahydrocannabinol and the MAGL inhibitor JZL184 persistently alter adult cocaine conditioning in contrasting ways

Using a songbird, zebra finches, as a developmental drug abuse model we found previously that cannabinoid agonists administered during the sensorimotor period of vocal learning (50-75 days of age) persistently alter song patterns and cocaine responsiveness in adulthood. However, these effects were not produced in adults exposed to similar treatment regimens. Currently, we have used the MAGL inhibitor, JZL184, to test whether enhanced endocannabinoid signaling may similarly alter cocaine responsiveness. We found that, as expected and consistent with prior results, repeated developmental (but not adult) treatments with Δ⁹-tetrahydrocannabinol (THC, 3 mg/kg QD IM) resulted in increased time spent in cocaine-paired chambers. Unexpectedly and in contrast, repeated developmental JZL184 (4 mg/kg QD IM) treatments decreased time spent in cocaine-conditioned chambers. That is, young finches repeatedly treated with JZL184 avoided cocaine-paired chambers later in adulthood, while similar development treatments with THC had the opposite effect. To begin to identify brain regions that may underly this differential responsiveness we used c-Fos expression as a marker of neuronal activity. Differences in c-Fos expression patterns following placement of cocaine-conditioned finches into vehicle- vs. cocaine-paired chambers suggest distinct involvement of circuits through striatal and amygdaloid regions in respective effects of THC and JZL184. Results demonstrate that, like exogenous cannabinoid exposure, inhibition of MAGL activity during late post-natal development persistently alters behavior in adulthood. Contrasting effects of THC vs. MAGL inhibition with JZL184 suggests the latter alters development of brain regions to favor promotion of aversive rather than appetitive cocaine responsiveness later in adulthood.

Spontaneous behaviour is structured by reinforcement without explicit reward

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.

Mesolimbic dopamine adapts the rate of learning from action

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions^1-3. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction⁴; however, so far there has been little consideration of how direct policy learning might inform our understanding⁵. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning⁶.

Task parameters influence operant response variability in mice

RATIONALE

During operant conditioning, animals associate actions with outcomes. However, patterns and rates of operant responding change over learning, which makes it difficult to distinguish changes in learning from general changes in performance or movement. Thus, understanding how task parameters influence movement execution is essential.

OBJECTIVES

To understand how specific operant task parameters influenced the repetition of future operant responses, we investigated the ability of operant conditioning schedules and contingencies to promote reproducible bouts of five lever presses in mice.

METHODS

Mice were trained on one of the four operant tasks to test three distinct hypotheses: (1) whether a cue presented concurrently with sucrose delivery influenced the pattern of lever pressing; (2) whether requiring animals to collect earned sucrose promoted the organization of responses into bouts; and (3) whether only reinforcing bouts where interresponse time (IRT) variances were below a target promoted reproducible patterns of operant behavior.

RESULTS

(1) Signaling reinforcer delivery with a cue increased learning rates but resulted in mice pressing the lever in fast succession until the cue turned on, rather than executing discrete bouts. (2) Requiring mice to collect the reinforcer between bouts had little effect on behavior. (3) A training strategy that directly reinforced bouts with low variance IRTs was not more effective than a traditional fixed ratio schedule at promoting reproducible action execution.

CONCLUSIONS

Together, our findings provide insights into the parameters of behavioral training that promote reproducible actions and that should be carefully selected when designing operant conditioning experiments.

Music and Brain Circuitry: Strategies for Strengthening Evidence-Based Research for Music-Based Interventions

The neuroscience of music and music-based interventions (MBIs) is a fascinating but challenging research field. While music is a ubiquitous component of every human society, MBIs may encompass listening to music, performing music, music-based movement, undergoing music education and training, or receiving treatment from music therapists. Unraveling the brain circuits activated and influenced by MBIs may help us gain better understanding of the therapeutic and educational values of MBIs by gathering strong research evidence. However, the complexity and variety of MBIs impose unique research challenges. This article reviews the recent endeavor led by the National Institutes of Health to support evidence-based research of MBIs and their impact on health and diseases. It also highlights fundamental challenges and strategies of MBI research with emphases on the utilization of animal models, human brain imaging and stimulation technologies, behavior and motion capturing tools, and computational approaches. It concludes with suggestions of basic requirements when studying MBIs and promising future directions to further strengthen evidence-based research on MBIs in connections with brain circuitry.SIGNIFICANCE STATEMENT Music and music-based interventions (MBI) engage a wide range of brain circuits and hold promising therapeutic potentials for a variety of health conditions. Comparative studies using animal models have helped in uncovering brain circuit activities involved in rhythm perception, while human imaging, brain stimulation, and motion capture technologies have enabled neural circuit analysis underlying the effects of MBIs on motor, affective/reward, and cognitive function. Combining computational analysis, such as prediction method, with mechanistic studies in animal models and humans may unravel the complexity of MBIs and their effects on health and disease.

Motor constellation theory: A model of infants' phonological development

Every normally developing human infant solves the difficult problem of mapping their native-language phonology, but the neural mechanisms underpinning this behavior remain poorly understood. Here, motor constellation theory, an integrative neurophonological model, is presented, with the goal of explicating this issue. It is assumed that infants' motor-auditory phonological mapping takes place through infants' orosensory "reaching" for phonological elements observed in the language-specific ambient phonology, via reference to kinesthetic feedback from motor systems (e.g., articulators), and auditory feedback from resulting speech and speech-like sounds. Attempts are regulated by basal ganglion-cerebellar speech neural circuitry, and successful attempts at reproduction are enforced through dopaminergic signaling. Early in life, the pace of anatomical development constrains mapping such that complete language-specific phonological mapping is prohibited by infants' undeveloped supralaryngeal vocal tract and undescended larynx; constraints gradually dissolve with age, enabling adult phonology. Where appropriate, reference is made to findings from animal and clinical models. Some implications for future modeling and simulation efforts, as well as clinical settings, are also discussed.

Birdsong

Have your ever felt as happy as a lark, feathered your nest or taken someone under your wing? As we watch birds, we cannot help but be struck by their uncannily familiar behaviors - singing, nest building, caring for their young - to name just a few. Songbirds - the oscine suborder of perching birds that constitute roughly half (∼4,000) of all known avian species - are noted for the songs that males and sometimes both sexes in this group sing to court mates and defend territory from rivals. Birdsongs contain several to many acoustically distinct syllables, typically organized into a stereotyped phrase, and span the same audio bandwidth that we exploit for speech and music, making them easy for us to hear and appreciate. Consequently, eavesdropping humans long ago detected the most striking parallel between songbirds and humans: juvenile songbirds learn to sing in a manner similar to a child learning to speak.

Nigrostriatal dopamine pathway regulates auditory discrimination behavior

The auditory striatum, the tail portion of dorsal striatum in basal ganglia, is implicated in perceptual decision-making, transforming auditory stimuli to action outcomes. Despite its known connections to diverse neurological conditions, the dopaminergic modulation of sensory striatal neuronal activity and its behavioral influences remain unknown. We demonstrated that the optogenetic inhibition of dopaminergic projections from the substantia nigra pars compacta to the auditory striatum specifically impairs mouse choice performance but not movement in an auditory frequency discrimination task. In vivo dopamine and calcium imaging in freely behaving mice revealed that this dopaminergic projection modulates striatal tone representations, and tone-evoked striatal dopamine release inversely correlated with the evidence strength of tones. Optogenetic inhibition of D1-receptor expressing neurons and pharmacological inhibition of D1 receptors in the auditory striatum dampened choice performance accuracy. Our study uncovers a phasic mechanism within the nigrostriatal system that regulates auditory decisions by modulating ongoing auditory perception.

The auditory striatum, the tail portion of dorsal striatum, is implicated in decision-making. This study uncovers a phasic mechanism within the nigrostriatal system that regulates auditory decisions by modulating ongoing auditory perception.

Shared mechanisms of auditory and non-auditory vocal learning in the songbird brain

Songbirds and humans share the ability to adaptively modify their vocalizations based on sensory feedback. Prior studies have focused primarily on the role that auditory feedback plays in shaping vocal output throughout life. In contrast, it is unclear how non-auditory information drives vocal plasticity. Here, we first used a reinforcement learning paradigm to establish that somatosensory feedback (cutaneous electrical stimulation) can drive vocal learning in adult songbirds. We then assessed the role of a songbird basal ganglia thalamocortical pathway critical to auditory vocal learning in this novel form of vocal plasticity. We found that both this circuit and its dopaminergic inputs are necessary for non-auditory vocal learning, demonstrating that this pathway is critical for guiding adaptive vocal changes based on both auditory and somatosensory signals. The ability of this circuit to use both auditory and somatosensory information to guide vocal learning may reflect a general principle for the neural systems that support vocal plasticity across species.