Dopamine neurons encode performance error in singing birds

ZEBrA: Zebra finch Expression Brain Atlas-A resource for comparative molecular neuroanatomy and brain evolution studies

An in-depth understanding of the genetics and evolution of brain function and behavior requires a detailed mapping of gene expression in functional brain circuits across major vertebrate clades. Here we present the Zebra finch Expression Brain Atlas (ZEBrA; www.zebrafinchatlas.org, RRID: SCR_012988), a web-based resource that maps the expression of genes linked to a broad range of functions onto the brain of zebra finches. ZEBrA is a first of its kind gene expression brain atlas for a bird species and a first for any sauropsid. ZEBrA's >3,200 high-resolution digital images of in situ hybridized sections for ~650 genes (as of June 2019) are presented in alignment with an annotated histological atlas and can be browsed down to cellular resolution. An extensive relational database connects expression patterns to information about gene function, mouse expression patterns and phenotypes, and gene involvement in human diseases and communication disorders. By enabling brain-wide gene expression assessments in a bird, ZEBrA provides important substrates for comparative neuroanatomy and molecular brain evolution studies. ZEBrA also provides unique opportunities for linking genetic pathways to vocal learning and motor control circuits, as well as for novel insights into the molecular basis of sex steroids actions, brain dimorphisms, reproductive and social behaviors, sleep function, and adult neurogenesis, among many fundamental themes.

The neurobiology of innate and learned vocalizations in rodents and songbirds

Vocalizations are an important medium for sexual and social signaling in mammals and birds. In most mammals other than humans, vocalizations are specified by innate mechanisms and develop normally in the absence of auditory experience. By contrast, juvenile songbirds memorize and copy the songs of adult tutors, a process with many parallels to human speech learning. Despite the centrality of vocal learning to human speech, vocal production in humans as well as in songbirds exploits ancestral circuitry for innate vocalizations, and effective vocal communication depends on the fluent blending of innate and learned elements. This review covers recent advances in our understanding of central mechanisms for learned and innate vocalizations in birds and mice, including brainstem mechanisms that help to 'gate' vocalizations on or off, cortical involvement in learned and innate vocalizations, and the delineation of circuits that evaluate and reinforce song performance to facilitate vocal learning.

Gene manipulation to test links between genome, brain and behavior in developing songbirds: a test case

Songbird research has made many seminal contributions to the fields of ethology, endocrinology, physiology, ecology, evolution and neurobiology. Genome manipulation is thus a promising new methodological strategy to enhance the existing strengths of the songbird system to advance and expand fundamental knowledge of how genetic sequences and regulation of genomic function support complex natural learned behaviors. In zebra finches (Taeniopygia guttata) in particular, a rich set of questions about the complex process of developmental song learning in juvenile males has been defined. This Review uses one area of zebra finch song learning to demonstrate how genome editing can advance causal investigations into known genome-brain-behavior relationships. Given the number and diversity of songbird species, comparative work leveraging genome manipulation would expand the influence of these birds in additional fields of ecology and evolution for song learning and other behaviors.

Involvement of the Cortico-Basal Ganglia-Thalamocortical Loop in Developmental Stuttering

Stuttering is a complex neurodevelopmental disorder that has to date eluded a clear explication of its pathophysiological bases. In this review, we utilize the Directions Into Velocities of Articulators (DIVA) neurocomputational modeling framework to mechanistically interpret relevant findings from the behavioral and neurological literatures on stuttering. Within this theoretical framework, we propose that the primary impairment underlying stuttering behavior is malfunction in the cortico-basal ganglia-thalamocortical (hereafter, cortico-BG) loop that is responsible for initiating speech motor programs. This theoretical perspective predicts three possible loci of impaired neural processing within the cortico-BG loop that could lead to stuttering behaviors: impairment within the basal ganglia proper; impairment of axonal projections between cerebral cortex, basal ganglia, and thalamus; and impairment in cortical processing. These theoretical perspectives are presented in detail, followed by a review of empirical data that make reference to these three possibilities. We also highlight any differences that are present in the literature based on examining adults versus children, which give important insights into potential core deficits associated with stuttering versus compensatory changes that occur in the brain as a result of having stuttered for many years in the case of adults who stutter. We conclude with outstanding questions in the field and promising areas for future studies that have the potential to further advance mechanistic understanding of neural deficits underlying persistent developmental stuttering.

Interfacing behavioral and neural circuit models for habit formation

Habits are an important mechanism by which organisms can automate the control of behavior to alleviate cognitive demand. However, transitions to habitual control are risky because they lead to inflexible responding in the face of change. The question of how the brain controls transitions into habit is thus an intriguing one. How do we regulate when our repeated actions become automated? When is it advantageous or disadvantageous to release actions from cognitive control? Decades of research have identified a variety of methods for eliciting habitual responding in animal models. Progress has also been made to understand which brain areas and neural circuits control transitions into habit. Here, I discuss existing research on behavioral and neural circuit models for habit formation (with an emphasis on striatal circuits), and discuss strategies for combining information from different paradigms and levels of analysis to prompt further progress in the field.

Memory circuits for vocal imitation

Many complex behaviors exhibited by social species are first learned by imitating the behavior of other more experienced individuals. Speech and language are the most widely appreciated behaviors learned in this way. Vocal imitation in songbirds is perhaps the best studied socially transmitted behavior, and research over the past few years has begun to crack the circuit mechanisms for how songbirds learn from vocal models. Studies in zebra finches are revealing an unexpected and essential role for premotor cortical circuits in forming the behavioral-goal memories used to guide song imitation, challenging the view that song memories used for imitation are stored in auditory circuits. Here, we provide a summary of this recent progress focusing on the What, Where, and How of tutor song memory, and propose a circuit hypothesis for song learning based on these recent findings.

Differential Song Deficits after Lentivirus-Mediated Knockdown of FoxP1, FoxP2, or FoxP4 in Area X of Juvenile Zebra Finches

Mutations in the transcription factors FOXP1 and FOXP2 are associated with speech impairments. FOXP1 is additionally linked to cognitive deficits, as is FOXP4. These FoxP proteins are highly conserved in vertebrates and expressed in comparable brain regions, including the striatum. In male zebra finches, experimental manipulation of FoxP2 in Area X, a striatal song nucleus essential for vocal production learning, affects song development, adult song production, dendritic spine density, and dopamine-regulated synaptic transmission of striatal neurons. We previously showed that, in the majority of Area X neurons FoxP1, FoxP2, and FoxP4 are coexpressed, can dimerize and multimerize with each other and differentially regulate the expression of target genes. These findings raise the possibility that FoxP1, FoxP2, and FoxP4 (FoxP1/2/4) affect neural function differently and in turn vocal learning. To address this directly, we downregulated FoxP1 or FoxP4 in Area X of juvenile zebra finches and compared the resulting song phenotypes with the previously described inaccurate and incomplete song learning after FoxP2 knockdown. We found that experimental downregulation of FoxP1 and FoxP4 led to impaired song learning with partly similar features as those reported for FoxP2 knockdowns. However, there were also specific differences between the groups, leading us to suggest that specific features of the song are differentially impacted by developmental manipulations of FoxP1/2/4 expression in Area X.SIGNIFICANCE STATEMENT We compared the effects of experimentally reduced expression of the transcription factors FoxP1, FoxP2, and FoxP4 in a striatal song nucleus, Area X, on vocal production learning in juvenile male zebra finches. We show, for the first time, that these temporally and spatially precise manipulations of the three FoxPs affect spectral and temporal song features differentially. This is important because it raises the possibility that the different FoxPs control different aspects of vocal learning through combinatorial gene expression or by acting in different microcircuits within Area X. These results are consistent with the deleterious effects of human FOXP1 and FOXP2 mutations on speech and language and add FOXP4 as a possible candidate gene for vocal disorders.

Corticobasal ganglia projecting neurons are required for juvenile vocal learning but not for adult vocal plasticity in songbirds

Birdsong, like human speech, consists of a sequence of temporally precise movements acquired through vocal learning. The learning of such sequential vocalizations depends on the neural function of the motor cortex and basal ganglia. However, it is unknown how the connections between cortical and basal ganglia components contribute to vocal motor skill learning, as mammalian motor cortices serve multiple types of motor action and most experimentally tractable animals do not exhibit vocal learning. Here, we leveraged the zebra finch, a songbird, as an animal model to explore the function of the connectivity between cortex-like (HVC) and basal ganglia (area X), connected by HVC_(X) projection neurons with temporally precise firing during singing. By specifically ablating HVC_(X) neurons, juvenile zebra finches failed to copy tutored syllable acoustics and developed temporally unstable songs with less sequence consistency. In contrast, HVC_(X)-ablated adults did not alter their learned song structure, but generated acoustic fluctuations and responded to auditory feedback disruption by the introduction of song deterioration, as did normal adults. These results indicate that the corticobasal ganglia input is important for learning the acoustic and temporal aspects of song structure, but not for generating vocal fluctuations that contribute to the maintenance of an already learned vocal pattern.

Adaptive Regulation of Motor Variability

Trial-to-trial movement variability can both drive motor learning and interfere with expert performance, suggesting benefits of regulating it in context-specific ways. Here we address whether and how the brain regulates motor variability as a function of performance by training rats to execute ballistic forelimb movements for reward. Behavioral datasets comprising millions of trials revealed that motor variability is regulated by two distinct processes. A fast process modulates variability as a function of recent trial outcomes, increasing it when performance is poor and vice versa. A slower process tunes the gain of the fast process based on the uncertainty in the task's reward landscape. Simulations demonstrated that this regulation strategy optimizes reward accumulation over a wide range of time horizons, while also promoting learning. Our results uncover a sophisticated algorithm implemented by the brain to adaptively regulate motor variability to improve task performance. VIDEO ABSTRACT.

A Modular Approach to Vocal Learning: Disentangling the Diversity of a Complex Behavioral Trait

Vocal learning is a behavioral trait in which the social and acoustic environment shapes the vocal repertoire of individuals. Over the past century, the study of vocal learning has progressed at the intersection of ecology, physiology, neuroscience, molecular biology, genomics, and evolution. Yet, despite the complexity of this trait, vocal learning is frequently described as a binary trait, with species being classified as either vocal learners or vocal non-learners. As a result, studies have largely focused on a handful of species for which strong evidence for vocal learning exists. Recent studies, however, suggest a continuum in vocal learning capacity across taxa. Here, we further suggest that vocal learning is a multi-component behavioral phenotype comprised of distinct yet interconnected modules. Discretizing the vocal learning phenotype into its constituent modules would facilitate integration of findings across a wider diversity of species, taking advantage of the ways in which each excels in a particular module, or in a specific combination of features. Such comparative studies can improve understanding of the mechanisms and evolutionary origins of vocal learning. We propose an initial set of vocal learning modules supported by behavioral and neurobiological data and highlight the need for diversifying the field in order to disentangle the complexity of the vocal learning phenotype.

Volitional control of vocalizations in corvid songbirds

Songbirds are renowned for their acoustically elaborate songs. However, it is unclear whether songbirds can cognitively control their vocal output. Here, we show that crows, songbirds of the corvid family, can be trained to exert control over their vocalizations. In a detection task, three male carrion crows rapidly learned to emit vocalizations in response to a visual cue with no inherent meaning (go trials) and to withhold vocalizations in response to another cue (catch trials). Two of these crows were then trained on a go/nogo task, with the cue colors reversed, in addition to being rewarded for withholding vocalizations to yet another cue (nogo trials). Vocalizations in response to the detection of the go cue were temporally precise and highly reliable in all three crows. Crows also quickly learned to withhold vocal output in nogo trials, showing that vocalizations were not produced by an anticipation of a food reward in correct trials. The results demonstrate that corvids can volitionally control the release and onset of their vocalizations, suggesting that songbird vocalizations are under cognitive control and can be decoupled from affective states.

Songbirds are renowned for their acoustically elaborate songs, but it is unclear whether they have cognitive control over their vocal output. Using operant conditioning, this study shows that carrion crows, songbirds of the corvid family, can exert control over their vocalizations.

Discrete Evaluative and Premotor Circuits Enable Vocal Learning in Songbirds

Virtuosic motor performance requires the ability to evaluate and modify individual gestures within a complex motor sequence. Where and how the evaluative and premotor circuits operate within the brain to enable such temporally precise learning is poorly understood. Songbirds can learn to modify individual syllables within their complex vocal sequences, providing a system for elucidating the underlying evaluative and premotor circuits. We combined behavioral and optogenetic methods to identify 2 afferents to the ventral tegmental area (VTA) that serve evaluative roles in syllable-specific learning and to establish that downstream cortico-basal ganglia circuits serve a learning role that is only premotor. Furthermore, song performance-contingent optogenetic stimulation of either VTA afferent was sufficient to drive syllable-specific learning, and these learning effects were of opposite valence. Finally, functional, anatomical, and molecular studies support the idea that these evaluative afferents bidirectionally modulate VTA dopamine neurons to enable temporally precise vocal learning.

Dopamine Depletion Affects Vocal Acoustics and Disrupts Sensorimotor Adaptation in Songbirds

Dopamine is hypothesized to convey error information in reinforcement learning tasks with explicit appetitive or aversive cues. However, during motor skill learning feedback signals arise from an animal's evaluation of sensory feedback resulting from its own behavior, rather than any external reward or punishment. It has previously been shown that intact dopaminergic signaling from the ventral tegmental area/substantia nigra pars compacta (VTA/SNc) complex is necessary for vocal learning when songbirds modify their vocalizations to avoid hearing distorted auditory feedback (playbacks of white noise). However, it remains unclear whether dopaminergic signaling underlies vocal learning in response to more naturalistic errors (pitch-shifted feedback delivered via headphones). We used male Bengalese finches (Lonchura striata var. domestica) to test the hypothesis that the necessity of dopamine signaling is shared between the two types of learning. We combined 6-hydroxydopamine (6-OHDA) lesions of dopaminergic terminals within Area X, a basal ganglia nucleus critical for song learning, with a headphones learning paradigm that shifted the pitch of auditory feedback and compared their learning to that of unlesioned controls. We found that 6-OHDA lesions affected song behavior in two ways. First, over a period of days lesioned birds systematically lowered their pitch regardless of the presence or absence of auditory errors. Second, 6-OHDA lesioned birds also displayed severe deficits in sensorimotor learning in response to pitch-shifted feedback. Our results suggest roles for dopamine in both motor production and auditory error processing, and a shared mechanism underlying vocal learning in response to both distorted and pitch-shifted auditory feedback.

Songbird Ventral Pallidum Sends Diverse Performance Error Signals to Dopaminergic Midbrain

Motor skills improve with practice, requiring outcomes to be evaluated against ever-changing performance benchmarks, yet it remains unclear how performance error signals are computed. Here, we show that the songbird ventral pallidum (VP) is required for song learning and sends diverse song timing and performance error signals to the ventral tegmental area (VTA). Viral tracing revealed inputs to VP from auditory and vocal motor thalamus, auditory and vocal motor cortex, and VTA. Our findings show that VP circuits, commonly associated with hedonic functions, signal performance error during motor sequence learning.

Beyond Critical Period Learning: Striatal FoxP2 Affects the Active Maintenance of Learned Vocalizations in Adulthood

In humans, mutations in the transcription factor forkhead box P2 (FOXP2) result in language disorders associated with altered striatal structure. Like speech, birdsong is learned through social interactions during maturational critical periods, and it relies on auditory feedback during initial learning and on-going maintenance. Hearing loss causes learned vocalizations to deteriorate in adult humans and songbirds. In the adult songbird brain, most FoxP2-enriched regions (e.g., cortex, thalamus) show a static expression level, but in the striatal song control nucleus, area X, FoxP2 is regulated by singing and social context: when juveniles and adults sing alone, its levels drop, and songs are more variable. When males sing to females, FoxP2 levels remain high, and songs are relatively stable: this “on-line” regulation implicates FoxP2 in ongoing vocal processes, but its role in the auditory-based maintenance of learned vocalization has not been examined. To test this, we overexpressed FoxP2 in both hearing and deafened adult zebra finches and assessed effects on song sung alone versus songs directed to females. In intact birds singing alone, no changes were detected between songs of males expressing FoxP2 or a GFP construct in area X, consistent with the marked stability of mature song in this species. In contrast, songs of males overexpressing FoxP2 became more variable and were less preferable to females, unlike responses to songs of GFP-expressing control males. In deafened birds, song deteriorated more rapidly following FoxP2 overexpression relative to GFP controls. Together, these experiments suggest that behavior-driven FoxP2 expression and auditory feedback interact to precisely maintain learned vocalizations.

D2 dopamine receptor activation induces female preference for male song in the monogamous zebra finch

The evolutionary conservation of neural mechanisms for forming and maintaining pair bonds is unclear. Oxytocin, vasopressin and dopamine (DA) transmitter systems have been shown to be important in pair-bond formation and maintenance in several vertebrate species. We examined the role of dopamine in formation of song preference in zebra finches, a monogamous bird. Male courtship song is an honest signal of sexual fitness; thus, we measured female song preference to evaluate the role of DA in mate selection and pair-bond formation, using an operant conditioning paradigm. We found that DA acting through the D2 receptor, but not the D1 receptor, can induce a song preference in unpaired female finches and that blocking the D2 receptor abolished song preference in paired females. These results suggest that similar neural mechanisms for pair-bond formation are evolutionarily conserved in rodents and birds.

Female Social Feedback Reveals Non-imitative Mechanisms of Vocal Learning in Zebra Finches

Learning of song in birds provides a powerful model for human speech development [1-3]. However, the degree to which songbirds and humans share social mechanisms of vocal learning is unknown. Although it has been demonstrated as a vocal learning mechanism in human infants [3-6], learning via active social feedback is considered rare and atypical among non-human animals [7]. We report here the first evidence that song learning in the zebra finch (Taeniopygia guttata), the most common model species of vocal learning and development, utilizes socially guided vocal learning. We demonstrate experimentally that the songs of juvenile zebra finches are guided toward mature vocal forms by real-time visual feedback from adult females that is contingent on their early, immature vocalizations. Using a video playback paradigm, we found that juvenile birds that received non-vocal female feedback contingently on their immature song learned significantly better and more accurate song than did yoked controls that received identical but non-contingent feedback. Both contingent and non-contingent groups sang at similar rates. Thus, we have provided the first evidence suggesting that non-imitative social learning is a crucial, potentially widespread mechanism of vocal development and have established a foundational parallel between humans and our most ubiquitous animal model of vocal learning: the crucial role of social feedback to immature vocalizations in the development of communication.

The Song Remains the Same

Deafness causes speech to deteriorate, but whether this deterioration reflects an active or passive process is unclear. Birdsong - a learned vocal behavior that resembles speech in its dependence on auditory feedback - also deteriorates following deafening. In their 2000 paper, Brainard and Doupe showed that, following deafening, birdsong deteriorates through an active process mediated by a cortex-basal ganglia (BG) circuit.

Inter- and intra-specific differences in muscarinic acetylcholine receptor expression in the neural pathways for vocal learning in songbirds

Acetylcholine receptors (AChRs) abound in the central nervous system of vertebrates. Muscarinic AChRs (mAChRs), a functional subclass of AChRs, mediate neuronal responses via intracellular signal transduction. They also play roles in sensorimotor coordination and motor skill learning by enhancing cortical plasticity. Learned birdsong is a complex motor skill acquired through sensorimotor coordination during a critical period. However, the functions of AChRs in the neural circuits for vocal learning and production remain largely unexplored. Here, we report the unique expression of mAChRs subunits (chrm2-5) in the song nuclei of zebra finches. The expression of excitatory subunits (chrm3 and chrm5) was downregulated in the song nuclei compared with the surrounding brain regions. In contrast, the expression of inhibitory mAChRs (chrm2 and chrm4) was upregulated in the premotor song nucleus HVC relative to the surrounding nidopallium. Chrm4 showed developmentally different expression in HVC during the critical period. Compared with chrm4, individual differences in chrm2 expression emerged in HVC early in the critical period. These individual differences in chrm2 expression persisted despite testosterone administration or auditory deprivation, which altered the timing of song stabilization. Instead, the variability in chrm2 expression in HVC correlated with parental genetics. In addition, chrm2 expression in HVC exhibited species differences and individual variability among songbird species. These results suggest that mAChRs play an underappreciated role in the development of species and individual differences in song patterns by modulating the excitability of HVC neurons, providing a potential insight into the gating of auditory responses in HVC neurons.

Cultural transmission in an ever-changing world: trial-and-error copying may be more robust than precise imitation

Cultural transmission facilitates the spread of behaviours within social groups and may lead to the establishment of stable traditions in both human and non-human animals. The fidelity of transmission is frequently emphasized as a core component of cultural evolution and as a prerequisite for cumulative culture. Fidelity is often considered a synonym of precise copying of observed behaviours. However, while precise copying guarantees reliable transmission in an ideal static world, it may be vulnerable to realistic variability in the actual environment. Here, we argue that fidelity may be more naturally achieved when the social learning mechanisms incorporate trial-and-error; and that the robustness of social transmission is thereby increased. We employed a simple model to demonstrate how culture that is produced through exact copying is fragile in an (even slightly) noisy world. When incorporating a certain degree of trial-and-error, however, cultures are more readily formed in a stochastic environment and are less vulnerable to rare ecological changes. We suggest that considering trial-and-error learning as a stabilizing component of social transmission may provide insights into cultural evolution in a realistic, variable, world.This article is part of the theme issue 'Bridging cultural gaps: interdisciplinary studies in human cultural evolution'.

Learning auditory discriminations from observation is efficient but less robust than learning from experience

Social learning enables complex societies. However, it is largely unknown how insights obtained from observation compare with insights gained from trial-and-error, in particular in terms of their robustness. Here, we use aversive reinforcement to train "experimenter" zebra finches to discriminate between auditory stimuli in the presence of an "observer" finch. We show that experimenters are slow to successfully discriminate the stimuli, but immediately generalize their ability to a new set of similar stimuli. By contrast, observers subjected to the same task are able to discriminate the initial stimulus set, but require more time for successful generalization. Drawing on concepts from machine learning, we suggest that observer learning has evolved to rapidly absorb sensory statistics without pressure to minimize neural resources, whereas learning from experience is endowed with a form of regularization that enables robust inference.

A subcortical circuit linking the cerebellum to the basal ganglia engaged in vocal learning

Speech is a complex sensorimotor skill, and vocal learning involves both the basal ganglia and the cerebellum. These subcortical structures interact indirectly through their respective loops with thalamo-cortical and brainstem networks, and directly via subcortical pathways, but the role of their interaction during sensorimotor learning remains undetermined. While songbirds and their song-dedicated basal ganglia-thalamo-cortical circuitry offer a unique opportunity to study subcortical circuits involved in vocal learning, the cerebellar contribution to avian song learning remains unknown. We demonstrate that the cerebellum provides a strong input to the song-related basal ganglia nucleus in zebra finches. Cerebellar signals are transmitted to the basal ganglia via a disynaptic connection through the thalamus and then conveyed to their cortical target and to the premotor nucleus controlling song production. Finally, cerebellar lesions impair juvenile song learning, opening new opportunities to investigate how subcortical interactions between the cerebellum and basal ganglia contribute to sensorimotor learning.

Vocal learning in songbirds requires cholinergic signaling in a motor cortex-like nucleus

Cholinergic inputs to cortex modulate plasticity and sensory processing, yet little is known about their role in motor control. Here, we show that cholinergic signaling in a songbird vocal motor cortical area, the robust nucleus of the arcopallium (RA), is required for song learning. Reverse microdialysis of nicotinic and muscarinic receptor antagonists into RA in juvenile birds did not significantly affect syllable timing or acoustic structure during vocal babbling. However, chronic blockade over weeks reduced singing quantity and impaired learning, resulting in an impoverished song with excess variability, abnormal acoustic features, and reduced similarity to tutor song. The demonstration that cholinergic signaling in a motor cortical area is required for song learning motivates the songbird as a tractable model system to identify roles of the basal forebrain cholinergic system in motor control. NEW & NOTEWORTHY Cholinergic inputs to cortex are evolutionarily conserved and implicated in sensory processing and synaptic plasticity. However, functions of cholinergic signals in motor areas are understudied and poorly understood. Here, we show that cholinergic signaling in a songbird vocal motor cortical area is not required for normal vocal variability during babbling but is essential for developmental song learning. Cholinergic modulation of motor cortex is thus required for learning but not for the ability to sing.

How Movement Modulates Hearing

Hearing is often viewed as a passive process: Sound enters the ear, triggers a cascade of activity through the auditory system, and culminates in an auditory percept. In contrast to a passive process, motor-related signals strongly modulate the auditory system from the eardrum to the cortex. The motor modulation of auditory activity is most well documented during speech and other vocalizations but also can be detected during a wide variety of other sound-generating behaviors. An influential idea is that these motor-related signals suppress neural responses to predictable movement-generated sounds, thereby enhancing sensitivity to environmental sounds during movement while helping to detect errors in learned acoustic behaviors, including speech and musicianship. Findings in humans, monkeys, songbirds, and mice provide new insights into the circuits that convey motor-related signals to the auditory system, while lending support to the idea that these signals function predictively to facilitate hearing and vocal learning.

Place preference and vocal learning rely on distinct reinforcers in songbirds

In reinforcement learning (RL) agents are typically tasked with maximizing a single objective function such as reward. But it remains poorly understood how agents might pursue distinct objectives at once. In machines, multiobjective RL can be achieved by dividing a single agent into multiple sub-agents, each of which is shaped by agent-specific reinforcement, but it remains unknown if animals adopt this strategy. Here we use songbirds to test if navigation and singing, two behaviors with distinct objectives, can be differentially reinforced. We demonstrate that strobe flashes aversively condition place preference but not song syllables. Brief noise bursts aversively condition song syllables but positively reinforce place preference. Thus distinct behavior-generating systems, or agencies, within a single animal can be shaped by correspondingly distinct reinforcement signals. Our findings suggest that spatially segregated vocal circuits can solve a credit assignment problem associated with multiobjective learning.

Music Evolution in the Laboratory: Cultural Transmission Meets Neurophysiology

In recent years, there has been renewed interest in the biological and cultural evolution of music, and specifically in the role played by perceptual and cognitive factors in shaping core features of musical systems, such as melody, harmony, and rhythm. One proposal originates in the language sciences. It holds that aspects of musical systems evolve by adapting gradually, in the course of successive generations, to the structural and functional characteristics of the sensory and memory systems of learners and “users” of music. This hypothesis has found initial support in laboratory experiments on music transmission. In this article, we first review some of the most important theoretical and empirical contributions to the field of music evolution. Next, we identify a major current limitation of these studies, i.e., the lack of direct neural support for the hypothesis of cognitive adaptation. Finally, we discuss a recent experiment in which this issue was addressed by using event-related potentials (ERPs). We suggest that the introduction of neurophysiology in cultural transmission research may provide novel insights on the micro-evolutionary origins of forms of variation observed in cultural systems.

A Basal Ganglia Circuit Sufficient to Guide Birdsong Learning

Learning vocal behaviors, like speech and birdsong, is thought to rely on continued performance evaluation. Whether candidate performance evaluation circuits in the brain are sufficient to guide vocal learning is not known. Here, we test the sufficiency of VTA projections to the vocal basal ganglia in singing zebra finches, a songbird species that learns to produce a complex and stereotyped multi-syllabic courtship song during development. We optogenetically manipulate VTA axon terminals in singing birds contingent on how the pitch of an individual song syllable is naturally performed. We find that optical inhibition and excitation of VTA terminals are each sufficient to reliably guide learned changes in song. Inhibition and excitation have opponent effects on future performances of targeted song syllables, consistent with positive and negative reinforcement of performance outcomes. These findings define a central role for reinforcement mechanisms in learning vocalizations and demonstrate minimal circuit elements for learning vocal behaviors. VIDEO ABSTRACT.

The biological and behavioral computations that influence dopamine responses

Phasic dopamine responses demonstrate remarkable simplicity; they code for the differences between received and predicted reward values. Yet this simplicity belies the subtle complexity of the psychological, computational, and contextual factors that influence this signal. Advances in behavioral paradigms and models, in monkeys and rodents, have demonstrated that phasic dopamine responses reflect numerous behavioral computations and factors including choice, subjective value, confidence, and context. The application of optogenetics has provided evidence that dopamine reward prediction error responses cause value learning. Furthermore, studies using advanced circuit tracing techniques have begun to uncover the biological network implementation of the reward learning algorithm. The purpose of this review is to summarize the recent advances in dopamine neurophysiology and synthesize an updated account of the behavioral function of dopamine signals.

A common neural circuit mechanism for internally guided and externally reinforced forms of motor learning

The complex skills underlying verbal and musical expression can be learned without external punishment or reward, indicating their learning is internally guided. The neural mechanisms that mediate internally guided learning are poorly understood, but a circuit comprising dopamine-releasing neurons in the midbrain ventral tegmental area (VTA) and their targets in the basal ganglia are important to externally reinforced learning. Juvenile zebra finches copy a tutor song in a process that is internally guided and, in adulthood, can learn to modify the fundamental frequency (pitch) of a target syllable in response to external reinforcement with white noise. Here we combined intersectional genetic ablation of VTA neurons, reversible blockade of dopamine receptors in the basal ganglia, and singing-triggered optogenetic stimulation of VTA terminals to establish that a common VTA-basal ganglia circuit enables internally guided song copying and externally reinforced syllable pitch learning.

A hypothesis on a role of oxytocin in the social mechanisms of speech and vocal learning

Language acquisition in humans and song learning in songbirds naturally happen as a social learning experience, providing an excellent opportunity to reveal social motivation and reward mechanisms that boost sensorimotor learning. Our knowledge about the molecules and circuits that control these social mechanisms for vocal learning and language is limited. Here we propose a hypothesis of a role for oxytocin (OT) in the social motivation and evolution of vocal learning and language. Building upon existing evidence, we suggest specific neural pathways and mechanisms through which OT might modulate vocal learning circuits in specific developmental stages.

Early life manipulations of vasopressin-family peptides alter vocal learning

Vocal learning from social partners is crucial for the successful development of communication in a wide range of species. Social interactions organize attention and enhance motivation to learn species-typical behaviour. However, the neurobiological mechanisms connecting social motivation and vocal learning are unknown. Using zebra finches (Taeniopygia guttata), a ubiquitous model for vocal learning, we show that manipulations of nonapeptide hormones in the vasopressin family (arginine vasotocin, AVT) early in development can promote or disrupt both song and social motivation. Young male zebra finches, like human infants, are socially gregarious and require interactive feedback from adult tutors to learn mature vocal forms. To investigate the role of social motivational mechanisms in song learning, in two studies, we injected hatchling males with AVT or Manning compound (MC, a nonapeptide receptor antagonist) on days 2-8 post-hatching and recorded song at maturity. In both studies, MC males produced a worse match to tutor song than controls. In study 2, which experimentally controlled for tutor and genetic factors, AVT males also learned song significantly better compared with controls. Furthermore, song similarity correlated with several measures of social motivation throughout development. These findings provide the first evidence that nonapeptides are critical to the development of vocal learning.

miR-9 regulates basal ganglia-dependent developmental vocal learning and adult vocal performance in songbirds

miR-9 is an evolutionarily conserved miRNA that is abundantly expressed in Area X, a basal ganglia nucleus required for vocal learning in songbirds. Here, we report that overexpression of miR-9 in Area X of juvenile zebra finches impairs developmental vocal learning, resulting in a song with syllable omission, reduced similarity to the tutor song, and altered acoustic features. miR-9 overexpression in juveniles also leads to more variable song performance in adulthood, and abolishes social context-dependent modulation of song variability. We further show that these behavioral deficits are accompanied by downregulation of FoxP1 and FoxP2, genes that are known to be associated with language impairments, as well as by disruption of dopamine signaling and widespread changes in the expression of genes that are important in circuit development and functions. These findings demonstrate a vital role for miR-9 in basal ganglia function and vocal communication, suggesting that dysregulation of miR-9 in humans may contribute to language impairments and related neurodevelopmental disorders.

Building a state space for song learning

The songbird system has shed light on how the brain produces precisely timed behavioral sequences, and how the brain implements reinforcement learning (RL). RL is a powerful strategy for learning what action to produce in each state, but requires a unique representation of the states involved in the task. Songbird RL circuitry is thought to operate using a representation of each moment within song syllables, consistent with the sparse sequential bursting of neurons in premotor cortical nucleus HVC. However, such sparse sequences are not present in very young birds, which sing highly variable syllables of random lengths. Here, we review and expand upon a model for how the songbird brain could construct latent sequences to support RL, in light of new data elucidating connections between HVC and auditory cortical areas. We hypothesize that learning occurs via four distinct plasticity processes: 1) formation of 'tutor memory' sequences in auditory areas; 2) formation of appropriately-timed latent HVC sequences, seeded by inputs from auditory areas spontaneously replaying the tutor song; 3) strengthening, during spontaneous replay, of connections from HVC to auditory neurons of corresponding timing in the 'tutor memory' sequence, aligning auditory and motor representations for subsequent song evaluation; and 4) strengthening of connections from premotor neurons to motor output neurons that produce the desired sounds, via well-described song RL circuitry.

Towards deep learning with segregated dendrites

Deep learning has led to significant advances in artificial intelligence, in part, by adopting strategies motivated by neurophysiology. However, it is unclear whether deep learning could occur in the real brain. Here, we show that a deep learning algorithm that utilizes multi-compartment neurons might help us to understand how the neocortex optimizes cost functions. Like neocortical pyramidal neurons, neurons in our model receive sensory information and higher-order feedback in electrotonically segregated compartments. Thanks to this segregation, neurons in different layers of the network can coordinate synaptic weight updates. As a result, the network learns to categorize images better than a single layer network. Furthermore, we show that our algorithm takes advantage of multilayer architectures to identify useful higher-order representations—the hallmark of deep learning. This work demonstrates that deep learning can be achieved using segregated dendritic compartments, which may help to explain the morphology of neocortical pyramidal neurons.

Artificial intelligence has made major progress in recent years thanks to a technique known as deep learning, which works by mimicking the human brain. When computers employ deep learning, they learn by using networks made up of many layers of simulated neurons. Deep learning has opened the door to computers with human – or even super-human – levels of skill in recognizing images, processing speech and controlling vehicles. But many neuroscientists are skeptical about whether the brain itself performs deep learning.

The patterns of activity that occur in computer networks during deep learning resemble those seen in human brains. But some features of deep learning seem incompatible with how the brain works. Moreover, neurons in artificial networks are much simpler than our own neurons. For instance, in the region of the brain responsible for thinking and planning, most neurons have complex tree-like shapes. Each cell has ‘roots’ deep inside the brain and ‘branches’ close to the surface. By contrast, simulated neurons have a uniform structure.

To find out whether networks made up of more realistic simulated neurons could be used to make deep learning more biologically realistic, Guerguiev et al. designed artificial neurons with two compartments, similar to the ‘roots’ and ‘branches’. The network learned to recognize hand-written digits more easily when it had many layers than when it had only a few. This shows that artificial neurons more like those in the brain can enable deep learning. It even suggests that our own neurons may have evolved their shape to support this process.

If confirmed, the link between neuronal shape and deep learning could help us develop better brain-computer interfaces. These allow people to use their brain activity to control devices such as artificial limbs. Despite advances in computing, we are still superior to computers when it comes to learning. Understanding how our own brains show deep learning could thus help us develop better, more human-like artificial intelligence in the future.

Discrete Circuits Support Generalized versus Context-Specific Vocal Learning in the Songbird

Motor skills depend on the reuse of individual gestures in multiple sequential contexts (e.g., a single phoneme in different words). Yet optimal performance requires that a given gesture be modified appropriately depending on the sequence in which it occurs. To investigate the neural architecture underlying such context-dependent modifications, we studied Bengalese finch song, which, like speech, consists of variable sequences of "syllables." We found that when birds are instructed to modify a syllable in one sequential context, learning generalizes across contexts; however, if unique instruction is provided in different contexts, learning is specific for each context. Using localized inactivation of a cortical-basal ganglia circuit specialized for song, we show that this balance between generalization and specificity reflects a hierarchical organization of neural substrates. Primary motor circuitry encodes a core syllable representation that contributes to generalization, while top-down input from cortical-basal ganglia circuitry biases this representation to enable context-specific learning.

AZI23'UTR Is a New SLC6A3 Downregulator Associated with an Epistatic Protection Against Substance Use Disorders

Regulated activity of SLC6A3, which encodes the human dopamine transporter (DAT), contributes to diseases such as substance abuse disorders (SUDs); however, the exact transcription mechanism remains poorly understood. Here, we used a common genetic variant of the gene, intron 1 DNP1B sequence, as bait to screen and clone a new transcriptional activity, ^AZI23'UTR, for SLC6A3. ^AZI23'UTR is a 3' untranslated region (3'UTR) of the human 5-Azacytidine Induced 2 gene (AZI2) but appeared to be transcribed independently of AZI2. Found to be present in both human cell nuclei and dopamine neurons, this RNA was shown to downregulate promoter activity through a variant-dependent mechanism in vitro. Both reduced RNA density ratio of ^AZI23'UTR/AZI2 and increased DAT mRNA levels were found in ethanol-naive alcohol-preferring rats. Secondary analysis of dbGaP GWAS datasets (Genome-Wide Association Studies based on the database of Genotypes and Phenotypes) revealed significant interactions between regions upstream of ^AZI23'UTR and SLC6A3 in SUDs. Jointly, our data suggest that ^AZI23'UTR confers variant-dependent transcriptional regulation of SLC6A3, a potential risk factor for SUDs.

Calcium dynamics control K-ATP channel-mediated bursting in substantia nigra dopamine neurons: a combined experimental and modeling study

Burst firing in medial substantia nigra (mSN) dopamine (DA) neurons has been selectively linked to novelty-induced exploration behavior in mice. Burst firing in mSN DA neurons, in contrast to lateral SN DA neurons, requires functional ATP-sensitive potassium (K-ATP) channels both in vitro and in vivo. However, the precise role of K-ATP channels in promoting burst firing is unknown. We show experimentally that L-type calcium channel activity in mSN DA neurons enhances open probability of K-ATP channels. We then generate a mathematical model to study the role of Ca²⁺ dynamics driving K-ATP channel function in mSN DA neurons during bursting. In our model, Ca²⁺ influx leads to local accumulation of ADP due to Ca-ATPase activity, which in turn activates K-ATP channels. If K-ATP channel activation reaches levels sufficient to terminate spiking, rhythmic bursting occurs. The model explains the experimental observation that, in vitro, coapplication of NMDA and a selective K-ATP channel opener, NN414, is required to elicit bursting as follows. Simulated NMDA receptor activation increases the firing rate and the rate of Ca²⁺ influx, which increases the activation of K-ATP. The model suggests that additional sources of hyperpolarization, such as GABAergic synaptic input, are recruited in vivo for burst termination or rebound burst discharge. The model predicts that NN414 increases the sensitivity of the K-ATP channel to ADP, promoting burst firing in vitro, and that that high levels of Ca²⁺ buffering, as might be expected in the calbindin-positive SN DA neuron subpopulation, promote rhythmic bursting pattern, consistent with experimental observations in vivo. NEW & NOTEWORTHY Recently identified distinct subpopulations of midbrain dopamine neurons exhibit differences in their two primary activity patterns in vivo: tonic (single spike) firing and phasic bursting. This study elucidates the biophysical basis of bursts specific to dopamine neurons in the medial substantia nigra, enabled by ATP-sensitive K⁺ channels and necessary for novelty-induced exploration. A better understanding of how dopaminergic signaling differs between subpopulations may lead to therapeutic strategies selectively targeted to specific subpopulations.

The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia

Computational models of reinforcement learning (RL) strive to produce behavior that maximises reward, and thus allow software or robots to behave adaptively [1]. At the core of RL models is a learned mapping between 'states'-situations or contexts that an agent might encounter in the world-and actions. A wealth of physiological and anatomical data suggests that the basal ganglia (BG) is important for learning these mappings [2,3]. However, the computations performed by specific circuits are unclear. In this brief review, we highlight recent work concerning the anatomy and physiology of BG circuits that suggest refinements in our understanding of computations performed by the basal ganglia. We focus on one important component of basal ganglia circuitry, midbrain dopamine neurons, drawing attention to data that has been cast as supporting or departing from the RL framework that has inspired experiments in basal ganglia research over the past two decades. We suggest that the parallel circuit architecture of the BG might be expected to produce variability in the response properties of different dopamine neurons, and that variability in response profile may not reflect variable functions, but rather different arguments that serve as inputs to a common function: the computation of prediction error.

Sexual dimorphism in striatal dopaminergic responses promotes monogamy in social songbirds

In many songbird species, males sing to attract females and repel rivals. How can gregarious, non-territorial songbirds such as zebra finches, where females have access to numerous males, sustain monogamy? We found that the dopaminergic reward circuitry of zebra finches can simultaneously promote social cohesion and breeding boundaries. Surprisingly, in unmated males but not in females, striatal dopamine neurotransmission was elevated after hearing songs. Behaviorally too, unmated males but not females persistently exchanged mild punishments in return for songs. Song reinforcement diminished when dopamine receptors were blocked. In females, we observed song reinforcement exclusively to the mate’s song, although their striatal dopamine neurotransmission was only slightly elevated. These findings suggest that song-triggered dopaminergic activation serves a dual function in social songbirds: as low-threshold social reinforcement in males and as ultra-selective sexual reinforcement in females. Co-evolution of sexually dimorphic reinforcement systems can explain the coexistence of gregariousness and monogamy.

While monogamy is rare within the animal kingdom, some species – including humans and many birds – can be highly social and yet sustain monogamous relationships. Zebra finches, for example, are among a number of species of songbirds in which numerous males and females live closely together but maintain monogamous partnerships. Male songbirds use their songs to attract females, who do not themselves sing. But if female birds are attracted to any male song, how do they manage to remain monogamous when surrounded by potential suitors?

In songbirds, and in humans too, a region of the brain called the striatum regulates both social and sexual behaviors. It does this by modulating the release of a molecule called dopamine, which is the brain’s reward signal. Tokarev et al. show that hearing songs triggers dopamine release within the striatum of unattached male zebra finches, but has no such effect in single females. Unattached male songbirds will also put up with irritating puffs of air in exchange for being able to watch videos of singing birds, whereas unattached females will not. Female songbirds with partners will tolerate the air puffs, but only if the videos are accompanied with the songs of their own mate.

These findings suggest that song serves as a generic social stimulus for zebra finch males, helping large numbers of birds to live together. By contrast, for a female zebra finch, the song of her partner is a highly selective sexual stimulus. These sex-specific responses to the same socially-relevant stimuli may explain how gregarious animals are able to maintain monogamous pair bonds. More generally, these results are a step towards understanding how brain reward systems regulate social interactions. Studying these mechanisms in songbird species with different social and mating systems could ultimately provide insights into social and sexual behavior in people.

2D MoS2 Neuromorphic Devices for Brain-Like Computational Systems

Hardware implementation of artificial synapses/neurons with 2D solid-state devices is of great significance for nanoscale brain-like computational systems. Here, 2D MoS₂ synaptic/neuronal transistors are fabricated by using poly(vinyl alcohol) as the laterally coupled, proton-conducting electrolytes. Fundamental synaptic functions, such as an excitatory postsynaptic current, paired-pulse facilitation, and a dynamic filter for information transmission of biological synapse, are successfully emulated. Most importantly, with multiple input gates and one modulatory gate, spiking-dependent logic operation/modulation, multiplicative neural coding, and neuronal gain modulation are also experimentally demonstrated. The results indicate that the intriguing 2D MoS₂ transistors are also very promising for the next-generation of nanoscale neuromorphic device applications.

Maturation, Behavioral Activation, and Connectivity of Adult-Born Medium Spiny Neurons in a Striatal Song Nucleus

Neurogenesis continues in the adult songbird brain. Many telencephalic song control regions incorporate new neurons into their existing circuits in adulthood. One song nucleus that receives many new neurons is Area X. Because this striatal region is crucial for song learning and song maintenance the recruitment of new neurons into Area X could influence these processes. As an entry point into addressing this possibility, we investigated the maturation and connectivity within the song circuit and behavioral activation of newly generated Area X neurons. Using BrdU birth dating and virally mediated GFP expression we followed adult-generated neurons from their place of birth in the ventricle to their place of incorporation into Area X. We show that newborn neurons receive glutamatergic input from pallial/cortical song nuclei. Additionally, backfills revealed that the new neurons connect to pallidal-like projection neurons that innervate the thalamus. Using in situ hybridization, we found that new neurons express the mRNA for D1- and D2-type dopamine receptors. Employing DARPP-32 (dopamine and cAMP-regulated phosphoprotein of 32 kDa) and EGR-1 (early growth response protein 1) as markers for neural maturation and activation, we established that at 42 days after labeling approximately 80% of new neurons were mature medium spiny neurons (MSNs) and could be activated by singing behavior. Finally, we compared the MSN density in Area X of birds up to seven years of age and found a significant increase with age, indicating that new neurons are constantly added to the nucleus. In summary, we provide evidence that newborn MSNs in Area X constantly functionally integrate into the circuit and are thus likely to play a role in the maintenance and regulation of adult song.