Statistical learning of recurring sound patterns encodes auditory objects in songbird forebrain

Direct brain recordings reveal implicit encoding of structure in random auditory streams

The brain excels at processing sensory input, even in rich or chaotic environments. Mounting evidence attributes this to sophisticated internal models of the environment that draw on statistical structures in the unfolding sensory input. Understanding how and where such modeling proceeds is a core question in statistical learning and predictive processing. In this context, we address the role of transitional probabilities as an implicit structure supporting the encoding of the temporal structure of a random auditory stream. Leveraging information-theoretical principles and the high spatiotemporal resolution of intracranial electroencephalography, we analyzed the trial-by-trial high-frequency activity representation of transitional probabilities. This unique approach enabled us to demonstrate how the brain automatically and continuously encodes structure in random stimuli and revealed the involvement of a network outside of the auditory system, including hippocampal, frontal, and temporal regions. Our work provides a comprehensive picture of the neural correlates of automatic encoding of implicit structure that can be the crucial substrate for the swift detection of patterns and unexpected events in the environment.

Domestic dogs (Canis familiaris) recognise meaningful content in monotonous streams of read speech

Domestic dogs (Canis familiaris) can recognize basic phonemic information from human speech and respond to commands. Commands are typically presented in isolation with exaggerated prosody known as dog-directed speech (DDS) register. Here, we investigate whether dogs can spontaneously identify meaningful phonemic content in a stream of putatively irrelevant speech spoken in monotonous prosody, without congruent prosodic cues. To test this ability, dogs were played recordings of their owners reading a meaningless text which included a short meaningful or meaningless phrase, either read with unchanged reading prosody or with an exaggerated DDS prosody. We measured the occurrence and duration of dogs' gaze at their owners. We found that, while dogs were more likely to detect and respond to inclusions that contained meaningful phrases spoken with DDS prosody, they were still able to detect these meaningful inclusions spoken in a neutral reading prosody. Dogs detected and responded to meaningless control phrases in DDS as frequently as to meaningful content in neutral reading prosody, but less often than to meaningful content in DDS. This suggests that, while DDS prosody facilitates the detection of meaningful content in human speech by capturing dogs' attention, dogs are nevertheless capable of spontaneously recognizing meaningful phonemic content within an unexaggerated stream of speech.

Implicit auditory memory in older listeners: From encoding to 6-month retention

Any listening task, from sound recognition to sound-based communication, rests on auditory memory which is known to decline in healthy ageing. However, how this decline maps onto multiple components and stages of auditory memory remains poorly characterised. In an online unsupervised longitudinal study, we tested ageing effects on implicit auditory memory for rapid tone patterns. The test required participants (younger, aged 20-30, and older adults aged 60-70) to quickly respond to rapid regularly repeating patterns emerging from random sequences. Patterns were novel in most trials (REGn), but unbeknownst to the participants, a few distinct patterns reoccurred identically throughout the sessions (REGr). After correcting for processing speed, the response times (RT) to REGn should reflect the information held in echoic and short-term memory before detecting the pattern; long-term memory formation and retention should be reflected by the RT advantage (RTA) to REGr vs REGn which is expected to grow with exposure. Older participants were slower than younger adults in detecting REGn and exhibited a smaller RTA to REGr. Computational simulations using a model of auditory sequence memory indicated that these effects reflect age-related limitations both in early and long-term memory stages. In contrast to ageing-related accelerated forgetting of verbal material, here older adults maintained stable memory traces for REGr patterns up to 6 months after the first exposure. The results demonstrate that ageing is associated with reduced short-term memory and long-term memory formation for tone patterns, but not with forgetting, even over surprisingly long timescales.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm

Introduction

Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level.

Methods

This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up.

Results

Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated.

Discussion

Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.

Auditory Pattern Discrimination in Budgerigars (Melopsittacus undulatus)

Auditory patterns carry information in human speech at multiple levels, including the surface relationships between sounds within words in phonology and the abstract structures of syntax. The sequences of other animal vocalizations, such as birdsong, can also be described as auditory patterns, but few studies have probed how the sequences are perceived at multiple levels. Past work shows that a small parrot species, the budgerigar (Melopsittacus undulatus), exceeds other birds in sequence perception and is even sensitive to abstract structure. But it is not known what level of auditory analysis is dominant in perception or what limits might exist in sensitivity to abstract structure. Here, budgerigars were tested on their ability to discriminate changes in an auditory pattern, AAB, i.e. sound-same different, to ask how they attended to surface relationships among the sounds and the abstract relationships of same/different among the elements. The results show that the budgerigars primarily used surface transitions between the sounds when discriminating the sequences, but were able to use the abstract relationships to a limited extent, largely restricted to two elements. This study provides insight into how budgerigars extract information from conspecific vocalizations and how their capacities compare to human speech perception.

Mismatch Responses Evoked by Sound Pattern Violation in the Songbird Forebrain Suggest Common Auditory Processing With Human

Learning sound patterns in the natural auditory scene and detecting deviant patterns are adaptive behaviors that aid animals in predicting future events and behaving accordingly. Mismatch negativity (MMN) is a component of the event-related potential (ERP) that is reported in humans when they are exposed to unexpected or rare stimuli. MMN has been studied in several non-human animals using an oddball task by presenting deviant pure tones that were interspersed within a sequence of standard pure tones and comparing the neural responses. While accumulating evidence suggests the homology of non-human animal MMN-like responses (MMRs) and human MMN, it is still not clear whether the function and neural mechanisms of MMRs and MMN are comparable. The Java sparrow (Lonchura oryzivora) is a songbird that is a vocal learner, is highly social, and maintains communication with flock members using frequently repeated contact calls and song. We expect that the songbird is a potentially useful animal model that will broaden our understanding of the characterization of MMRs. Due to this, we chose this species to explore MMRs to the deviant sounds in the single sound oddball task using both pure tones and natural vocalizations. MMRs were measured in the caudomedial nidopallium (NCM), a higher-order auditory area. We recorded local field potentials under freely moving conditions. Significant differences were observed in the negative component between deviant and standard ERPs, both to pure tones and natural vocalizations in the oddball sequence. However, the subsequent experiments using the randomized standard sequence and regular pattern sequence suggest the possibility that MMR elicited in the oddball paradigm reflects the adaptation to a repeated standard sound but not the genuine deviance detection. Furthermore, we presented contact call triplet sequences and investigated MMR in the NCM in response to sound sequence order. We found a significant negative shift in response to a difference in sequence pattern. This demonstrates MMR elicited by violation of the pattern of the triplet sequence and the ability to extract sound sequence information in the songbird auditory forebrain. Our study sheds light on the electrophysiological properties of auditory sensory memory processing, expanding the scope of characterization of MMN-like responses beyond simple deviance detection, and provides a comparative perspective on syntax processing in human.

Neural processes underlying statistical learning for speech segmentation in dogs

To learn words, humans extract statistical regularities from speech. Multiple species use statistical learning also to process speech, but the neural underpinnings of speech segmentation in non-humans remain largely unknown. Here, we investigated computational and neural markers of speech segmentation in dogs, a phylogenetically distant mammal that efficiently navigates humans' social and linguistic environment. Using electroencephalography (EEG), we compared event-related responses (ERPs) for artificial words previously presented in a continuous speech stream with different distributional statistics. Results revealed an early effect (220-470 ms) of transitional probability and a late component (590-790 ms) modulated by both word frequency and transitional probability. Using fMRI, we searched for brain regions sensitive to statistical regularities in speech. Structured speech elicited lower activity in the basal ganglia, a region involved in sequence learning, and repetition enhancement in the auditory cortex. Speech segmentation in dogs, similar to that of humans, involves complex computations, engaging both domain-general and modality-specific brain areas. VIDEO ABSTRACT.

Distinct timescales for the neuronal encoding of vocal signals in a high-order auditory area

The ability of the auditory system to selectively recognize natural sound categories while maintaining a certain degree of tolerance towards variations within these categories, which may have functional roles, is thought to be crucial for vocal communication. To date, it is still largely unknown how the balance between tolerance and sensitivity to variations in acoustic signals is coded at a neuronal level. Here, we investigate whether neurons in a high-order auditory area in zebra finches, a songbird species, are sensitive to natural variations in vocal signals by recording their responses to repeated exposures to identical and variant sound sequences. We used the songs of male birds which tend to be highly repetitive with only subtle variations between renditions. When playing these songs to both anesthetized and awake birds, we found that variations between songs did not affect the neuron firing rate but the temporal reliability of responses. This suggests that auditory processing operates on a range of distinct timescales, namely a short one to detect variations in vocal signals, and longer ones that allow the birds to tolerate variations in vocal signal structure and to encode the global context.

Statistical Properties of Musical Creativity: Roles of Hierarchy and Uncertainty in Statistical Learning

Creativity is part of human nature and is commonly understood as a phenomenon whereby something original and worthwhile is formed. Owing to this ability, humans can produce innovative information that often facilitates growth in our society. Creativity also contributes to esthetic and artistic productions, such as music and art. However, the mechanism by which creativity emerges in the brain remains debatable. Recently, a growing body of evidence has suggested that statistical learning contributes to creativity. Statistical learning is an innate and implicit function of the human brain and is considered essential for brain development. Through statistical learning, humans can produce and comprehend structured information, such as music. It is thought that creativity is linked to acquired knowledge, but so-called "eureka" moments often occur unexpectedly under subconscious conditions, without the intention to use the acquired knowledge. Given that a creative moment is intrinsically implicit, we postulate that some types of creativity can be linked to implicit statistical knowledge in the brain. This article reviews neural and computational studies on how creativity emerges within the framework of statistical learning in the brain (i.e., statistical creativity). Here, we propose a hierarchical model of statistical learning: statistically chunking into a unit (hereafter and shallow statistical learning) and combining several units (hereafter and deep statistical learning). We suggest that deep statistical learning contributes dominantly to statistical creativity in music. Furthermore, the temporal dynamics of perceptual uncertainty can be another potential causal factor in statistical creativity. Considering that statistical learning is fundamental to brain development, we also discuss how typical versus atypical brain development modulates hierarchical statistical learning and statistical creativity. We believe that this review will shed light on the key roles of statistical learning in musical creativity and facilitate further investigation of how creativity emerges in the brain.

Learning to predict: Neuronal signatures of auditory expectancy in human event-related potentials

Learning to anticipate future states of the world based on statistical regularities in the environment is a key component of perception and is vital for the survival of many organisms. Such statistical learning and prediction are crucial for acquiring language and music appreciation. Importantly, learned expectations can be implicitly derived from exposure to sensory input, without requiring explicit information regarding contingencies in the environment. Whereas many previous studies of statistical learning have demonstrated larger neuronal responses to unexpected versus expected stimuli, the neuronal bases of the expectations themselves remain poorly understood. Here we examined behavioral and neuronal signatures of learned expectancy via human scalp-recorded event-related brain potentials (ERPs). Participants were instructed to listen to a series of sounds and press a response button as quickly as possible upon hearing a target noise burst, which was either reliably or unreliably preceded by one of three pure tones in low-, mid-, and high-frequency ranges. Participants were not informed about the statistical contingencies between the preceding tone ‘cues’ and the target. Over the course of a stimulus block, participants responded more rapidly to reliably cued targets. This behavioral index of learned expectancy was paralleled by a negative ERP deflection, designated as a neuronal contingency response (CR), which occurred immediately prior to the onset of the target. The amplitude and latency of the CR were systematically modulated by the strength of the predictive relationship between the cue and the target. Re-averaging ERPs with respect to the latency of behavioral responses revealed no consistent relationship between the CR and the motor response, suggesting that the CR represents a neuronal signature of learned expectancy or anticipatory attention. Our results demonstrate that statistical regularities in an auditory input stream can be implicitly learned and exploited to influence behavior. Furthermore, we uncover a potential ‘prediction signal’ that reflects this fundamental learning process.

Statistical Learning of Unfamiliar Sounds as Trajectories Through a Perceptual Similarity Space

In typical statistical learning studies, researchers define sequences in terms of the probability of the next item in the sequence given the current item (or items), and they show that high probability sequences are treated as more familiar than low probability sequences. Existing accounts of these phenomena all assume that participants represent statistical regularities more or less as they are defined by the experimenters-as sequential probabilities of symbols in a string. Here we offer an alternative, or possibly supplementary, hypothesis. Specifically, rather than identifying or labeling individual stimuli discretely in order to predict the next item in a sequence, we need only assume that the participant is able to represent the stimuli as evincing particular similarity relations to one another, with sequences represented as trajectories through this similarity space. We present experiments in which this hypothesis makes sharply different predictions from hypotheses based on the assumption that sequences are learned over discrete, labeled stimuli. We also present a series of simulation models that encode stimuli as positions in a continuous two-dimensional space, and predict the next location from the current location. Although no model captures all of the data presented here, the results of three critical experiments are more consistent with the view that participants represent trajectories through similarity space rather than sequences of discrete labels under particular conditions.

Non-adjacent Dependency Learning in Humans and Other Animals

Learning and processing natural language requires the ability to track syntactic relationships between words and phrases in a sentence, which are often separated by intervening material. These nonadjacent dependencies can be studied using artificial grammar learning paradigms and structured sequence processing tasks. These approaches have been used to demonstrate that human adults, infants and some nonhuman animals are able to detect and learn dependencies between nonadjacent elements within a sequence. However, learning nonadjacent dependencies appears to be more cognitively demanding than detecting dependencies between adjacent elements, and only occurs in certain circumstances. In this review, we discuss different types of nonadjacent dependencies in language and in artificial grammar learning experiments, and how these differences might impact learning. We summarize different types of perceptual cues that facilitate learning, by highlighting the relationship between dependent elements bringing them closer together either physically, attentionally, or perceptually. Finally, we review artificial grammar learning experiments in human adults, infants, and nonhuman animals, and discuss how similarities and differences observed across these groups can provide insights into how language is learned across development and how these language-related abilities might have evolved.

Structured Sequence Learning: Animal Abilities, Cognitive Operations, and Language Evolution

Human language is a salient example of a neurocognitive system that is specialized to process complex dependencies between sensory events distributed in time, yet how this system evolved and specialized remains unclear. Artificial Grammar Learning (AGL) studies have generated a wealth of insights into how human adults and infants process different types of sequencing dependencies of varying complexity. The AGL paradigm has also been adopted to examine the sequence processing abilities of nonhuman animals. We critically evaluate this growing literature in species ranging from mammals (primates and rats) to birds (pigeons, songbirds, and parrots) considering also cross-species comparisons. The findings are contrasted with seminal studies in human infants that motivated the work in nonhuman animals. This synopsis identifies advances in knowledge and where uncertainty remains regarding the various strategies that nonhuman animals can adopt for processing sequencing dependencies. The paucity of evidence in the few species studied to date and the need for follow-up experiments indicate that we do not yet understand the limits of animal sequence processing capacities and thereby the evolutionary pattern. This vibrant, yet still budding, field of research carries substantial promise for advancing knowledge on animal abilities, cognitive substrates, and language evolution.

Long-term implicit memory for sequential auditory patterns in humans

Memory, on multiple timescales, is critical to our ability to discover the structure of our surroundings, and efficiently interact with the environment. We combined behavioural manipulation and modelling to investigate the dynamics of memory formation for rarely reoccurring acoustic patterns. In a series of experiments, participants detected the emergence of regularly repeating patterns within rapid tone-pip sequences. Unbeknownst to them, a few patterns reoccurred every ~3 min. All sequences consisted of the same 20 frequencies and were distinguishable only by the order of tone-pips. Despite this, reoccurring patterns were associated with a rapidly growing detection-time advantage over novel patterns. This effect was implicit, robust to interference, and persisted for 7 weeks. The results implicate an interplay between short (a few seconds) and long-term (over many minutes) integration in memory formation and demonstrate the remarkable sensitivity of the human auditory system to sporadically reoccurring structure within the acoustic environment.

Statistical learning of transition patterns in the songbird auditory forebrain

Statistical learning of transition patterns between sounds—a striking capability of the auditory system—plays an essential role in animals’ survival (e.g., detect deviant sounds that signal danger). However, the neural mechanisms underlying this capability are still not fully understood. We recorded extracellular multi-unit and single-unit activity in the auditory forebrain of awake male zebra finches while presenting rare repetitions of a single sound in a long sequence of sounds (canary and zebra finch song syllables) patterned in either an alternating or random order at different inter-stimulus intervals (ISI). When preceding stimuli were regularly alternating (alternating condition), a repeated stimulus violated the preceding transition pattern and was a deviant. When preceding stimuli were in random order (control condition), a repeated stimulus did not violate any regularities and was not a deviant. At all ISIs tested (1 s, 3 s, or jittered at 0.8–1.2 s), deviant repetition enhanced neural responses in the alternating condition in a secondary auditory area (caudomedial nidopallium, NCM) but not in the primary auditory area (Field L2); in contrast, repetition suppressed responses in the control condition in both Field L2 and NCM. When stimuli were presented in the classical oddball paradigm at jittered ISI (0.8–1.2 s), neural responses in both NCM and Field L2 were stronger when a stimulus occurred as deviant with low probability than when the same stimulus occurred as standard with high probability. Together, these results demonstrate: (1) classical oddball effect exists even when ISI is jittered and the onset of a stimulus is not fully predictable; (2) neurons in NCM can learn transition patterns between sounds at multiple ISIs and detect violation of these transition patterns; (3) sensitivity to deviant sounds increases from Field L2 to NCM in the songbird auditory forebrain. Further studies using the current paradigms may help us understand the neural substrate of statistical learning and even speech comprehension.

Adaptive Efficient Coding of Correlated Acoustic Properties

Natural sounds such as vocalizations often have covarying acoustic attributes, resulting in redundancy in neural coding. The efficient coding hypothesis proposes that sensory systems are able to detect such covariation and adapt to reduce redundancy, leading to more efficient neural coding. Recent psychoacoustic studies have shown the auditory system can rapidly adapt to efficiently encode two covarying dimensions as a single dimension, following passive exposure to sounds in which temporal and spectral attributes covaried in a correlated fashion. However, these studies observed a cost to this adaptation, which was a loss of sensitivity to the orthogonal dimension. Here we explore the neural basis of this psychophysical phenomenon by recording single-unit responses from the primary auditory cortex in awake ferrets exposed passively to stimuli with two correlated attributes, similar in stimulus design to the psychoacoustic experiments in humans. We found: (1) the signal-to-noise ratio of spike-rate coding of cortical responses driven by sounds with correlated attributes remained unchanged along the exposure dimension, but was reduced along the orthogonal dimension; (2) performance of a decoder trained with spike data to discriminate stimuli along the orthogonal dimension was equally reduced; (3) correlations between neurons tuned to the two covarying attributes decreased after exposure; and (4) these exposure effects still occurred if sounds were correlated along two acoustic dimensions, but varied randomly along a third dimension. These neurophysiological results are consistent with the efficient coding hypothesis and may help deepen our understanding of how the auditory system encodes and represents acoustic regularities and covariance.SIGNIFICANCE STATEMENT The efficient coding (EC) hypothesis (Attneave, 1954; Barlow, 1961) proposes that the neural code in sensory systems efficiently encodes natural stimuli by minimizing the number of spikes to transmit a sensory signal. Results of recent psychoacoustic studies in humans are consistent with the EC hypothesis in that, following passive exposure to stimuli with correlated attributes, the auditory system rapidly adapts so as to more efficiently encode the two covarying dimensions as a single dimension. In the current neurophysiological experiments, using a similar stimulus design and the experimental paradigm to the psychoacoustic studies of Stilp et al. (2010) and Stilp and Kluender (2011, 2012, 2016), we recorded responses from single neurons in the auditory cortex of the awake ferret, showing adaptive efficient neural coding of two correlated acoustic attributes.

Neuronal Encoding in a High-Level Auditory Area: From Sequential Order of Elements to Grammatical Structure

Sensitivity to the sequential structure of communication sounds is fundamental not only for language comprehension in humans but also for song recognition in songbirds. By quantifying single-unit responses, we first assessed whether the sequential order of song elements, called syllables, in conspecific songs is encoded in a secondary auditory cortex-like region of the zebra finch brain. Based on a habituation/dishabituation paradigm, we show that, after multiple repetitions of the same conspecific song, rearranging syllable order reinstated strong responses. A large proportion of neurons showed sensitivity to song context in which syllables occurred providing support for the nonlinear processing of syllable sequences. Sensitivity to the temporal order of items within a sequence should enable learning its underlying structure, an ability considered a core mechanism of the human language faculty. We show that repetitions of songs that were ordered according to a specific grammatical structure (i.e., ABAB or AABB structures; A and B denoting song syllables) led to different responses in both anesthetized and awake birds. Once responses were decreased due to song repetitions, the transition from one structure to the other could affect the firing rates and/or the spike patterns. Our results suggest that detection was based on local differences rather than encoding of the global song structure as a whole. Our study demonstrates that a high-level auditory region provides neuronal mechanisms to help discriminate stimuli that differ in their sequential structure.SIGNIFICANCE STATEMENT Sequence processing has been proposed as a potential precursor of language syntax. As a sequencing operation, the encoding of the temporal order of items within a sequence may help in recognition of relationships between adjacent items and in learning the underlying structure. Taking advantage of the stimulus-specific adaptation phenomenon observed in a high-level auditory region of the zebra finch brain, we addressed this question at the neuronal level. Reordering elements within conspecific songs reinstated robust responses. Neurons also detected changes in the structure of artificial songs, and this detection depended on local transitions between adjacent or nonadjacent syllables. These findings establish the songbird as a model system for deciphering the mechanisms underlying sequence processing at the single-cell level.

Capacities and neural mechanisms for auditory statistical learning across species

Statistical learning has been proposed as a possible mechanism by which individuals can become sensitive to the structures of language fundamental for speech perception. Since its description in human infants, statistical learning has been described in human adults and several non-human species as a general process by which animals learn about stimulus-relevant statistics. The neurobiology of statistical learning is beginning to be understood, but many questions remain about the underlying mechanisms. Why is the developing brain particularly sensitive to stimulus and environmental statistics, and what neural processes are engaged in the adult brain to enable learning from statistical regularities in the absence of external reward or instruction? This review will survey the statistical learning abilities of humans and non-human animals with a particular focus on communicative vocalizations. We discuss the neurobiological basis of statistical learning, and specifically what can be learned by exploring this process in both humans and laboratory animals. Finally, we describe advantages of studying vocal communication in rodents as a means to further our understanding of the cortical plasticity mechanisms engaged during statistical learning. We examine the use of rodents in the context of pup retrieval, which is an auditory-based and experience-dependent form of maternal behavior.

Neural Correlate of Transition Violation and Deviance Detection in the Songbird Auditory Forebrain

Deviants are stimuli that violate one's prediction about the incoming stimuli. Studying deviance detection helps us understand how nervous system learns temporal patterns between stimuli and forms prediction about the future. Detecting deviant stimuli is also critical for animals' survival in the natural environment filled with complex sounds and patterns. Using natural songbird vocalizations as stimuli, we recorded multi-unit and single-unit activity from the zebra finch auditory forebrain while presenting rare repeated stimuli after regular alternating stimuli (alternating oddball experiment) or rare deviant among multiple different common stimuli (context oddball experiment). The alternating oddball experiment showed that neurons were sensitive to rare repetitions in regular alternations. In the absence of expectation, repetition suppresses neural responses to the 2nd stimulus in the repetition. When repetition violates expectation, neural responses to the 2nd stimulus in the repetition were stronger than expected. The context oddball experiment showed that a stimulus elicits stronger neural responses when it is presented infrequently as a deviant among multiple common stimuli. As the acoustic differences between deviant and common stimuli increase, the response enhancement also increases. These results together showed that neural encoding of a stimulus depends not only on the acoustic features of the stimulus but also on the preceding stimuli and the transition patterns between them. These results also imply that the classical oddball effect may result from a combination of repetition suppression and deviance enhancement. Classification analyses showed that the difficulties in decoding the stimulus responsible for the neural responses differed for deviants in different experimental conditions. These findings suggest that learning transition patterns and detecting deviants in natural sequences may depend on a hierarchy of neural mechanisms, which may be involved in more complex forms of auditory processing that depend on the transition patterns between stimuli, such as speech processing.

Auditory and Visual Sequence Learning in Humans and Monkeys using an Artificial Grammar Learning Paradigm

Language flexibly supports the human ability to communicate using different sensory modalities, such as writing and reading in the visual modality and speaking and listening in the auditory domain. Although it has been argued that nonhuman primate communication abilities are inherently multisensory, direct behavioural comparisons between human and nonhuman primates are scant. Artificial grammar learning (AGL) tasks and statistical learning experiments can be used to emulate ordering relationships between words in a sentence. However, previous comparative work using such paradigms has primarily investigated sequence learning within a single sensory modality. We used an AGL paradigm to evaluate how humans and macaque monkeys learn and respond to identically structured sequences of either auditory or visual stimuli. In the auditory and visual experiments, we found that both species were sensitive to the ordering relationships between elements in the sequences. Moreover, the humans and monkeys produced largely similar response patterns to the visual and auditory sequences, indicating that the sequences are processed in comparable ways across the sensory modalities. These results provide evidence that human sequence processing abilities stem from an evolutionarily conserved capacity that appears to operate comparably across the sensory modalities in both human and nonhuman primates. The findings set the stage for future neurobiological studies to investigate the multisensory nature of these sequencing operations in nonhuman primates and how they compare to related processes in humans.

Implicit Memory for Complex Sounds in Higher Auditory Cortex of the Ferret

Responses of auditory cortical neurons encode sound features of incoming acoustic stimuli and also are shaped by stimulus context and history. Previous studies of mammalian auditory cortex have reported a variable time course for such contextual effects ranging from milliseconds to minutes. However, in secondary auditory forebrain areas of songbirds, long-term stimulus-specific neuronal habituation to acoustic stimuli can persist for much longer periods of time, ranging from hours to days. Such long-term habituation in the songbird is a form of long-term auditory memory that requires gene expression. Although such long-term habituation has been demonstrated in avian auditory forebrain, this phenomenon has not previously been described in the mammalian auditory system. Utilizing a similar version of the avian habituation paradigm, we explored whether such long-term effects of stimulus history also occur in auditory cortex of a mammalian auditory generalist, the ferret. Following repetitive presentation of novel complex sounds, we observed significant response habituation in secondary auditory cortex, but not in primary auditory cortex. This long-term habituation appeared to be independent for each novel stimulus and often lasted for at least 20 min. These effects could not be explained by simple neuronal fatigue in the auditory pathway, because time-reversed sounds induced undiminished responses similar to those elicited by completely novel sounds. A parallel set of pupillometric response measurements in the ferret revealed long-term habituation effects similar to observed long-term neural habituation, supporting the hypothesis that habituation to passively presented stimuli is correlated with implicit learning and long-term recognition of familiar sounds.SIGNIFICANCE STATEMENT Long-term habituation in higher areas of songbird auditory forebrain is associated with gene expression and is correlated with recognition memory. Similar long-term auditory habituation in mammals has not been previously described. We studied such habituation in single neurons in the auditory cortex of awake ferrets that were passively listening to repeated presentations of various complex sounds. Responses exhibited long-lasting habituation (at least 20 min) in the secondary, but not primary auditory cortex. Habituation ceased when stimuli were played backward, despite having identical spectral content to the original sound. This long-term neural habituation correlated with similar habituation of ferret pupillary responses to repeated presentations of the same stimuli, suggesting that stimulus habituation is retained as a long-term behavioral memory.

Recent advances in understanding the auditory cortex

Our ability to make sense of the auditory world results from neural processing that begins in the ear, goes through multiple subcortical areas, and continues in the cortex. The specific contribution of the auditory cortex to this chain of processing is far from understood. Although many of the properties of neurons in the auditory cortex resemble those of subcortical neurons, they show somewhat more complex selectivity for sound features, which is likely to be important for the analysis of natural sounds, such as speech, in real-life listening conditions. Furthermore, recent work has shown that auditory cortical processing is highly context-dependent, integrates auditory inputs with other sensory and motor signals, depends on experience, and is shaped by cognitive demands, such as attention. Thus, in addition to being the locus for more complex sound selectivity, the auditory cortex is increasingly understood to be an integral part of the network of brain regions responsible for prediction, auditory perceptual decision-making, and learning. In this review, we focus on three key areas that are contributing to this understanding: the sound features that are preferentially represented by cortical neurons, the spatial organization of those preferences, and the cognitive roles of the auditory cortex.

Neurophysiological Markers of Statistical Learning in Music and Language: Hierarchy, Entropy, and Uncertainty

Statistical learning (SL) is a method of learning based on the transitional probabilities embedded in sequential phenomena such as music and language. It has been considered an implicit and domain-general mechanism that is innate in the human brain and that functions independently of intention to learn and awareness of what has been learned. SL is an interdisciplinary notion that incorporates information technology, artificial intelligence, musicology, and linguistics, as well as psychology and neuroscience. A body of recent study has suggested that SL can be reflected in neurophysiological responses based on the framework of information theory. This paper reviews a range of work on SL in adults and children that suggests overlapping and independent neural correlations in music and language, and that indicates disability of SL. Furthermore, this article discusses the relationships between the order of transitional probabilities (TPs) (i.e., hierarchy of local statistics) and entropy (i.e., global statistics) regarding SL strategies in human's brains; claims importance of information-theoretical approaches to understand domain-general, higher-order, and global SL covering both real-world music and language; and proposes promising approaches for the application of therapy and pedagogy from various perspectives of psychology, neuroscience, computational studies, musicology, and linguistics.

Relative salience of syllable structure and syllable order in zebra finch song

There is a rich history of behavioral and neurobiological research focused on the 'syntax' of birdsong as a model for human language and complex auditory perception. Zebra finches are one of the most widely studied songbird species in this area of investigation. As they produce song syllables in a fixed sequence, it is reasonable to assume that adult zebra finches are also sensitive to the order of syllables within their song; however, results from electrophysiological and behavioral studies provide somewhat mixed evidence on exactly how sensitive zebra finches are to syllable order as compared, say, to syllable structure. Here, we investigate how well adult zebra finches can discriminate changes in syllable order relative to changes in syllable structure in their natural song motifs. In addition, we identify a possible role for experience in enhancing sensitivity to syllable order. We found that both male and female adult zebra finches are surprisingly poor at discriminating changes to the order of syllables within their species-specific song motifs, but are extraordinarily good at discriminating changes to syllable structure (i.e., reversals) in specific syllables. Direct experience or familiarity with a song, either using the bird's own song (BOS) or the song of a flock mate as the test stimulus, improved both male and female zebra finches' sensitivity to syllable order. However, even with experience, birds remained much more sensitive to structural changes in syllables. These results help to clarify some of the ambiguities from the literature on the discriminability of changes in syllable order in zebra finches, provide potential insight on the ethological significance of zebra finch song features, and suggest new avenues of investigation in using zebra finches as animal models for sequential sound processing.

Statistical learning in songbirds: from self-tutoring to song culture

At the onset of vocal development, both songbirds and humans produce variable vocal babbling with broadly distributed acoustic features. Over development, these vocalizations differentiate into the well-defined, categorical signals that characterize adult vocal behaviour. A broadly distributed signal is ideal for vocal exploration, that is, for matching vocal production to the statistics of the sensory input. The developmental transition to categorical signals is a gradual process during which the vocal output becomes differentiated and stable. But does it require categorical input? We trained juvenile zebra finches with playbacks of their own developing song, produced just a few moments earlier, updated continuously over development. Although the vocalizations of these self-tutored (ST) birds were initially broadly distributed, birds quickly developed categorical signals, as fast as birds that were trained with a categorical, adult song template. By contrast, siblings of those birds that received no training (isolates) developed phonological categories much more slowly and never reached the same level of category differentiation as their ST brothers. Therefore, instead of simply mirroring the statistical properties of their sensory input, songbirds actively transform it into distinct categories. We suggest that the early self-generation of phonological categories facilitates the establishment of vocal culture by making the song easier to transmit at the micro level, while promoting stability of shared vocabulary at the group level over generations.

This article is part of the themed issue ‘New frontiers for statistical learning in the cognitive sciences’.

Familiar But Unexpected: Effects of Sound Context Statistics on Auditory Responses in the Songbird Forebrain

Rapid discrimination of salient acoustic signals in the noisy natural environment may depend, not only on specific stimulus features, but also on previous experience that generates expectations about upcoming events. We studied the neural correlates of expectation in the songbird forebrain by using natural vocalizations as stimuli and manipulating the category and familiarity of context sounds. In our paradigm, we recorded bilaterally from auditory neurons in awake adult male zebra finches with multiple microelectrodes during repeated playback of a conspecific song, followed by further playback of this test song in different interleaved sequences with other conspecific or heterospecific songs. Significant enhancement in the auditory response to the test song was seen when its acoustic features differed from the statistical distribution of context song features, but not when it shared the same distribution. Enhancement was also seen when the time of occurrence of the test song was uncertain. These results show that auditory forebrain responses in awake animals in the passive hearing state are modulated dynamically by previous auditory experience and imply that the auditory system can identify the category of a sound based on the global features of the acoustic context. Furthermore, this probability-dependent enhancement in responses to surprising stimuli is independent of stimulus-specific adaptation, which tracks familiarity, suggesting that the two processes could coexist in auditory processing. These findings establish the songbird as a model system for studying these phenomena and contribute to our understanding of statistical learning and the origin of human ERP phenomena to unexpected stimuli.SIGNIFICANCE STATEMENT Traditional auditory neurophysiology has mapped acoustic features of sounds to the response properties of neurons; however, growing evidence suggests that neurons can also encode the probability of sounds. We recorded responses of songbird auditory neurons in a novel paradigm that presented a familiar test stimulus in a sequence with similar or dissimilar sounds. The responses encode, not only stimulus familiarity, but also the expectation for a class of sounds based on the recent statistics of varying sounds in the acoustic context. Our approach thus provides a model system that uses a controlled stimulus paradigm to understand the mechanisms by which top-down processes (expectation and memory) and bottom-up processes (based on stimulus features) interact in sensory coding.

Sequence learning modulates neural responses and oscillatory coupling in human and monkey auditory cortex

Learning complex ordering relationships between sensory events in a sequence is fundamental for animal perception and human communication. While it is known that rhythmic sensory events can entrain brain oscillations at different frequencies, how learning and prior experience with sequencing relationships affect neocortical oscillations and neuronal responses is poorly understood. We used an implicit sequence learning paradigm (an "artificial grammar") in which humans and monkeys were exposed to sequences of nonsense words with regularities in the ordering relationships between the words. We then recorded neural responses directly from the auditory cortex in both species in response to novel legal sequences or ones violating specific ordering relationships. Neural oscillations in both monkeys and humans in response to the nonsense word sequences show strikingly similar hierarchically nested low-frequency phase and high-gamma amplitude coupling, establishing this form of oscillatory coupling-previously associated with speech processing in the human auditory cortex-as an evolutionarily conserved biological process. Moreover, learned ordering relationships modulate the observed form of neural oscillatory coupling in both species, with temporally distinct neural oscillatory effects that appear to coordinate neuronal responses in the monkeys. This study identifies the conserved auditory cortical neural signatures involved in monitoring learned sequencing operations, evident as modulations of transient coupling and neuronal responses to temporally structured sensory input.

Conserved Sequence Processing in Primate Frontal Cortex

An important aspect of animal perception and cognition is learning to recognize relationships between environmental events that predict others in time, a form of relational knowledge that can be assessed using sequence-learning paradigms. Humans are exquisitely sensitive to sequencing relationships, and their combinatorial capacities, most saliently in the domain of language, are unparalleled. Recent comparative research in human and nonhuman primates has obtained behavioral and neuroimaging evidence for evolutionarily conserved substrates involved in sequence processing. The findings carry implications for the origins of domain-general capacities underlying core language functions in humans. Here, we synthesize this research into a 'ventrodorsal gradient' model, where frontal cortex engagement along this axis depends on sequencing complexity, mapping onto the sequencing capacities of different species.

Generalizing prosodic patterns by a non-vocal learning mammal

Prosody, a salient aspect of speech that includes rhythm and intonation, has been shown to help infants acquire some aspects of syntax. Recent studies have shown that birds of two vocal learning species are able to categorize human speech stimuli based on prosody. In the current study, we found that the non-vocal learning rat could also discriminate human speech stimuli based on prosody. Not only that, but rats were able to generalize to novel stimuli they had not been trained with, which suggests that they had not simply memorized the properties of individual stimuli, but learned a prosodic rule. When tested with stimuli with either one or three out of the four prosodic cues removed, the rats did poorly, suggesting that all cues were necessary for the rats to solve the task. This result is in contrast to results with humans and budgerigars, both of which had previously been studied using the same paradigm. Humans and budgerigars both learned the task and generalized to novel items, but were also able to solve the task with some of the cues removed. In conclusion, rats appear to have some of the perceptual abilities necessary to generalize prosodic patterns, in a similar though not identical way to the vocal learning species that have been studied.

Hierarchical emergence of sequence sensitivity in the songbird auditory forebrain

Bengalese finches (Lonchura striata var. domestica) generate more complex sequences in their songs than zebra finches. Because of this, we chose this species to explore the signal processing of sound sequence in the primary auditory forebrain area, field L, and in a secondary area, the caudomedial nidopallium (NCM). We simultaneously recorded activity from multiple single units in urethane-anesthetized birds. We successfully replicated the results of a previous study in awake zebra finches examining stimulus-specific habituation of NCM neurons to conspecific songs. Then, we used an oddball paradigm and compared the neural response to deviant sounds that were presented infrequently, with the response to standard sounds, which were presented frequently. In a single sound oddball task, two different song elements were assigned for the deviant and standard sounds. The response bias to deviant elements was larger in NCM than in field L. In a triplet sequence oddball task, two triplet sequences containing elements ABC and ACB were assigned as the deviant and standard. Only neurons in NCM that displayed broad-shaped spike waveforms had sensitivity to the difference in element order. Our results suggest the hierarchical processing of complex sound sequences in the songbird auditory forebrain.

Juvenile zebra finches learn the underlying structural regularities of their fathers' song

Natural behaviors, such as foraging, tool use, social interaction, birdsong, and language, exhibit branching sequential structure. Such structure should be learnable if it can be inferred from the statistics of early experience. We report that juvenile zebra finches learn such sequential structure in song. Song learning in finches has been extensively studied, and it is generally believed that young males acquire song by imitating tutors (Zann, 1996). Variability in the order of elements in an individual’s mature song occurs, but the degree to which variation in a zebra finch’s song follows statistical regularities has not been quantified, as it has typically been dismissed as production error (Sturdy et al., 1999). Allowing for the possibility that such variation in song is non-random and learnable, we applied a novel analytical approach, based on graph-structured finite-state grammars, to each individual’s full corpus of renditions of songs. This method does not assume syllable-level correspondence between individuals. We find that song variation can be described by probabilistic finite-state graph grammars that are individually distinct, and that the graphs of juveniles are more similar to those of their fathers than to those of other adult males. This grammatical learning is a new parallel between birdsong and language. Our method can be applied across species and contexts to analyze complex variable learned behaviors, as distinct as foraging, tool use, and language.

Developmental insights into mature cognition

Three cases are described that illustrate new ways in which developmental research is informing the study of cognition in adults: statistical learning, neural substrates of cognition, and extended concepts. Developmental research has made clear the ubiquity of statistical learning while also revealing is limitations as a stand-alone way to acquire knowledge. With respect to neural substrates, development has uncovered links between executive processing and fronto-striatal circuits while also pointing to many aspects of high-level cognition that may not be neatly reducible to coherent neural descriptions. For extended concepts, children have made especially clear the weaknesses of intuitive theories in both children and adults while also illustrating other cognitive capacities that are used at all ages to navigate the socially distributed aspects of knowledge.