Recursive syntactic pattern learning by songbirds

Formalising the role of behaviour in neuroscience

We develop a mathematical approach to formally proving that certain neural computations and representations exist based on patterns observed in an organism's behaviour. To illustrate, we provide a simple set of conditions under which an ant's ability to determine how far it is from its nest would logically imply neural structures isomorphic to the natural numbers ℕ . We generalise these results to arbitrary behaviours and representations and show what mathematical characterisation of neural computation and representation is simplest while being maximally predictive of behaviour. We develop this framework in detail using a path integration example, where an organism's ability to search for its nest in the correct location implies representational structures isomorphic to two-dimensional coordinates under addition. We also study a system for processinga n b n strings common in comparative work. Our approach provides an objective way to determine what theory of a physical system is best, addressing a fundamental challenge in neuroscientific inference. These results motivate considering which neurobiological structures have the requisite formal structure and are otherwise physically plausible given relevant physical considerations such as generalisability, information density, thermodynamic stability and energetic cost.

Neural population dynamics in songbird RA and HVC during learned motor-vocal behavior

Complex, learned motor behaviors involve the coordination of large-scale neural activity across multiple brain regions, but our understanding of the population-level dynamics within different regions tied to the same behavior remains limited. Here, we investigate the neural population dynamics underlying learned vocal production in awake-singing songbirds. We use Neuropixels probes to record the simultaneous extracellular activity of populations of neurons in two regions of the vocal motor pathway. In line with observations made in non-human primates during limb-based motor tasks, we show that the population-level activity in both the premotor nucleus HVC and the motor nucleus RA is organized on low-dimensional neural manifolds upon which coordinated neural activity is well described by temporally structured trajectories during singing behavior. Both the HVC and RA latent trajectories provide relevant information to predict vocal sequence transitions between song syllables. However, the dynamics of these latent trajectories differ between regions. Our state-space models suggest a unique and continuous-over-time correspondence between the latent space of RA and vocal output, whereas the corresponding relationship for HVC exhibits a higher degree of neural variability. We then demonstrate that comparable high-fidelity reconstruction of continuous vocal outputs can be achieved from HVC and RA neural latents and spiking activity. Unlike those that use spiking activity, however, decoding models using neural latents generalize to novel sub-populations in each region, consistent with the existence of preserved manifolds that confine vocal-motor activity in HVC and RA.

Network analysis reveals context-dependent structural complexity of social calls in serrate-legged small treefrogs

Vocal communication plays an important role in survival, reproduction, and animal social association. Birds and mammals produce complex vocal sequence to convey context-dependent information. Vocalizations are conspicuous features of the behavior of most anuran species (frogs and toads), and males usually alter their calling strategies according to ecological context to improve the attractiveness/competitiveness. However, very few studies have focused on the variation of vocal sequence in anurans. In the present study, we used both conventional method and network analysis to investigate the context-dependent vocal repertoire, vocal sequence, and call network structure in serrate-legged small treefrogs Kurixalus odontotarsus. We found that male K. odontotarsus modified their vocal sequence by switching to different call types and increasing repertoire size in the presence of a competitive rival. Specifically, compared with before and after the playback of advertisement calls, males emitted fewer advertisement calls, but more aggressive calls, encounter calls, and compound calls during the playback period. Network analysis revealed that the mean degree, mean closeness, and mean betweenness of the call networks significantly decreased during the playback period, which resulted in lower connectivity. In addition, the increased proportion of one-way motifs and average path length also indicated that the connectivity of the call network decreased in competitive context. However, the vocal sequence of K. odontotarsus did not display a clear small-world network structure, regardless of context. Our study presents a paradigm to apply network analysis to vocal sequence in anurans and has important implications for understanding the evolution and function of sequence patterns.

Recursive self-embedded vocal motifs in wild orangutans

Recursive procedures that allow placing a vocal signal inside another of a similar kind provide a neuro-computational blueprint for syntax and phonology in spoken language and human song. There are, however, no known vocal sequences among nonhuman primates arranged in self-embedded patterns that evince vocal recursion or potential incipient or evolutionary transitional forms thereof, suggesting a neuro-cognitive transformation exclusive to humans. Here, we uncover that wild flanged male orangutan long calls feature rhythmically isochronous call sequences nested within isochronous call sequences, consistent with two hierarchical strata. Remarkably, three temporally and acoustically distinct call rhythms in the lower stratum were not related to the overarching rhythm at the higher stratum by any low multiples, which suggests that these recursive structures were neither the result of parallel non-hierarchical procedures nor anatomical artifacts of bodily constraints or resonances. Findings represent a case of temporally recursive hominid vocal combinatorics in the absence of syntax, semantics, phonology, or music. Second-order combinatorics, 'sequences within sequences', involving hierarchically organized and cyclically structured vocal sounds in ancient hominids may have preluded the evolution of recursion in modern language-able humans.

Sequence representation as an early step in the evolution of language

Human language is unique in its compositional, open-ended, and sequential form, and its evolution is often solely explained by advantages of communication. However, it has proven challenging to identify an evolutionary trajectory from a world without language to a world with language, especially while at the same time explaining why such an advantageous phenomenon has not evolved in other animals. Decoding sequential information is necessary for language, making domain-general sequence representation a tentative basic requirement for the evolution of language and other uniquely human phenomena. Here, using formal evolutionary analyses of the utility of sequence representation we show that sequence representation is exceedingly costly and that current memory systems found in animals may prevent abilities necessary for language to emerge. For sequence representation to evolve, flexibility allowing for ignoring irrelevant information is necessary. Furthermore, an abundance of useful sequential information and extensive learning opportunities are required, two conditions that were likely fulfilled early in human evolution. Our results provide a novel, logically plausible trajectory for the evolution of uniquely human cognition and language, and support the hypothesis that human culture is rooted in sequential representational and processing abilities.

Overcoming bias in the comparison of human language and animal communication

Human language is a powerful communicative and cognitive tool. Scholars have long sought to characterize its uniqueness, but each time a property is proposed to set human language apart (e.g., reference, syntax), some (attenuated) version of that property is found in animals. Recently, the uniqueness argument has shifted from linguistic rules to cognitive capacities underlying them. Scholars argue that human language is unique because it relies on ostension and inference, while animal communication depends on simple associations and largely hardwired signals. Such characterizations are often borne out in published data, but these empirical findings are driven by radical differences in the ways animal and human communication are studied. The field of animal communication has been dramatically shaped by the "code model," which imagines communication as involving information packets that are encoded, transmitted, decoded, and interpreted. This framework standardized methods for studying meaning in animal signals, but it does not allow for the nuance, ambiguity, or contextual variation seen in humans. The code model is insidious. It is rarely referenced directly, but it significantly shapes how we study animals. To compare animal communication and human language, we must acknowledge biases resulting from the different theoretical models used. By incorporating new approaches that break away from searching for codes, we may find that animal communication and human language are characterized by differences of degree rather than kind.

Brain-imaging evidence for compression of binary sound sequences in human memory

According to the language-of-thought hypothesis, regular sequences are compressed in human memory using recursive loops akin to a mental program that predicts future items. We tested this theory by probing memory for 16-item sequences made of two sounds. We recorded brain activity with functional MRI and magneto-encephalography (MEG) while participants listened to a hierarchy of sequences of variable complexity, whose minimal description required transition probabilities, chunking, or nested structures. Occasional deviant sounds probed the participants' knowledge of the sequence. We predicted that task difficulty and brain activity would be proportional to the complexity derived from the minimal description length in our formal language. Furthermore, activity should increase with complexity for learned sequences, and decrease with complexity for deviants. These predictions were upheld in both fMRI and MEG, indicating that sequence predictions are highly dependent on sequence structure and become weaker and delayed as complexity increases. The proposed language recruited bilateral superior temporal, precentral, anterior intraparietal, and cerebellar cortices. These regions overlapped extensively with a localizer for mathematical calculation, and much less with spoken or written language processing. We propose that these areas collectively encode regular sequences as repetitions with variations and their recursive composition into nested structures.

Higher-order dialectic variation and syntactic convergence in the complex warble song of budgerigars

Dialectic signatures in animal acoustic signals are key in the identification of and association with group members. Complex vocal sequences may also convey information about behavioral state, and may thus vary according to social environment. Some bird species, such as psittaciforms, learn and modify their complex acoustic signals throughout their lives. However, the structure and function of vocal sequences in open-ended vocal learners remains understudied. Here, we examined vocal sequence variation in the warble song of budgerigars, and how these change upon contact between social groups. Budgerigars are open-ended vocal learners which exhibit fission-fusion flock dynamics in the wild. We found that two captive colonies of budgerigars exhibited colony-specific differences in the syntactic structure of their vocal sequences. Individuals from the two colonies differed in the propensity to repeat certain note types, forming repetitive motifs which served as higher-order signatures of colony identity. When the two groups were brought into contact, their vocal sequences converged, and these colony-specific repetitive patterns disappeared, with males from both erstwhile colonies now producing similar sequences with similar syntactic structure. We present data suggesting that the higher-order temporal arrangement of notes/vocal units is modified throughout life by social learning as groups of birds continually associate and dissociate. Our study sheds light on the importance of examining signal structure at multiple levels of organization, and the potential for psittaciform birds as model systems to examine the influence of learning and social environment on acoustic signals.

The evolution of hierarchically structured communication

Human language sentences are standardly understood as exhibiting considerable hierarchical structure: they can and typically do contain parts that in turn contain parts, etc. In other words, sentences are thought to generally exhibit significant nested part-whole structure. As far as we can tell, this is not a feature of the gestural or vocal communication systems of our great ape relatives. So, one of the many challenges we face in providing a theory of human language evolution is to explain the evolution of hierarchically structured communication in our line. This article takes up that challenge. More specifically, I first present and motivate an account of hierarchical structure in language that departs significantly from the orthodox conception of such structure in linguistics and evolutionary discussions that draw on linguistic theory. On the account I propose, linguistic structure, including hierarchical structure, is treated as a special case of structured action. This account is rooted in the cognitive neuroscience of action, as opposed to (formal) linguistic theory. Among other things, such an account enables us to see how selection for enhanced capacities of act organization and act control in actors, and for act interpretation in observers, might have constructed the brain machinery necessary for the elaborate forms of hierarchically structured communication that we humans engage in. I flesh out this line of thought, emphasizing in particular the role of hominin technique and technology, and the social learning thereof, as evolutionary drivers of this brain machinery.

A test of memory for stimulus sequences in great apes

Identifying cognitive capacities underlying the human evolutionary transition is challenging, and many hypotheses exist for what makes humans capable of, for example, producing and understanding language, preparing meals, and having culture on a grand scale. Instead of describing processes whereby information is processed, recent studies have suggested that there are key differences between humans and other animals in how information is recognized and remembered. Such constraints may act as a bottleneck for subsequent information processing and behavior, proving important for understanding differences between humans and other animals. We briefly discuss different sequential aspects of cognition and behavior and the importance of distinguishing between simultaneous and sequential input, and conclude that explicit tests on non-human great apes have been lacking. Here, we test the memory for stimulus sequences-hypothesis by carrying out three tests on bonobos and one test on humans. Our results show that bonobos' general working memory decays rapidly and that they fail to learn the difference between the order of two stimuli even after more than 2,000 trials, corroborating earlier findings in other animals. However, as expected, humans solve the same sequence discrimination almost immediately. The explicit test on whether bonobos represent stimulus sequences as an unstructured collection of memory traces was not informative as no differences were found between responses to the different probe tests. However, overall, this first empirical study of sequence discrimination on non-human great apes supports the idea that non-human animals, including the closest relatives to humans, lack a memory for stimulus sequences. This may be an ability that sets humans apart from other animals and could be one reason behind the origin of human culture.

Learning and organization of within-session sequences by pigeons (Columba livia)

Most animals engage in complex activities that are the combination of simpler actions expressed over a period of time. The mechanisms organizing such sequential behavior have been of long-standing biological and psychological interest. Previously, we observed pigeons' anticipatory behavior with a within-session sequence involving four choice alternatives suggestive of a potential understanding of the overall order and sequence of the items within a session. In that task, each colored alternative was correct for 24 consecutive trials as presented in a predictable sequence (i.e., A first, then B, then C, then D). To test whether these four already-trained pigeons possessed a sequential and linked representation of the ABCD items, we added a second four-item sequence involving new and distinct colored choice alternatives (i.e., E first for 24 trials, then F, then G, then H) and then alternated these ABCD and EFGH sequences over successive sessions. Over three manipulations, we tested and trained trials composed of combinations of elements from both sequences. We determined that pigeons did not learn any within-sequence associations among the elements. Despite the availability and explicit utility of such sequence cues, the data suggest instead that pigeons learned the discrimination tasks as a series of temporal associations among independent elements. This absence of any sequential linkage is consistent with the hypothesis that such representations are difficult to form in pigeons. This pattern of data suggests that for repeated sequential activities in birds, and potentially other animals including humans, there are highly effective, but underappreciated, clock-like mechanisms that control the ordering of behaviors.

Exploring the neurobiology of Merge at a basic level: insights from a novel artificial grammar paradigm

Introduction

Human language allows us to generate an infinite number of linguistic expressions. It's proposed that this competence is based on a binary syntactic operation, Merge, combining two elements to form a new constituent. An increasing number of recent studies have shifted from complex syntactic structures to two-word constructions to investigate the neural representation of this operation at the most basic level.

Methods

This fMRI study aimed to develop a highly flexible artificial grammar paradigm for testing the neurobiology of human syntax at a basic level. During scanning, participants had to apply abstract syntactic rules to assess whether a given two-word artificial phrase could be further merged with a third word. To control for lower-level template-matching and working memory strategies, an additional non-mergeable word-list task was set up.

Results

Behavioral data indicated that participants complied with the experiment. Whole brain and region of interest (ROI) analyses were performed under the contrast of "structure > word-list." Whole brain analysis confirmed significant involvement of the posterior inferior frontal gyrus [pIFG, corresponding to Brodmann area (BA) 44]. Furthermore, both the signal intensity in Broca's area and the behavioral performance showed significant correlations with natural language performance in the same participants. ROI analysis within the language atlas and anatomically defined Broca's area revealed that only the pIFG was reliably activated.

Discussion

Taken together, these results support the notion that Broca's area, particularly BA 44, works as a combinatorial engine where words are merged together according to syntactic information. Furthermore, this study suggests that the present artificial grammar may serve as promising material for investigating the neurobiological basis of syntax, fostering future cross-species studies.

Evolution of acoustic signals associated with cooperative parental behavior in a poison frog

The emergence of complex social interactions is predicted to be an important selective force in the diversification of communication systems. Parental care presents a key social context in which to study the evolution of novel signals, as care often requires communication and behavioral coordination between parents and is an evolutionary stepping-stone toward increasingly complex social systems. Anuran amphibians (frogs and toads) are a classic model of acoustic communication and the vocal repertoires of many species have been characterized in the contexts of advertisement, courtship, and aggression, yet quantitative descriptions of calls elicited in the context of parental care are lacking. The biparental poison frog, Ranitomeya imitator, exhibits a remarkable parenting behavior in which females, cued by the calls of their male partners, feed tadpoles unfertilized eggs. Here, we characterized and compared calls across three social contexts, for the first time including a parental care context. We found that egg-feeding calls share some properties with both advertisement and courtship calls but also had unique properties. Multivariate analysis revealed high classification success for advertisement and courtship calls but misclassified nearly half of egg feeding calls as either advertisement or courtship calls. Egg feeding and courtship calls both contained less identity information than advertisement calls, as expected for signals used in close-range communication where uncertainty about identity is low and additional signal modalities may be used. Taken together, egg-feeding calls likely borrowed and recombined elements of both ancestral call types to solicit a novel, context-dependent parenting response.

Bat vocal sequences enhance contextual information independently of syllable order

Many animals, humans included, rely on acoustic vocalizations for communication. The complexity of non-human vocal communication has been under debate one of the main open questions being: What could be the function of multi-syllabic vocal sequences? We address these questions by analyzing fruit-bat vocal communication. We use neural networks to encode the vocalizations, and statistical models to examine the information conveyed by sequences of vocalizations. We show that fruit-bat vocal sequences potentially convey more contextual information than individual syllables, but that the order of the syllables within the sequence is unimportant for context. Specifically, sequences are composed of slightly modified syllables, thus increasing the probability of context-specificity. We note that future behavioral, e.g., playback experiments are needed in order to validate the biological relevance of our statistical results. We hypothesize that such sequences might have served as pre-syntax precursors in the evolution of animal communication.

Cognitive Mechanisms Underlying Recursive Pattern Processing in Human Adults

The capacity to generate recursive sequences is a marker of rich, algorithmic cognition, and perhaps unique to humans. Yet, the precise processes driving recursive sequence generation remain mysterious. We investigated three potential cognitive mechanisms underlying recursive pattern processing: hierarchical reasoning, ordinal reasoning, and associative chaining. We developed a Bayesian mixture model to quantify the extent to which these three cognitive mechanisms contribute to adult humans' performance in a sequence generation task. We further tested whether recursive rule discovery depends upon relational information, either perceptual or semantic. We found that the presence of relational information facilitates hierarchical reasoning and drives the generation of recursive sequences across novel depths of center embedding. In the absence of relational information, the use of ordinal reasoning predominates. Our results suggest that hierarchical reasoning is an important cognitive mechanism underlying recursive pattern processing and can be deployed across embedding depths and relational domains.

Coevolution of language and tools in the human brain: An ALE meta-analysis of neural activation during syntactic processing and tool use

Language and complex tool use are often cited as behaviors unique to humans and may be evolutionarily linked owing to the underlying cognitive processes they have in common. We executed a quantitative activation likelihood estimation (ALE) meta-analysis (GingerALE 2.3) on published, whole-brain neuroimaging studies to identify areas associated with syntactic processing and/or tool use in humans. Significant clusters related to syntactic processing were identified in areas known to be related to language production and comprehension, including bilateral Broca's area in the inferior frontal gyrus. Tool use activation clusters were all in the left hemisphere and included the primary motor cortex and premotor cortex, in addition to other areas involved with sensorimotor transformation. Activation shared by syntactic processing and tool use was only significant at one cluster, located in the pars opercularis of the left inferior frontal gyrus. This minimal overlap between syntactic processing and tool use activation from our meta-analysis of neuroimaging studies indicates that there is not a widespread common neural network between the two. Broca's area may serve as an important hub that was initially recruited in early human evolution in the context of simple tool use, but was eventually co-opted for linguistic purposes, including the sequential and hierarchical ordering processes that characterize syntax. In the future, meta-analyses of additional components of language may allow for a more comprehensive examination of the functional networks that underlie the coevolution of human language and complex tool use.

Bare and Constructional Compositionality

This paper proposes a typology of compositionality as manifest in human language and animal communication. At the heart of the typology is a distinction between bare compositionality, in which the meaning of a complex expression is determined solely by the meanings of its constituents, and constructional compositionality, in which the meaning of a complex expression is determined by the meanings of its constituents and also by various aspects of its structure. Bare and constructional compositionality may be observed in human language as well as in various animal communication systems, including primates and birds. Architecturally, bare compositionality provides the foundations for constructional compositionality, while phylogenetically, bare compositionality is a potential starting point for the evolution of constructional compositionality in animal communication and human language.

Lessons learned in animal acoustic cognition through comparisons with humans

Humans are an interesting subject of study in comparative cognition. While humans have a lot of anecdotal and subjective knowledge about their own minds and behaviors, researchers tend not to study humans the way they study other species. Instead, comparisons between humans and other animals tend to be based on either assumptions about human behavior and cognition, or very different testing methods. Here we emphasize the importance of using insider knowledge about humans to form interesting research questions about animal cognition while simultaneously stepping back and treating humans like just another species as if one were an alien researcher. This perspective is extremely helpful to identify what aspects of cognitive processes may be interesting and relevant across the animal kingdom. Here we outline some examples of how this objective human-centric approach has helped us to move forward knowledge in several areas of animal acoustic cognition (rhythm, harmonicity, and vocal units). We describe how this approach works, what kind of benefits we obtain, and how it can be applied to other areas of animal cognition. While an objective human-centric approach is not useful when studying traits that do not occur in humans (e.g., magnetic spatial navigation), it can be extremely helpful when studying traits that are relevant to humans (e.g., communication). Overall, we hope to entice more people working in animal cognition to use a similar approach to maximize the benefits of being part of the animal kingdom while maintaining a detached and scientific perspective on the human species.

Recursive sequence generation in crows

Recursion, the process of embedding structures within similar structures, is often considered a foundation of symbolic competence and a uniquely human capability. To understand its evolution, we can study the recursive aptitudes of nonhuman animals. We adopted the behavioral protocol of a recent study demonstrating that humans and nonhuman primates grasp recursion. We presented sequences of bracket pair stimuli (e.g., [ ] and { }) to crows who were instructed to peck at training lists. They were then tested on their ability to transfer center-embedded structure to never-before-seen pairings of brackets. We reveal that crows have recursive capacities; they perform on par with children and even outperform macaques. The crows continued to produce recursive sequences after extending to longer and thus deeper embeddings. These results demonstrate that recursive capabilities are not limited to the primate genealogy and may have occurred separately from or before human symbolic competence in different animal taxa.

Detecting non-adjacent dependencies is the exception rather than the rule

Statistical learning refers to our sensitivity to the distributional properties of our environment. Humans have been shown to readily detect the dependency relationship of events that occur adjacently in a stream of stimuli but processing non-adjacent dependencies (NADs) appears more challenging. In the present study, we tested the ability of human participants to detect NADs in a new Hebb-naming task that has been proposed recently to study regularity detection in a noisy environment. In three experiments, we found that most participants did not manage to extract NADs. These results suggest that the ability to learn NADs in noise is the exception rather than the rule. They provide new information about the limits of statistical learning mechanisms.

Syntactic chunking reveals a core syntactic representation of multi-digit numbers, which is generative and automatic

Representing the base-10 structure of numbers is a challenging cognitive ability, unique to humans, but it is yet unknown how precisely this is done. Here, we examined whether and how literate adults represent a number’s full syntactic structure. In 5 experiments, participants repeated number-word sequences and we systematically varied the order of words within each sequence. Repetition on grammatical sequences (e.g., two hundred ninety-seven) was better than on non-grammatical ones (hundred seven two ninety). We conclude that the participants represented the number’s full syntactic structure and used it to merge number words into chunks in short-term memory. Accuracy monotonously improved for sequences with increasingly longer grammatical segments, up to a limit of ~ 4 words per segment, irrespectively of the number of digits, and worsened thereafter. Namely, short chunks improved memorization, whereas oversized chunks disrupted memorization. This chunk size limit suggests that the chunks are not based on predefined structures, whose size limit is not expected to be so low, but are created ad hoc by a generative process, such as the hierarchical syntactic representation hypothesized in Michael McCloskey’s number-processing model. Chunking occurred even when it disrupted performance, as in the oversized chunks, and even when external cues for chunking were controlled for or were removed. We conclude that the above generative process operates automatically rather than voluntarily. To date, this is the most detailed account of the core representation of the syntactic structure of numbers—a critical aspect of numerical literacy and of the ability to read and write numbers.

Neurosymbolic Systems of Perception and Cognition: The Role of Attention

A cognitive architecture aimed at cumulative learning must provide the necessary information and control structures to allow agents to learn incrementally and autonomously from their experience. This involves managing an agent's goals as well as continuously relating sensory information to these in its perception-cognition information processing stack. The more varied the environment of a learning agent is, the more general and flexible must be these mechanisms to handle a wider variety of relevant patterns, tasks, and goal structures. While many researchers agree that information at different levels of abstraction likely differs in its makeup and structure and processing mechanisms, agreement on the particulars of such differences is not generally shared in the research community. A dual processing architecture (often referred to as System-1 and System-2) has been proposed as a model of cognitive processing, and they are often considered as responsible for low- and high-level information, respectively. We posit that cognition is not binary in this way and that knowledge at any level of abstraction involves what we refer to as neurosymbolic information, meaning that data at both high and low levels must contain both symbolic and subsymbolic information. Further, we argue that the main differentiating factor between the processing of high and low levels of data abstraction can be largely attributed to the nature of the involved attention mechanisms. We describe the key arguments behind this view and review relevant evidence from the literature.

Syntax Acquisition in Healthy Adults and Post-Stroke Individuals: The Intriguing Role of Grammatical Preference, Statistical Learning, and Education

Previous work has provided contrasting evidence on syntax acquisition. Syntax-internal factors, i.e., instinctive knowledge of the universals of grammar (UG) for finite-state grammar (FSG) and phrase-structure grammar (PSG) but also syntax-external factors such as language competence, working memory (WM) and demographic factors may affect syntax acquisition. This study employed an artificial grammar paradigm to identify which factors predicted syntax acquisition. Thirty-seven healthy individuals and forty-nine left-hemispheric stroke patients (fourteen with aphasia) read syllable sequences adhering to or violating FSG and PSG. They performed preference classifications followed by grammatical classifications (after training). Results showed the best classification accuracy for sequences adhering to UG, with performance predicted by syntactic competence and spatial WM. Classification of ungrammatical sequences improved after training and was predicted by verbal WM. Although accuracy on FSG was better than on PSG, generalization was fully possible only for PSG. Education was the best predictor of syntax acquisition, while aphasia and lesion volume were not predictors. This study shows a clear preference for UG, which is influenced by spatial and linguistic knowledge, but not by the presence of aphasia. Verbal WM supported the identification of rule violations. Moreover, the acquisition of FSG and PSG was related to partially different mechanisms, but both depended on education.

Long-distance dependencies in birdsong syntax

Songbird syntax is generally thought to be simple, in particular lacking long-distance dependencies in which one element affects choice of another occurring considerably later in the sequence. Here, we test for long-distance dependencies in the sequences of songs produced by song sparrows (Melospiza melodia). Song sparrows sing with eventual variety, repeating each song type in a consecutive series termed a 'bout'. We show that in switching between song types, song sparrows follow a 'cycling rule', cycling through their repertoires in close to the minimum possible number of bouts. Song sparrows do not cycle in a set order but rather vary the order of song types from cycle to cycle. Cycling in a variable order strongly implies long-distance dependencies, in which choice of the next type depends on the song types sung over the past cycle, in the range of 9-10 bouts. Song sparrows also follow a 'bout length rule', whereby the number of repetitions of a song type in a bout is positively associated with the length of the interval until that type recurs. This rule requires even longer distance dependencies that cross one another; such dependencies are characteristic of more complex levels of syntax than previously attributed to non-human animals.

Modelling how cleaner fish approach an ephemeral reward task demonstrates a role for ecologically tuned chunking in the evolution of advanced cognition

What makes cognition “advanced” is an open and not precisely defined question. One perspective involves increasing the complexity of associative learning, from conditioning to learning sequences of events (“chaining”) to representing various cue combinations as “chunks.” Here we develop a weighted graph model to study the mechanism enabling chunking ability and the conditions for its evolution and success, based on the ecology of the cleaner fish Labroides dimidiatus. In some environments, cleaners must learn to serve visitor clients before resident clients, because a visitor leaves if not attended while a resident waits for service. This challenge has been captured in various versions of the ephemeral reward task, which has been proven difficult for a range of cognitively capable species. We show that chaining is the minimal requirement for solving this task in its common simplified laboratory format that involves repeated simultaneous exposure to an ephemeral and permanent food source. Adding ephemeral–ephemeral and permanent–permanent combinations, as cleaners face in the wild, requires individuals to have chunking abilities to solve the task. Importantly, chunking parameters need to be calibrated to ecological conditions in order to produce adaptive decisions. Thus, it is the fine-tuning of this ability, which may be the major target of selection during the evolution of advanced associative learning.

What makes cognition ‘advanced’ is an open and not precisely defined question. In this study, a cognitive model of cleaner fish learning the ephemeral-reward task demonstrates how a critical step in cognitive evolution may be understood as the evolution of chunking and its tuning to fit ecological conditions.

Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions

Recently developed methods in computational neuroethology have enabled increasingly detailed and comprehensive quantification of animal movements and behavioral kinematics. Vocal communication behavior is well poised for application of similar large-scale quantification methods in the service of physiological and ethological studies. This review describes emerging techniques that can be applied to acoustic and vocal communication signals with the goal of enabling study beyond a small number of model species. We review a range of modern computational methods for bioacoustics, signal processing, and brain-behavior mapping. Along with a discussion of recent advances and techniques, we include challenges and broader goals in establishing a framework for the computational neuroethology of vocal communication.

Neural circuits and symbolic processing

The ability to use symbols is a defining feature of human intelligence. However, neuroscience has yet to explain the fundamental neural circuit mechanisms for flexibly representing and manipulating abstract concepts. This article will review the research on neural models for symbolic processing. The review first focuses on the question of how symbols could possibly be represented in neural circuits. The review then addresses how neural symbolic representations could be flexibly combined to meet a wide range of reasoning demands. Finally, the review assesses the research on program synthesis and proposes that the most flexible neural representation of symbolic processing would involve the capacity to rapidly synthesize neural operations analogous to lambda calculus to solve complex cognitive tasks.

Overlooked evidence for semantic compositionality and signal reduction in wild chimpanzees (Pan troglodytes)

Recent discoveries of semantic compositionality in Japanese tits have enlivened the discussions on the presence of this phenomenon in wild animal communication. Data on semantic compositionality in wild apes are lacking, even though language experiments with captive apes have demonstrated they are capable of semantic compositionality. In this paper, I revisit the study by Boesch (Hum. Evol. 6:81–89, 1991) who investigated drumming sequences by an alpha male in a chimpanzee (Pan troglodytes) community in the Taï National Park, Côte d’Ivoire. A reanalysis of the data reveals that the alpha male produced semantically compositional combined messages of travel direction change and resting period initiation. Unlike the Japanese tits, the elements of the compositional expression were not simply juxtaposed but displayed structural reduction, while one of the two elements in the expression coded the meanings of both elements. These processes show relative resemblance to blending and fusion in human languages. Also unlike the tits, the elements of the compositional expression did not have a fixed order, although there was a fixed distribution of drumming events across the trees used for drumming. Because the elements of the expression appear to carry verb-like meanings, the compositional expression also resembles simple verb-verb constructions and short paratactic combinations of two clauses found across languages. In conclusion, the reanalysis suggests that semantic compositionality and phenomena resembling paratactic combinations of two clauses might have been present in the communication of the last common ancestor of chimpanzees and humans, not necessarily in the vocal modality.

Evolving the capacity for socially guided vocal learning in songbirds: a preliminary study

Socially guided vocal learning, the ability to use contingent reactions from social partners to guide immature vocalizations to more mature forms, is thought to be a rare ability known to be used only by humans, marmosets and two unrelated songbird species (brown-headed cowbirds and zebra finches). However, this learning strategy has never been investigated in the vast majority of species that are known to modify their vocalizations over development. We propose a novel, preliminary evolutionary modelling approach that uses ecological, reproductive and developmental traits to predict which species may incorporate social influences as part of their vocal learning system. We demonstrate our model using data from 28 passerines. We found three highly predictive traits: temporal overlap between sensory (memorization) and sensorimotor (practice) phases of song learning, song used for mate attraction, and social gregariousness outside the breeding season. Species with these traits were distributed throughout the clade, suggesting that a trait-based approach may yield new insights into the evolution of learning strategies that cannot be gleaned from phylogenetic relatedness alone. Our model suggests several previously uninvestigated and unexpected species as likely socially guided vocal learners and offers new insight into the evolution and development of vocal learning. This article is part of the theme issue 'Vocal learning in animals and humans'.

European starlings (sturnus vulgaris) discriminate rhythms by rate, not temporal patterns

Humans can perceive a regular psychological pulse in music known as the beat. The evolutionary origins and neural mechanisms underlying this ability are hypothetically linked to imitative vocal learning, a rare trait found only in some species of mammals and birds. Beat perception has been demonstrated in vocal learning parrots but not in songbirds. We trained European starlings (Sturnus vulgaris) on two sound discriminations to investigate their perception of the beat and temporal structure in rhythmic patterns. First, we trained birds on a two-choice discrimination between rhythmic patterns of tones that contain or lack a regular beat. Despite receiving extensive feedback, the starlings were unable to distinguish the first two patterns. Next, we probed the temporal cues that starlings use for discriminating rhythms in general. We trained birds to discriminate a baseline set of isochronous and triplet tone sequences. On occasional probe trials, we presented transformations of the baseline patterns. The starlings' responses to the probes suggest they relied on absolute temporal features to sort the sounds into "fast" and "slow" and otherwise ignored patterns that were present. Our results support that starlings attend to local features in rhythms and are less sensitive to the global temporal organization.

Why Are No Animal Communication Systems Simple Languages?

Individuals of some animal species have been taught simple versions of human language despite their natural communication systems failing to rise to the level of a simple language. How is it, then, that some animals can master a version of language, yet none of them deploy this capacity in their own communication system? I first examine the key design features that are often used to evaluate language-like properties of natural animal communication systems. I then consider one candidate animal system, bird song, because it has several of the key design features or their precursors, including social learning and cultural transmission of their vocal signals. I conclude that although bird song communication is nuanced and complex, and has the acoustic potential for productivity, it is not productive – it cannot be used to say many different things. Finally, I discuss the debate over whether animal communication should be viewed as a cooperative information transmission process, as we typically view human language, or as a competitive process where signaler and receiver vie for control. The debate points to a necessary condition for the evolution of a simple language that has generally been overlooked: the degree of to which the interests of the signaler and receiver align. While strong cognitive and signal production mechanisms are necessary pre-adaptations for a simple language, they are not sufficient. Also necessary is the existence of identical or near-identical interests of signaler and receiver and a socio-ecology that requires high-level cooperation across a range of contexts. In the case of our hominid ancestors, these contexts included hunting, gathering, child care and, perhaps, warfare. I argue that the key condition for the evolution of human language was the extreme interdependency that existed among unrelated individuals in the hunter-gatherer societies of our hominid ancestors. This extreme interdependency produced multiple prosocial adaptations for effective intragroup cooperation, which in partnership with advanced cognitive abilities, set the stage for the evolution of language.

High-capacity auditory memory for vocal communication in a social songbird

Effective vocal communication often requires the listener to recognize the identity of a vocalizer, and this recognition is dependent on the listener's ability to form auditory memories. We tested the memory capacity of a social songbird, the zebra finch, for vocalizer identities using conditioning experiments and found that male and female zebra finches can remember a large number of vocalizers (mean, 42) based solely on the individual signatures found in their songs and distance calls. These memories were formed within a few trials, were generalized to previously unheard renditions, and were maintained for up to a month. A fast and high-capacity auditory memory for vocalizer identity has not been demonstrated previously in any nonhuman animals and is an important component of vocal communication in social species.

Statistical Learning of Unfamiliar Sounds as Trajectories Through a Perceptual Similarity Space

In typical statistical learning studies, researchers define sequences in terms of the probability of the next item in the sequence given the current item (or items), and they show that high probability sequences are treated as more familiar than low probability sequences. Existing accounts of these phenomena all assume that participants represent statistical regularities more or less as they are defined by the experimenters-as sequential probabilities of symbols in a string. Here we offer an alternative, or possibly supplementary, hypothesis. Specifically, rather than identifying or labeling individual stimuli discretely in order to predict the next item in a sequence, we need only assume that the participant is able to represent the stimuli as evincing particular similarity relations to one another, with sequences represented as trajectories through this similarity space. We present experiments in which this hypothesis makes sharply different predictions from hypotheses based on the assumption that sequences are learned over discrete, labeled stimuli. We also present a series of simulation models that encode stimuli as positions in a continuous two-dimensional space, and predict the next location from the current location. Although no model captures all of the data presented here, the results of three critical experiments are more consistent with the view that participants represent trajectories through similarity space rather than sequences of discrete labels under particular conditions.

An evolutionary account of intermodality differences in statistical learning

The cognitive mechanisms underlying statistical learning are engaged for the purposes of speech processing and language acquisition. However, these mechanisms are shared by a wide variety of species that do not possess the language faculty. Moreover, statistical learning operates across domains, including nonlinguistic material. Ancient mechanisms for segmenting continuous sensory input into discrete constituents have evolved for general-purpose segmentation of the environment and been readopted for processing linguistic input. Linguistic input provides a rich set of cues for the boundaries between sequential constituents. Such input engages a wider variety of more specialized mechanisms operating on these language-specific cues, thus potentially reducing the role of conditional statistics in tokenizing a continuous linguistic stream. We provide an explicit within-subject comparison of the utility of statistical learning in language versus nonlanguage domains across the visual and auditory modalities. The results showed that in the auditory modality statistical learning is more efficient with speech-like input, while in the visual modality efficiency is higher with nonlanguage input. We suggest that the speech faculty has been important for individual fitness for an extended period, leading to the adaptation of statistical learning mechanisms for speech processing. This is not the case in the visual modality, in which linguistic material presents a less ecological type of sensory input.

Nonadjacent dependency processing in monkeys, apes, and humans

The ability to track syntactic relationships between words, particularly over distances ("nonadjacent dependencies"), is a critical faculty underpinning human language, although its evolutionary origins remain poorly understood. While some monkey species are reported to process auditory nonadjacent dependencies, comparative data from apes are missing, complicating inferences regarding shared ancestry. Here, we examined nonadjacent dependency processing in common marmosets, chimpanzees, and humans using "artificial grammars": strings of arbitrary acoustic stimuli composed of adjacent (nonhumans) or nonadjacent (all species) dependencies. Individuals from each species (i) generalized the grammars to novel stimuli and (ii) detected grammatical violations, indicating that they processed the dependencies between constituent elements. Furthermore, there was no difference between marmosets and chimpanzees in their sensitivity to nonadjacent dependencies. These notable similarities between monkeys, apes, and humans indicate that nonadjacent dependency processing, a crucial cognitive facilitator of language, is an ancestral trait that evolved at least ~40 million years before language itself.

Emotional Voice Intonation: A Communication Code at the Origins of Speech Processing and Word-Meaning Associations?

The aim of the present work is to investigate the facilitating effect of vocal emotional intonation on the evolution of the following processes involved in language: (a) identifying and producing phonemes, (b) processing compositional rules underlying vocal utterances, and (c) associating vocal utterances with meanings. To this end, firstly, I examine research on the presence of these abilities in animals, and the biologically ancient nature of emotional vocalizations. Secondly, I review research attesting to the facilitating effect of emotional voice intonation on these abilities in humans. Thirdly, building on these studies in animals and humans, and through taking an evolutionary perspective, I provide insights for future empirical work on the facilitating effect of emotional intonation on these three processes in animals and preverbal humans. In this work, I highlight the importance of a comparative approach to investigate language evolution empirically. This review supports Darwin’s hypothesis, according to which the ability to express emotions through voice modulation was a key step in the evolution of spoken language.

Exploring Variation Between Artificial Grammar Learning Experiments: Outlining a Meta-Analysis Approach

Artificial grammar learning (AGL) has become an important tool used to understand aspects of human language learning and whether the abilities underlying learning may be unique to humans or found in other species. Successful learning is typically assumed when human or animal participants are able to distinguish stimuli generated by the grammar from those that are not at a level better than chance. However, the question remains as to what subjects actually learn in these experiments. Previous studies of AGL have frequently introduced multiple potential contributors to performance in the training and testing stimuli, but meta-analysis techniques now enable us to consider these multiple information sources for their contribution to learning-enabling intended and unintended structures to be assessed simultaneously. We present a blueprint for meta-analysis approaches to appraise the effect of learning in human and other animal studies for a series of artificial grammar learning experiments, focusing on studies that examine auditory and visual modalities. We identify a series of variables that differ across these studies, focusing on both structural and surface properties of the grammar, and characteristics of training and test regimes, and provide a first step in assessing the relative contribution of these design features of artificial grammars as well as species-specific effects for learning.

Structured Sequence Learning: Animal Abilities, Cognitive Operations, and Language Evolution

Human language is a salient example of a neurocognitive system that is specialized to process complex dependencies between sensory events distributed in time, yet how this system evolved and specialized remains unclear. Artificial Grammar Learning (AGL) studies have generated a wealth of insights into how human adults and infants process different types of sequencing dependencies of varying complexity. The AGL paradigm has also been adopted to examine the sequence processing abilities of nonhuman animals. We critically evaluate this growing literature in species ranging from mammals (primates and rats) to birds (pigeons, songbirds, and parrots) considering also cross-species comparisons. The findings are contrasted with seminal studies in human infants that motivated the work in nonhuman animals. This synopsis identifies advances in knowledge and where uncertainty remains regarding the various strategies that nonhuman animals can adopt for processing sequencing dependencies. The paucity of evidence in the few species studied to date and the need for follow-up experiments indicate that we do not yet understand the limits of animal sequence processing capacities and thereby the evolutionary pattern. This vibrant, yet still budding, field of research carries substantial promise for advancing knowledge on animal abilities, cognitive substrates, and language evolution.

Recursive sequence generation in monkeys, children, U.S. adults, and native Amazonians

The question of what computational capacities, if any, differ between humans and nonhuman animals has been at the core of foundational debates in cognitive psychology, anthropology, linguistics, and animal behavior. The capacity to form nested hierarchical representations is hypothesized to be essential to uniquely human thought, but its origins in evolution, development, and culture are controversial. We used a nonlinguistic sequence generation task to test whether subjects generalize sequential groupings of items to a center-embedded, recursive structure. Children (3 to 5 years old), U.S. adults, and adults from a Bolivian indigenous group spontaneously induced recursive structures from ambiguous training data. In contrast, monkeys did so only with additional exposure. We quantify these patterns using a Bayesian mixture model over logically possible strategies. Our results show that recursive hierarchical strategies are robust in human thought, both early in development and across cultures, but the capacity itself is not unique to humans.

Baboons (Papio papio) Process a Context-Free but Not a Context-Sensitive Grammar

Language processing involves the ability to master supra-regular grammars, that go beyond the level of complexity of regular grammars. This ability has been hypothesized to be a uniquely human capacity. Our study probed baboons' capacity to learn two supra-regular grammars of different levels of complexity: a context-free grammar generating sequences following a mirror structure (e.g., AB | BA, ABC | CBA) and a context-sensitive grammar generating sequences following a repeat structure (e.g., AB | AB, ABC | ABC), the latter requiring greater computational power to be processed. Fourteen baboons were tested in a prediction task, requiring them to track a moving target on a touchscreen. In distinct experiments, sequences of target locations followed one of the above two grammars, with rare violations. Baboons showed slower response times when violations occurred in mirror sequences, but did not react to violations in repeat sequences, suggesting that they learned the context-free (mirror) but not the context-sensitive (repeat) grammar. By contrast, humans tested with the same task learned both grammars. These data suggest a difference in sensitivity in baboons between a context-free and a context-sensitive grammar.

Somatodendritic consistency check for temporal feature segmentation

The brain identifies potentially salient features within continuous information streams to process hierarchical temporal events. This requires the compression of information streams, for which effective computational principles are yet to be explored. Backpropagating action potentials can induce synaptic plasticity in the dendrites of cortical pyramidal neurons. By analogy with this effect, we model a self-supervising process that increases the similarity between dendritic and somatic activities where the somatic activity is normalized by a running average. We further show that a family of networks composed of the two-compartment neurons performs a surprisingly wide variety of complex unsupervised learning tasks, including chunking of temporal sequences and the source separation of mixed correlated signals. Common methods applicable to these temporal feature analyses were previously unknown. Our results suggest the powerful ability of neural networks with dendrites to analyze temporal features. This simple neuron model may also be potentially useful in neural engineering applications.

The authors propose a learning rule for a neuron model with dendrite. In their model, somatodendritic interaction implements self-supervised learning applicable to a wide range of sequence learning tasks, including spike pattern detection, chunking temporal input and blind source separation.

Statistical learning for vocal sequence acquisition in a songbird

Birdsong is a learned communicative behavior that consists of discrete acoustic elements (“syllables”) that are sequenced in a controlled manner. While the learning of the acoustic structure of syllables has been extensively studied, relatively little is known about sequence learning in songbirds. Statistical learning could contribute to the acquisition of vocal sequences, and we investigated the nature and extent of sequence learning at various levels of song organization in the Bengalese finch, Lonchura striata var. domestica. We found that, under semi-natural conditions, pupils (sons) significantly reproduced the sequence statistics of their tutor’s (father’s) songs at multiple levels of organization (e.g., syllable repertoire, prevalence, and transitions). For example, the probability of syllable transitions at “branch points” (relatively complex sequences that are followed by multiple types of transitions) were significantly correlated between the songs of tutors and pupils. We confirmed the contribution of learning to sequence similarities between fathers and sons by experimentally tutoring juvenile Bengalese finches with the songs of unrelated tutors. We also discovered that the extent and fidelity of sequence similarities between tutors and pupils were significantly predicted by the prevalence of sequences in the tutor’s song and that distinct types of sequence modifications (e.g., syllable additions or deletions) followed distinct patterns. Taken together, these data provide compelling support for the role of statistical learning in vocal production learning and identify factors that could modulate the extent of vocal sequence learning.

Merge-Generability as the Key Concept of Human Language: Evidence From Neuroscience

Ever since the inception of generative linguistics, various dependency patterns have been widely discussed in the literature, particularly as they pertain to the hierarchy based on “weak generation” – the so-called Chomsky Hierarchy. However, humans can make any possible dependency patterns by using artificial means on a sequence of symbols (e.g., computer programing). The differences between sentences in human language and general symbol sequences have been routinely observed, but the question as to why such differences exist has barely been raised. Here, we address this problem and propose a theoretical explanation in terms of a new concept of “Merge-generability,” that is, whether the structural basis for a given dependency is provided by the fundamental operation Merge. In our functional magnetic resonance imaging (fMRI) study, we tested the judgments of noun phrase (NP)-predicate (Pred) pairings in sentences of Japanese, an SOV language that allows natural, unbounded nesting configurations. We further introduced two pseudo-adverbs, which artificially force dependencies that do not conform to structures generated by Merge, i.e., non-Merge-generable; these adverbs enable us to manipulate Merge-generability (Natural or Artificial). By employing this novel paradigm, we obtained the following results. Firstly, the behavioral data clearly showed that an NP-Pred matching task became more demanding under the Artificial conditions than under the Natural conditions, reflecting cognitive loads that could be covaried with the increased number of words. Secondly, localized activation in the left frontal cortex, as well as in the left middle temporal gyrus and angular gyrus, was observed for the [Natural – Artificial] contrast, indicating specialization of these left regions in syntactic processing. Any activation due to task difficulty was completely excluded from activations in these regions, because the Natural conditions were always easier than the Artificial ones. And finally, the [Artificial – Natural] contrast resulted in the dorsal portion of the left frontal cortex, together with wide-spread regions required for general cognitive demands. These results indicate that Merge-generable sentences are processed in these specific regions in contrast to non-Merge-generable sentences, demonstrating that Merge is indeed a fundamental operation, which comes into play especially under the Natural conditions.

Sound sequences in birdsong: how much do birds really care?

The complex and melodic nature of many birds' songs has raised interest in potential parallels between avian vocal sequences and human speech. The similarities between birdsong and speech in production and learning are well established, but surprisingly little is known about how birds perceive song sequences. One popular laboratory songbird, the zebra finch (Taeniopygia guttata), has recently attracted attention as an avian model for human speech, in part because the male learns to produce the individual elements in its song motif in a fixed sequence. But psychoacoustic evidence shows that adult zebra finches are relatively insensitive to the sequential features of song syllables. Instead, zebra finches and other birds seem to be exquisitely sensitive to the acoustic details of individual syllables to a degree that is beyond human hearing capacity. Based on these findings, we present a finite-state model of zebra finch perception of song syllable sequences and discuss the rich informational capacity of their vocal system. Furthermore, we highlight the abilities of budgerigars (Melopsittacus undulatus), a parrot species, to hear sequential features better than zebra finches and suggest that neurophysiological investigations comparing these species could prove fruitful for uncovering neural mechanisms for auditory sequence perception in human speech. This article is part of the theme issue 'What can animal communication teach us about human language?'

Bio-Linguistics: Monkeys Break Through the Syntax Barrier

Macaque monkeys can be trained to produce complex spatial sequences beyond the simplest levels of grammar previously known from animal studies. This indicates cognitive capabilities in the spatial-motor domain that approach the computational complexity level of human syntax.