A new model of sensorimotor coupling in the development of speech

Cross-modal plasticity in children with cochlear implant: converging evidence from EEG and functional near-infrared spectroscopy

Over the first years of life, the brain undergoes substantial organization in response to environmental stimulation. In a silent world, it may promote vision by (i) recruiting resources from the auditory cortex and (ii) making the visual cortex more efficient. It is unclear when such changes occur and how adaptive they are, questions that children with cochlear implants can help address. Here, we examined 7-18 years old children: 50 had cochlear implants, with delayed or age-appropriate language abilities, and 25 had typical hearing and language. High-density electroencephalography and functional near-infrared spectroscopy were used to evaluate cortical responses to a low-level visual task. Evidence for a 'weaker visual cortex response' and 'less synchronized or less inhibitory activity of auditory association areas' in the implanted children with language delays suggests that cross-modal reorganization can be maladaptive and does not necessarily strengthen the dominant visual sense.

A systematic review of altered resting-state networks in early deafness and implications for cochlear implantation outcomes

Auditory deprivation following congenital/pre-lingual deafness (C/PD) can drastically affect brain development and its functional organisation. This systematic review intends to extend current knowledge of the impact of C/PD and deafness duration on brain resting-state networks (RSNs), review changes in RSNs and spoken language outcomes post-cochlear implant (CI) and draw conclusions for future research. The systematic literature search followed the PRISMA guideline. Two independent reviewers searched four electronic databases using combined keywords: 'auditory deprivation', 'congenital/prelingual deafness', 'resting-state functional connectivity' (RSFC), 'resting-state fMRI' and 'cochlear implant'. Seventeen studies (16 cross-sectional and one longitudinal) met the inclusion criteria. Using the Crowe Critical Appraisal Tool, the publications' quality was rated between 65.0% and 92.5% (mean: 84.10%), ≥80% in 13 out of 17 studies. A few studies were deficient in sampling and/or ethical considerations. According to the findings, early auditory deprivation results in enhanced RSFC between the auditory network and brain networks involved in non-verbal communication, and high levels of spontaneous neural activity in the auditory cortex before CI are evidence of occupied auditory cortical areas with other sensory modalities (cross-modal plasticity) and sub-optimal CI outcomes. Overall, current evidence supports the idea that moreover intramodal and cross-modal plasticity, the entire brain adaptation following auditory deprivation contributes to spoken language development and compensatory behaviours.

Motor Processing in Children With Cochlear Implants as Assessed by Functional Near-Infrared Spectroscopy

Auditory-motor and visual-motor networks are often coupled in daily activities, such as when listening to music and dancing; but these networks are known to be highly malleable as a function of sensory input. Thus, congenital deafness may modify neural activities within the connections between the motor, auditory, and visual cortices. Here, we investigated whether the cortical responses of children with cochlear implants (CI) to a simple and repetitive motor task would differ from that of children with typical hearing (TH) and we sought to understand whether this response related to their language development. Participants were 75 school-aged children, including 50 with CI (with varying language abilities) and 25 controls with TH. We used functional near-infrared spectroscopy (fNIRS) to record cortical responses over the whole brain, as children squeezed the back triggers of a joystick that vibrated or not with the squeeze. Motor cortex activity was reflected by an increase in oxygenated hemoglobin concentration (HbO) and a decrease in deoxygenated hemoglobin concentration (HbR) in all children, irrespective of their hearing status. Unexpectedly, the visual cortex (supposedly an irrelevant region) was deactivated in this task, particularly for children with CI who had good language skills when compared to those with CI who had language delays. Presence or absence of vibrotactile feedback made no difference in cortical activation. These findings support the potential of fNIRS to examine cognitive functions related to language in children with CI.

The acquisition of speech categories: Beyond perceptual narrowing, beyond unsupervised learning and beyond infancy

An early achievement in language is carving a variable acoustic space into categories. The canonical story is that infants accomplish this by the second year, when only unsupervised learning is plausible. I challenge this view, synthesizing five lines of developmental, phonetic and computational work. First, unsupervised learning may be insufficient given the statistics of speech (including infant-directed). Second, evidence that infants "have" speech categories rests on tenuous methodological assumptions. Third, the fact that the ecology of the learning environment is unsupervised does not rule out more powerful error driven learning mechanisms. Fourth, several implicit supervisory signals are available to older infants. Finally, development is protracted through adolescence, enabling richer avenues for development. Infancy may be a time of organizing the auditory space, but true categorization only arises via complex developmental cascades later in life. This has implications for critical periods, second language acquisition, and our basic framing of speech perception.

LaDIVA: A neurocomputational model providing laryngeal motor control for speech acquisition and production

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (fo) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA’s laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal fo perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.

Do Infants Really Learn Phonetic Categories?

Early changes in infants’ ability to perceive native and nonnative speech sound contrasts are typically attributed to their developing knowledge of phonetic categories. We critically examine this hypothesis and argue that there is little direct evidence of category knowledge in infancy. We then propose an alternative account in which infants’ perception changes because they are learning a perceptual space that is appropriate to represent speech, without yet carving up that space into phonetic categories. If correct, this new account has substantial implications for understanding early language development.

Alterations of Regional Homogeneity in Children With Congenital Sensorineural Hearing Loss: A Resting-State fMRI Study

Background: Brain functional alterations have been observed in children with congenital sensorineural hearing loss (CSNHL). The purpose of this study was to assess the alterations of regional homogeneity in children with CSNHL.

Methods: Forty-five children with CSNHL and 20 healthy controls were enrolled into this study. Brain resting-state functional MRI (rs-fMRI) for regional homogeneity including the Kendall coefficient consistency (KCC-ReHo) and the coherence-based parameter (Cohe-ReHo) was analyzed and compared between the two groups, i.e., the CSNHL group and the healthy control group.

Results: Compared to the healthy controls, children with CSNHL showed increased Cohe-ReHo values in left calcarine and decreased values in bilateral ventrolateral prefrontal cortex (VLPFC) and right dorsolateral prefrontal cortex (DLPFC). Children with CSNHL also had increased KCC-ReHo values in the left calcarine, cuneus, precentral gyrus, and right superior parietal lobule (SPL) and decreased values in the left VLPFC and right DLPFC. Correlations were detected between the ReHo values and age of the children with CSNHL. There were positive correlations between ReHo values in the pre-cuneus/pre-frontal cortex and age (p < 0.05). There were negative correlations between ReHo values in bilateral temporal lobes, fusiform gyrus, parahippocampal gyrus and precentral gyrus, and age (p < 0.05).

Conclusion: Children with CSNHL had RoHo alterations in the auditory, visual, motor, and other related brain cortices as compared to the healthy controls with normal hearing. There were significant correlations between ReHo values and age in brain regions involved in information integration and processing. Our study showed promising data using rs-fMRI ReHo parameters to assess brain functional alterations in children with CSNHL.

Orthogonal neural codes for speech in the infant brain

Creating invariant representations from an everchanging speech signal is a major challenge for the human brain. Such an ability is particularly crucial for preverbal infants who must discover the phonological, lexical, and syntactic regularities of an extremely inconsistent signal in order to acquire language. Within the visual domain, an efficient neural solution to overcome variability consists in factorizing the input into a reduced set of orthogonal components. Here, we asked whether a similar decomposition strategy is used in early speech perception. Using a 256-channel electroencephalographic system, we recorded the neural responses of 3-mo-old infants to 120 natural consonant-vowel syllables with varying acoustic and phonetic profiles. Using multivariate pattern analyses, we show that syllables are factorized into distinct and orthogonal neural codes for consonants and vowels. Concerning consonants, we further demonstrate the existence of two stages of processing. A first phase is characterized by orthogonal and context-invariant neural codes for the dimensions of manner and place of articulation. Within the second stage, manner and place codes are integrated to recover the identity of the phoneme. We conclude that, despite the paucity of articulatory motor plans and speech production skills, pre-babbling infants are already equipped with a structured combinatorial code for speech analysis, which might account for the rapid pace of language acquisition during the first year.

Characterization of Speech and Language Phenotype in GLUT1DS

Background: To analyze the oral motor, speech and language phenotype in a sample of pediatric patients with GLUT 1 transporter deficiency syndrome (GLUT1DS). Methods: eight Italian-speaking children with GLUT1DS (aged 4.6–15.4 years) in stable treatment with ketogenic diet from a variable time underwent a specific and standardized speech and language assessment battery. Results: All patients showed deficits with different degrees of impairment in multiple speech and language areas. In particular, orofacial praxis, parallel and total movements were the most impaired in the oromotor domain; in the speech domain patients obtained a poor performance in the diadochokinesis rate and in the repetition of words that resulted as severely deficient in seven out of eight patients; in the language domain the most affected abilities were semantic/phonological fluency and receptive grammar. Conclusions: GLUT1DS is associated to different levels of speech and language impairment, which should guide diagnostic and therapeutic intervention. Larger population data are needed to identify more precisely a speech and language profile in GLUT1DS patients.

Goal-Directed Exploration for Learning Vowels and Syllables: A Computational Model of Speech Acquisition

Infants learn to speak rapidly during their first years of life, gradually improving from simple vowel-like sounds to larger consonant-vowel complexes. Learning to control their vocal tract in order to produce meaningful speech sounds is a complex process which requires to learn the relationship between motor and sensory processes. In this paper, a computational framework is proposed that models the problem of learning articulatory control for a physiologically plausible 3-D vocal tract model using a developmentally-inspired approach. The system babbles and explores efficiently in a low-dimensional space of goals that are relevant to the learner in its synthetic environment. The learning process is goal-directed and self-organized, and yields an inverse model of the mapping between sensory space and motor commands. This study provides a unified framework that can be used for learning static as well as dynamic motor representations. The successful learning of vowel and syllable sounds as well as the benefit of active and adaptive learning strategies are demonstrated. Categorical perception is found in the acquired models, suggesting that the framework has the potential to replicate phenomena of human speech acquisition.

Preverbal Production and Early Lexical Development in Children With Cochlear Implants: A Longitudinal Study Following Pre-implanted Children Until 12 Months After Cochlear Implant Activation

Studies have shown that children vary in the trajectories of their language development after cochlear implant (CI) activation. The aim of the present study is to assess the preverbal and lexical development of a group of 20 Italian-speaking children observed longitudinally before CI activation and at three, 6 and 12 months after CI surgery (mean age at the first session: 17.5 months; SD: 8.3; and range: 10–35). The group of children with CIs (G-CI) was compared with two groups of normally-hearing (NH) children, one age-matched (G-NHA; mean age at the first session: 17.4 months; SD: 8.0; and range: 10–34) and one language-matched (G-NHL; n = 20; mean age at the first session: 11.2 months; SD: 0.4; and range: 11–12). The spontaneous interactions between children and their mothers during free-play were transcribed. Preverbal babbling production and first words were considered for each child. Data analysis showed significant differences in babbling and word production between groups, with a lower production of words in children with CIs compared to the G-NHA group and a higher production of babbling compared to the G-NHL children. Word production 1 year after activation was significantly lower for the children with CIs than for language-matched children only when maternal education was controlled for. Furthermore, latent class growth analysis showed that children with CIs belonged mainly to classes that exhibited a low level of initial production but also progressive increases over time. Babbling production had a statistically significant effect on lexical growth but not on class membership, and only for groups showing slower and constant increases. Results highlight the importance of preverbal vocal patterns for later lexical development and may support families and speech therapists in the early identification of risk and protective factors for language delay in children with CIs.

A Neurobiologically Constrained Cortex Model of Semantic Grounding With Spiking Neurons and Brain-Like Connectivity

One of the most controversial debates in cognitive neuroscience concerns the cortical locus of semantic knowledge and processing in the human brain. Experimental data revealed the existence of various cortical regions relevant for meaning processing, ranging from semantic hubs generally involved in semantic processing to modality-preferential sensorimotor areas involved in the processing of specific conceptual categories. Why and how the brain uses such complex organization for conceptualization can be investigated using biologically constrained neurocomputational models. Here, we improve pre-existing neurocomputational models of semantics by incorporating spiking neurons and a rich connectivity structure between the model ‘areas’ to mimic important features of the underlying neural substrate. Semantic learning and symbol grounding in action and perception were simulated by associative learning between co-activated neuron populations in frontal, temporal and occipital areas. As a result of Hebbian learning of the correlation structure of symbol, perception and action information, distributed cell assembly circuits emerged across various cortices of the network. These semantic circuits showed category-specific topographical distributions, reaching into motor and visual areas for action- and visually-related words, respectively. All types of semantic circuits included large numbers of neurons in multimodal connector hub areas, which is explained by cortical connectivity structure and the resultant convergence of phonological and semantic information on these zones. Importantly, these semantic hub areas exhibited some category-specificity, which was less pronounced than that observed in primary and secondary modality-preferential cortices. The present neurocomputational model integrates seemingly divergent experimental results about conceptualization and explains both semantic hubs and category-specific areas as an emergent process causally determined by two major factors: neuroanatomical connectivity structure and correlated neuronal activation during language learning.

Modality-independent recruitment of inferior frontal cortex during speech processing in human infants

Despite increasing interest in the development of audiovisual speech perception in infancy, the underlying mechanisms and neural processes are still only poorly understood. In addition to regions in temporal cortex associated with speech processing and multimodal integration, such as superior temporal sulcus, left inferior frontal cortex (IFC) has been suggested to be critically involved in mapping information from different modalities during speech perception. To further illuminate the role of IFC during infant language learning and speech perception, the current study examined the processing of auditory, visual and audiovisual speech in 6-month-old infants using functional near-infrared spectroscopy (fNIRS). Our results revealed that infants recruit speech-sensitive regions in frontal cortex including IFC regardless of whether they processed unimodal or multimodal speech. We argue that IFC may play an important role in associating multimodal speech information during the early steps of language learning.

Integración auditiva, visual y propioceptiva como sustrato del desarrollo del lenguaje

Introducción. El desarrollo del lenguaje es un proceso complejo considerado como marcador evolutivo del ser humano y puede ser comprendido a partir de la contribución de los sistemas sensoriales y de los eventos que ocurren en periodos críticos del desarrollo.Objetivo. Realizar una revisión de cómo se da la integración de la información auditiva, visual y propioceptiva y cómo se refleja en el desarrollo del lenguaje, destacando el papel de la interacción social como contexto que favorece este proceso.Materiales y métodos. Se utilizaron los términos MeSH “Language Development”; “Visual Perception”; “Hearing”; y “Proprioception en las bases de datos MEDLINE y Embase, limitando la búsqueda principal a artículos escritos en inglés, español y portugués.Resultados. El punto de partida lo constituye la información auditiva, la cual, en el primer año de vida, permite la discriminación de los elementos del ambiente que corresponden al lenguaje; luego un pico en su adquisición y posteriormente una etapa de máxima discriminación lingüística. La información visual proporciona la correspondencia del lenguaje en imágenes, sustrato de nominación y comprensión de palabras, además de la interpretación e imitación del componente emocional en la gesticulación. La información propioceptiva ofrece la retroalimentación de los patrones de ejecución motora empleados en la producción del lenguaje.Conclusión. El estudio del desarrollo lenguaje desde la integración sensorial ofrece nuevas perspectivas para el abordaje e intervención de sus desviaciones.

Neural reuse of action perception circuits for language, concepts and communication

Neurocognitive and neurolinguistics theories make explicit statements relating specialized cognitive and linguistic processes to specific brain loci. These linking hypotheses are in need of neurobiological justification and explanation. Recent mathematical models of human language mechanisms constrained by fundamental neuroscience principles and established knowledge about comparative neuroanatomy offer explanations for where, when and how language is processed in the human brain. In these models, network structure and connectivity along with action- and perception-induced correlation of neuronal activity co-determine neurocognitive mechanisms. Language learning leads to the formation of action perception circuits (APCs) with specific distributions across cortical areas. Cognitive and linguistic processes such as speech production, comprehension, verbal working memory and prediction are modelled by activity dynamics in these APCs, and combinatorial and communicative-interactive knowledge is organized in the dynamics within, and connections between APCs. The network models and, in particular, the concept of distributionally-specific circuits, can account for some previously not well understood facts about the cortical 'hubs' for semantic processing and the motor system's role in language understanding and speech sound recognition. A review of experimental data evaluates predictions of the APC model and alternative theories, also providing detailed discussion of some seemingly contradictory findings. Throughout, recent disputes about the role of mirror neurons and grounded cognition in language and communication are assessed critically.

Advantages of comparative studies in songbirds to understand the neural basis of sensorimotor integration

Sensorimotor integration is the process through which the nervous system creates a link between motor commands and associated sensory feedback. This process allows for the acquisition and refinement of many behaviors, including learned communication behaviors such as speech and birdsong. Consequently, it is important to understand fundamental mechanisms of sensorimotor integration, and comparative analyses of this process can provide vital insight. Songbirds offer a powerful comparative model system to study how the nervous system links motor and sensory information for learning and control. This is because the acquisition, maintenance, and control of birdsong critically depend on sensory feedback. Furthermore, there is an incredible diversity of song organizations across songbird species, ranging from songs with simple, stereotyped sequences to songs with complex sequencing of vocal gestures, as well as a wide diversity of song repertoire sizes. Despite this diversity, the neural circuitry for song learning, control, and maintenance remains highly similar across species. Here, we highlight the utility of songbirds for the analysis of sensorimotor integration and the insights about mechanisms of sensorimotor integration gained by comparing different songbird species. Key conclusions from this comparative analysis are that variation in song sequence complexity seems to covary with the strength of feedback signals in sensorimotor circuits and that sensorimotor circuits contain distinct representations of elements in the vocal repertoire, possibly enabling evolutionary variation in repertoire sizes. We conclude our review by highlighting important areas of research that could benefit from increased comparative focus, with particular emphasis on the integration of new technologies.

Neurocomputational Consequences of Evolutionary Connectivity Changes in Perisylvian Language Cortex

The human brain sets itself apart from that of its primate relatives by specific neuroanatomical features, especially the strong linkage of left perisylvian language areas (frontal and temporal cortex) by way of the arcuate fasciculus (AF). AF connectivity has been shown to correlate with verbal working memory-a specifically human trait providing the foundation for language abilities-but a mechanistic explanation of any related causal link between anatomical structure and cognitive function is still missing. Here, we provide a possible explanation and link, by using neurocomputational simulations in neuroanatomically structured models of the perisylvian language cortex. We compare networks mimicking key features of cortical connectivity in monkeys and humans, specifically the presence of relatively stronger higher-order "jumping links" between nonadjacent perisylvian cortical areas in the latter, and demonstrate that the emergence of working memory for syllables and word forms is a functional consequence of this structural evolutionary change. We also show that a mere increase of learning time is not sufficient, but that this specific structural feature, which entails higher connectivity degree of relevant areas and shorter sensorimotor path length, is crucial. These results offer a better understanding of specifically human anatomical features underlying the language faculty and their evolutionary selection advantage.SIGNIFICANCE STATEMENT Why do humans have superior language abilities compared to primates? Recently, a uniquely human neuroanatomical feature has been demonstrated in the strength of the arcuate fasciculus (AF), a fiber pathway interlinking the left-hemispheric language areas. Although AF anatomy has been related to linguistic skills, an explanation of how this fiber bundle may support language abilities is still missing. We use neuroanatomically structured computational models to investigate the consequences of evolutionary changes in language area connectivity and demonstrate that the human-specific higher connectivity degree and comparatively shorter sensorimotor path length implicated by the AF entail emergence of verbal working memory, a prerequisite for language learning. These results offer a better understanding of specifically human anatomical features for language and their evolutionary selection advantage.

In search of a learning model

While the four commentaries reflect a range of different perspectives on my target paper (Vihman, 2017), all basically accept the overall approach, which has been central to my research for 30 years. Each commentary proposes ways of deepening aspects of the ideas expressed or points out limitations and potential areas in which elaboration would be useful. This response takes up each commentary in turn.

The hearing ear is always found close to the speaking tongue: Review of the role of the motor system in speech perception

Does "the motor system" play "a role" in speech perception? If so, where, how, and when? We conducted a systematic review that addresses these questions using both qualitative and quantitative methods. The qualitative review of behavioural, computational modelling, non-human animal, brain damage/disorder, electrical stimulation/recording, and neuroimaging research suggests that distributed brain regions involved in producing speech play specific, dynamic, and contextually determined roles in speech perception. The quantitative review employed region and network based neuroimaging meta-analyses and a novel text mining method to describe relative contributions of nodes in distributed brain networks. Supporting the qualitative review, results show a specific functional correspondence between regions involved in non-linguistic movement of the articulators, covertly and overtly producing speech, and the perception of both nonword and word sounds. This distributed set of cortical and subcortical speech production regions are ubiquitously active and form multiple networks whose topologies dynamically change with listening context. Results are inconsistent with motor and acoustic only models of speech perception and classical and contemporary dual-stream models of the organization of language and the brain. Instead, results are more consistent with complex network models in which multiple speech production related networks and subnetworks dynamically self-organize to constrain interpretation of indeterminant acoustic patterns as listening context requires.

The role of left inferior frontal cortex during audiovisual speech perception in infants

In the first year of life, infants' speech perception attunes to their native language. While the behavioral changes associated with native language attunement are fairly well mapped, the underlying mechanisms and neural processes are still only poorly understood. Using fNIRS and eye tracking, the current study investigated 6-month-old infants' processing of audiovisual speech that contained matching or mismatching auditory and visual speech cues. Our results revealed that infants' speech-sensitive brain responses in inferior frontal brain regions were lateralized to the left hemisphere. Critically, our results further revealed that speech-sensitive left inferior frontal regions showed enhanced responses to matching when compared to mismatching audiovisual speech, and that infants with a preference to look at the speaker's mouth showed an enhanced left inferior frontal response to speech compared to infants with a preference to look at the speaker's eyes. These results suggest that left inferior frontal regions play a crucial role in associating information from different modalities during native language attunement, fostering the formation of multimodal phonological categories.

Perception drives production across sensory modalities: A network for sensorimotor integration of visual speech

Sensory information is critical for movement control, both for defining the targets of actions and providing feedback during planning or ongoing movements. This holds for speech motor control as well, where both auditory and somatosensory information have been shown to play a key role. Recent clinical research demonstrates that individuals with severe speech production deficits can show a dramatic improvement in fluency during online mimicking of an audiovisual speech signal suggesting the existence of a visuomotor pathway for speech motor control. Here we used fMRI in healthy individuals to identify this new visuomotor circuit for speech production. Participants were asked to perceive and covertly rehearse nonsense syllable sequences presented auditorily, visually, or audiovisually. The motor act of rehearsal, which is prima facie the same whether or not it is cued with a visible talker, produced different patterns of sensorimotor activation when cued by visual or audiovisual speech (relative to auditory speech). In particular, a network of brain regions including the left posterior middle temporal gyrus and several frontoparietal sensorimotor areas activated more strongly during rehearsal cued by a visible talker versus rehearsal cued by auditory speech alone. Some of these brain regions responded exclusively to rehearsal cued by visual or audiovisual speech. This result has significant implications for models of speech motor control, for the treatment of speech output disorders, and for models of the role of speech gesture imitation in development.

Learning to Produce Syllabic Speech Sounds via Reward-Modulated Neural Plasticity

At around 7 months of age, human infants begin to reliably produce well-formed syllables containing both consonants and vowels, a behavior called canonical babbling. Over subsequent months, the frequency of canonical babbling continues to increase. How the infant's nervous system supports the acquisition of this ability is unknown. Here we present a computational model that combines a spiking neural network, reinforcement-modulated spike-timing-dependent plasticity, and a human-like vocal tract to simulate the acquisition of canonical babbling. Like human infants, the model's frequency of canonical babbling gradually increases. The model is rewarded when it produces a sound that is more auditorily salient than sounds it has previously produced. This is consistent with data from human infants indicating that contingent adult responses shape infant behavior and with data from deaf and tracheostomized infants indicating that hearing, including hearing one's own vocalizations, is critical for canonical babbling development. Reward receipt increases the level of dopamine in the neural network. The neural network contains a reservoir with recurrent connections and two motor neuron groups, one agonist and one antagonist, which control the masseter and orbicularis oris muscles, promoting or inhibiting mouth closure. The model learns to increase the number of salient, syllabic sounds it produces by adjusting the base level of muscle activation and increasing their range of activity. Our results support the possibility that through dopamine-modulated spike-timing-dependent plasticity, the motor cortex learns to harness its natural oscillations in activity in order to produce syllabic sounds. It thus suggests that learning to produce rhythmic mouth movements for speech production may be supported by general cortical learning mechanisms. The model makes several testable predictions and has implications for our understanding not only of how syllabic vocalizations develop in infancy but also for our understanding of how they may have evolved.

Infant-caregiver interactions affect the early development of vocalization

Vocal communication is a unique means to bilaterally exchange messages in real-time. The developmental origin of such communication is the vocal interactions between an infant and a caregiver, and one of the big mysteries is how the infant learns to vocalize the mother tongue of the caregiver. Many theories claim to explain an infant's capability to imitate a caregiver based on acoustic matching. However, the acoustic qualities of the infant and the caregiver are quite different, and, therefore, cannot fully explain the imitation. Instead, the interaction itself may have an important role, but the mechanism is still unclear. In this article, we review studies addressing this problem using constructive approaches based on cognitive developmental robotics.

Learning to pronounce first words in three languages: an investigation of caregiver and infant behavior using a computational model of an infant

Words are made up of speech sounds. Almost all accounts of child speech development assume that children learn the pronunciation of first language (L1) speech sounds by imitation, most claiming that the child performs some kind of auditory matching to the elements of ambient speech. However, there is evidence to support an alternative account and we investigate the non-imitative child behavior and well-attested caregiver behavior that this account posits using Elija, a computational model of an infant. Through unsupervised active learning, Elija began by discovering motor patterns, which produced sounds. In separate interaction experiments, native speakers of English, French and German then played the role of his caregiver. In their first interactions with Elija, they were allowed to respond to his sounds if they felt this was natural. We analyzed the interactions through phonemic transcriptions of the caregivers' utterances and found that they interpreted his output within the framework of their native languages. Their form of response was almost always a reformulation of Elija's utterance into well-formed sounds of L1. Elija retained those motor patterns to which a caregiver responded and formed associations between his motor pattern and the response it provoked. Thus in a second phase of interaction, he was able to parse input utterances in terms of the caregiver responses he had heard previously, and respond using his associated motor patterns. This capacity enabled the caregivers to teach Elija to pronounce some simple words in their native languages, by his serial imitation of the words' component speech sounds. Overall, our results demonstrate that the natural responses and behaviors of human subjects to infant-like vocalizations can take a computational model from a biologically plausible initial state through to word pronunciation. This provides support for an alternative to current auditory matching hypotheses for how children learn to pronounce.

The emergence of mirror-like response properties from domain-general principles in vision and audition

Like Cook et al., we suggest that mirror neurons are a fascinating product of cross-modal learning. As predicted by an associative account, responses in motor regions are observed for novel and/or abstract visual stimuli such as point-light and android movements. Domain-specific mirror responses also emerge as a function of audiomotor expertise that is slowly acquired over years of intensive training.

Semantic embodiment, disembodiment or misembodiment? In search of meaning in modules and neuron circuits

"Embodied" proposals claim that the meaning of at least some words, concepts and constructions is grounded in knowledge about actions and objects. An alternative "disembodied" position locates semantics in a symbolic system functionally detached from sensorimotor modules. This latter view is not tenable theoretically and has been empirically falsified by neuroscience research. A minimally-embodied approach now claims that action-perception systems may "color", but not represent, meaning; however, such minimal embodiment (misembodiment?) still fails to explain why action and perception systems exert causal effects on the processing of symbols from specific semantic classes. Action perception theory (APT) offers neurobiological mechanisms for "embodied" referential, affective and action semantics along with "disembodied" mechanisms of semantic abstraction, generalization and symbol combination, which draw upon multimodal brain systems. In this sense, APT suggests integrative-neuromechanistic explanations of why both sensorimotor and multimodal areas of the human brain differentially contribute to specific facets of meaning and concepts.

Prespeech motor learning in a neural network using reinforcement

Vocal motor development in infancy provides a crucial foundation for language development. Some significant early accomplishments include learning to control the process of phonation (the production of sound at the larynx) and learning to produce the sounds of one's language. Previous work has shown that social reinforcement shapes the kinds of vocalizations infants produce. We present a neural network model that provides an account of how vocal learning may be guided by reinforcement. The model consists of a self-organizing map that outputs to muscles of a realistic vocalization synthesizer. Vocalizations are spontaneously produced by the network. If a vocalization meets certain acoustic criteria, it is reinforced, and the weights are updated to make similar muscle activations increasingly likely to recur. We ran simulations of the model under various reinforcement criteria and tested the types of vocalizations it produced after learning in the different conditions. When reinforcement was contingent on the production of phonated (i.e. voiced) sounds, the network's post-learning productions were almost always phonated, whereas when reinforcement was not contingent on phonation, the network's post-learning productions were almost always not phonated. When reinforcement was contingent on both phonation and proximity to English vowels as opposed to Korean vowels, the model's post-learning productions were more likely to resemble the English vowels and vice versa.

Sensorimotor learning and the ontogeny of the mirror neuron system

Mirror neurons, which have now been found in the human and songbird as well as the macaque, respond to both the observation and the performance of the same action. It has been suggested that their matching response properties have evolved as an adaptation for action understanding; alternatively, these properties may arise through sensorimotor experience. Here I review mirror neuron response characteristics from the perspective of ontogeny; I discuss the limited evidence for mirror neurons in early development; and I describe the growing body of evidence suggesting that mirror neuron responses can be modified through experience, and that sensorimotor experience is the critical type of experience for producing mirror neuron responses.

Early vocalization of preterm infants with extremely low birth weight (ELBW), Part II: From canonical babbling up to the appearance of the first word

The aim of this study was to systematically describe the preverbal development of preterm infants from canonical babbling up to the first word and to compare it with that of healthy full-term infants. In addition, the amount of vocalization between the preterm and full-term groups was compared. The sample consisted of 18 preterm infants with extremely low birth weight and 11 full-term infants. The development of preverbal vocalization before variegated babbling did not differ between the groups. Instead, the preterm infants failed to produce more different kinds of canonical syllable types than the full-term infants. However, they showed a larger variance of variegated babbling skills and remained in the babbling phase longer before reaching the first meaningful word compared with the full-term infants. Following the onset of canonical babbling, the preterm infants produced fewer vocalizations than the full-term infants and they reached the first word later than the full-term infants.

Do production patterns influence the processing of speech in prelinguistic infants?

The headturn preference procedure was used to test 18 infants on their response to three different passages chosen to reflect their individual production patterns. The passages contained nonwords with consonants in one of three categories: (a) often produced by that infant ('own'), (b) rarely produced by that infant but common at that age ('other'), and (c) not generally produced by infants. Infants who had a single 'own' consonant showed no significant preference for either 'own' (a) or 'other' (b) passages. In contrast, infants' with two 'own' consonants exhibited greater attention to 'other' passages (b). Both groups attended equally to the passage featuring consonants rarely produced by infants of that age (c). An analysis of a sample of the infant-directed speech ruled out the mothers' speech as a source of the infant preferences. The production-based shift to a focus on the 'other' passage suggests that nascent production abilities combine with emergent perceptual experience to facilitate word learning.

The role of vocal practice in constructing phonological working memory

PURPOSE

In this study, the authors looked for effects of vocal practice on phonological working memory.

METHOD

A longitudinal design was used, combining both naturalistic observations and a nonword repetition test. Fifteen 26-month-olds (12 of whom were followed from age 11 months) were administered a nonword test including real words, "standard" nonwords (identical for all children), and nonwords based on individual children's production inventory (in and out words).

RESULTS

A strong relationship was found between (a) length of experience with consonant production and (b) nonword repetition and between (a) differential experience with specific consonants through production and (b) performance on the in versus out words.

CONCLUSIONS

Performance depended on familiarity with words or their subunits and was strongest for real words, weaker for in words, and weakest for out words. The results demonstrate the important role of speech production in the construction of phonological working memory.

Neural development of networks for audiovisual speech comprehension

Everyday conversation is both an auditory and a visual phenomenon. While visual speech information enhances comprehension for the listener, evidence suggests that the ability to benefit from this information improves with development. A number of brain regions have been implicated in audiovisual speech comprehension, but the extent to which the neurobiological substrate in the child compares to the adult is unknown. In particular, developmental differences in the network for audiovisual speech comprehension could manifest through the incorporation of additional brain regions, or through different patterns of effective connectivity. In the present study we used functional magnetic resonance imaging and structural equation modeling (SEM) to characterize the developmental changes in network interactions for audiovisual speech comprehension. The brain response was recorded while children 8- to 11-years-old and adults passively listened to stories under audiovisual (AV) and auditory-only (A) conditions. Results showed that in children and adults, AV comprehension activated the same fronto-temporo-parietal network of regions known for their contribution to speech production and perception. However, the SEM network analysis revealed age-related differences in the functional interactions among these regions. In particular, the influence of the posterior inferior frontal gyrus/ventral premotor cortex on supramarginal gyrus differed across age groups during AV, but not A speech. This functional pathway might be important for relating motor and sensory information used by the listener to identify speech sounds. Further, its development might reflect changes in the mechanisms that relate visual speech information to articulatory speech representations through experience producing and perceiving speech.

Active perception: sensorimotor circuits as a cortical basis for language

Action and perception are functionally linked in the brain, but a hotly debated question is whether perception and comprehension of stimuli depend on motor circuits. Brain language mechanisms are ideal for addressing this question. Neuroimaging investigations have found specific motor activations when subjects understand speech sounds, word meanings and sentence structures. Moreover, studies involving transcranial magnetic stimulation and patients with lesions affecting inferior frontal regions of the brain have shown contributions of motor circuits to the comprehension of phonemes, semantic categories and grammar. These data show that language comprehension benefits from frontocentral action systems, indicating that action and perception circuits are interdependent.

Recruitment and Consolidation of Cell Assemblies for Words by Way of Hebbian Learning and Competition in a Multi-Layer Neural Network

Current cognitive theories postulate either localist representations of knowledge or fully overlapping, distributed ones. We use a connectionist model that closely replicates known anatomical properties of the cerebral cortex and neurophysiological principles to show that Hebbian learning in a multi-layer neural network leads to memory traces (cell assemblies) that are both distributed and anatomically distinct. Taking the example of word learning based on action-perception correlation, we document mechanisms underlying the emergence of these assemblies, especially (i) the recruitment of neurons and consolidation of connections defining the kernel of the assembly along with (ii) the pruning of the cell assembly's halo (consisting of very weakly connected cells). We found that, whereas a learning rule mapping covariance led to significant overlap and merging of assemblies, a neurobiologically grounded synaptic plasticity rule with fixed LTP/LTD thresholds produced minimal overlap and prevented merging, exhibiting competitive learning behaviour. Our results are discussed in light of current theories of language and memory. As simulations with neurobiologically realistic neural networks demonstrate here spontaneous emergence of lexical representations that are both cortically dispersed and anatomically distinct, both localist and distributed cognitive accounts receive partial support.

Verbal working memory and language production: Common approaches to the serial ordering of verbal information

Verbal working memory (WM) tasks typically involve the language production architecture for recall; however, language production processes have had a minimal role in theorizing about WM. A framework for understanding verbal WM results is presented here. In this framework, domain-specific mechanisms for serial ordering in verbal WM are provided by the language production architecture, in which positional, lexical, and phonological similarity constraints are highly similar to those identified in the WM literature. These behavioral similarities are paralleled in computational modeling of serial ordering in both fields. The role of long-term learning in serial ordering performance is emphasized, in contrast to some models of verbal WM. Classic WM findings are discussed in terms of the language production architecture. The integration of principles from both fields illuminates the maintenance and ordering mechanisms for verbal information.

Task specificity in early oral motor development

This article addresses a long-standing clinical and theoretical debate regarding the potential relationship between speech and nonspeech behaviors in the developing system. The review is motivated by the high popularity of nonspeech oral motor exercises (NSOMEs), including alimentary behaviors such as chewing, in the treatment of speech disorders in young children. The similarities and differences in the behavioral characteristics, sensory requirements, and task goals for speech and nonspeech oromotor behaviors are compared. Integrated theoretical paradigms and empirical data on the development of early oromotor behaviors are discussed. Although the efficacy of NSOMEs remains empirically untested at this time, studies of typical developmental speech physiology fail to support a theoretical framework promoting the use of NSOMEs. Well-designed empirical studies are necessary, however, to establish the efficacy of NSOMEs for specific clinical population and treatment targets.

Précis of neuroconstructivism: how the brain constructs cognition

Neuroconstructivism: How the Brain Constructs Cognition proposes a unifying framework for the study of cognitive development that brings together (1) constructivism (which views development as the progressive elaboration of increasingly complex structures), (2) cognitive neuroscience (which aims to understand the neural mechanisms underlying behavior), and (3) computational modeling (which proposes formal and explicit specifications of information processing). The guiding principle of our approach is context dependence, within and (in contrast to Marr [1982]) between levels of organization. We propose that three mechanisms guide the emergence of representations: competition, cooperation, and chronotopy; which themselves allow for two central processes: proactivity and progressive specialization. We suggest that the main outcome of development is partial representations, distributed across distinct functional circuits. This framework is derived by examining development at the level of single neurons, brain systems, and whole organisms. We use the terms encellment, embrainment, and embodiment to describe the higher-level contextual influences that act at each of these levels of organization. To illustrate these mechanisms in operation we provide case studies in early visual perception, infant habituation, phonological development, and object representations in infancy. Three further case studies are concerned with interactions between levels of explanation: social development, atypical development and within that, developmental dyslexia. We conclude that cognitive development arises from a dynamic, contextual change in embodied neural structures leading to partial representations across multiple brain regions and timescales, in response to proactively specified physical and social environment.

A neuroanatomically grounded Hebbian-learning model of attention-language interactions in the human brain

Meaningful familiar stimuli and senseless unknown materials lead to different patterns of brain activation. A late major neurophysiological response indexing ‘sense’ is the negative component of event-related potential peaking at around 400 ms (N400), an event-related potential that emerges in attention-demanding tasks and is larger for senseless materials (e.g. meaningless pseudowords) than for matched meaningful stimuli (words). However, the mismatch negativity (latency 100–250 ms), an early automatic brain response elicited under distraction, is larger to words than to pseudowords, thus exhibiting the opposite pattern to that seen for the N400. So far, no theoretical account has been able to reconcile and explain these findings by means of a single, mechanistic neural model. We implemented a neuroanatomically grounded neural network model of the left perisylvian language cortex and simulated: (i) brain processes of early language acquisition and (ii) cortical responses to familiar word and senseless pseudoword stimuli. We found that variation of the area-specific inhibition (the model correlate of attention) modulated the simulated brain response to words and pseudowords, producing either an N400- or a mismatch negativity-like response depending on the amount of inhibition (i.e. available attentional resources). Our model: (i) provides a unifying explanatory account, at cortical level, of experimental observations that, so far, had not been given a coherent interpretation within a single framework; (ii) demonstrates the viability of purely Hebbian, associative learning in a multilayered neural network architecture; and (iii) makes clear predictions on the effects of attention on latency and magnitude of event-related potentials to lexical items. Such predictions have been confirmed by recent experimental evidence.