Orthogonal neural codes for speech in the infant brain

Statistical learning beyond words in human neonates

Interest in statistical learning in developmental studies stems from the observation that 8-month-olds were able to extract words from a monotone speech stream solely using the transition probabilities (TP) between syllables (Saffran et al., 1996). A simple mechanism was thus part of the human infant's toolbox for discovering regularities in language. Since this seminal study, observations on statistical learning capabilities have multiplied across domains and species, challenging the hypothesis of a dedicated mechanism for language acquisition. Here, we leverage the two dimensions conveyed by speech -speaker identity and phonemes- to examine (1) whether neonates can compute TPs on one dimension despite irrelevant variation on the other and (2) whether the linguistic dimension enjoys an advantage over the voice dimension. In two experiments, we exposed neonates to artificial speech streams constructed by concatenating syllables while recording EEG. The sequence had a statistical structure based either on the phonetic content, while the voices varied randomly (Experiment 1) or on voices with random phonetic content (Experiment 2). After familiarisation, neonates heard isolated duplets adhering, or not, to the structure they were familiarised with. In both experiments, we observed neural entrainment at the frequency of the regularity and distinct Event-Related Potentials (ERP) to correct and incorrect duplets, highlighting the universality of statistical learning mechanisms and suggesting it operates on virtually any dimension the input is factorised. However, only linguistic duplets elicited a specific ERP component, potentially an N400 precursor, suggesting a lexical stage triggered by phonetic regularities already at birth. These results show that, from birth, multiple input regularities can be processed in parallel and feed different higher-order networks.

Neural representation of sensorimotor features in language-motor areas during auditory and visual perception

Speech processing involves a complex interplay between sensory and motor systems in the brain, essential for early language development. Recent studies have extended this sensory-motor interaction to visual word processing, emphasizing the connection between reading and handwriting during literacy acquisition. Here we show how language-motor areas encode motoric and sensory features of language stimuli during auditory and visual perception, using functional magnetic resonance imaging (fMRI) combined with representational similarity analysis. Chinese-speaking adults completed tasks involving the perception of spoken syllables and written characters, alongside syllable articulation and finger writing tasks to localize speech-motor and writing-motor areas. We found that both language-motor and sensory areas generally encode production-related motoric features across modalities, indicating cooperative interactions between motor and sensory systems. Notably, sensory encoding within sensorimotor areas was observed during auditory speech perception, but not in visual character perception. These findings underscore the dual encoding capacities of language-motor areas, revealing both shared and distinct neural representation patterns across modalities, which may be linked to innate sensory-motor mechanisms and modality-specific processing demands. Our results shed light on the sensorimotor integration mechanisms underlying language perception, highlighting the importance of a cross-modality perspective.

[Applications and prospects of electroencephalography technology in neurorehabilitation assessment and treatment]

With the high incidence of neurological diseases such as stroke and mental illness, rehabilitation treatments for neurological disorders have received widespread attention. Electroencephalography (EEG) technology, despite its excellent temporal resolution, has historically been limited in application due to its insufficient spatial resolution, and is mainly confined to preoperative assessment, intraoperative monitoring, and epilepsy detection. However, traditional constraints of EEG technology are being overcome with the popularization of EEG technology with high-density over 64-lead, the application of innovative analysis techniques and the integration of multimodal techniques, which are significantly broadening its applications in clinical settings. These advancements have not only reinforced the irreplaceable role of EEG technology in neurorehabilitation assessment, but also expanded its therapeutic potential through its combined use with technologies such as transcranial magnetic stimulation, transcranial electrical stimulation and brain-computer interfaces. This article reviewed the applications, advancements, and future prospects of EEG technology in neurorehabilitation assessment and treatment. Advancements in technology and interdisciplinary collaboration are expected to drive new applications and innovations in EEG technology within the neurorehabilitation field, providing patients with more precise and personalized rehabilitation strategies.

On the ability of standard and brain-constrained deep neural networks to support cognitive superposition: a position paper

The ability to coactivate (or "superpose") multiple conceptual representations is a fundamental function that we constantly rely upon; this is crucial in complex cognitive tasks requiring multi-item working memory, such as mental arithmetic, abstract reasoning, and language comprehension. As such, an artificial system aspiring to implement any of these aspects of general intelligence should be able to support this operation. I argue here that standard, feed-forward deep neural networks (DNNs) are unable to implement this function, whereas an alternative, fully brain-constrained class of neural architectures spontaneously exhibits it. On the basis of novel simulations, this proof-of-concept article shows that deep, brain-like networks trained with biologically realistic Hebbian learning mechanisms display the spontaneous emergence of internal circuits (cell assemblies) having features that make them natural candidates for supporting superposition. Building on previous computational modelling results, I also argue that, and offer an explanation as to why, in contrast, modern DNNs trained with gradient descent are generally unable to co-activate their internal representations. While deep brain-constrained neural architectures spontaneously develop the ability to support superposition as a result of (1) neurophysiologically accurate learning and (2) cortically realistic between-area connections, backpropagation-trained DNNs appear to be unsuited to implement this basic cognitive operation, arguably necessary for abstract thinking and general intelligence. The implications of this observation are briefly discussed in the larger context of existing and future artificial intelligence systems and neuro-realistic computational models.

Modality-Specific and Amodal Language Processing by Single Neurons

According to psycholinguistic theories, during language processing, spoken and written words are first encoded along independent phonological and orthographic dimensions, then enter into modality-independent syntactic and semantic codes. Non-invasive brain imaging has isolated several cortical regions putatively associated with those processing stages, but lacks the resolution to identify the corresponding neural codes. Here, we describe the firing responses of over 1000 neurons, and mesoscale field potentials from over 1400 microwires and 1500 iEEG contacts in 21 awake neurosurgical patients with implanted electrodes during written and spoken sentence comprehension. Using forward modeling of temporal receptive fields, we determined which sensory or abstract dimensions are encoded. We observed a double dissociation between superior temporal neurons sensitive to phonemes and phonological features and previously unreported ventral occipito-temporal neurons sensitive to letters and orthographic features. We also discovered novel neurons, primarily located in middle temporal and inferior frontal areas, which are modality-independent and show responsiveness to higher linguistic features. Overall, these findings show how language processing can be linked to neural dynamics, across multiple brain regions at various resolutions and down to the level of single neurons.

Revisiting the functional monitoring of brain development in premature neonates. A new direction in clinical care and research

The first 1000 days of life are of paramount importance for neonatal development. Premature newborns are exposed early to the external environment, modifying the fetal exposome and leading to overexposure in some sensory domains and deprivation in others. The resulting neurodevelopmental effects may persist throughout the individual's lifetime. Several neonatal neuromonitoring techniques can be used to investigate neural mechanisms in early postnatal development. EEG is the most widely used, as it is easy to perform, even at the patient's bedside. It is not expensive and provides information with a high temporal resolution and relatively good spatial resolution when performed in high-density mode. Functional near-infrared spectroscopy (fNIRS), a technique for monitoring vascular network dynamics, can also be used at the patient's bedside. It is not expensive and has a good spatial resolution at the cortical surface. These two techniques can be combined for simultaneous monitoring of the neuronal and vascular networks in premature newborns, providing insight into neurodevelopment before term. However, the extent to which more general conclusions about fetal development can be drawn from findings for premature neonates remains unclear due to considerable differences in environmental and medical situations. Fetal MEG (fMEG, as an alternative to EEG for preterm infants) and fMRI (as an alternative to fNIRS for preterm infants) can also be used to investigate fetal neurodevelopment on a trimester-specific basis. These techniques should be used for validation purposes as they are the only tools available for evaluating neuronal dysfunction in the fetus at the time of the gene-environment interactions influencing transient neuronal progenitor populations in brain structures. But what do these techniques tell us about early neurodevelopment? We address this question here, from two points of view. We first discuss spontaneous neural activity and its electromagnetic and hemodynamic correlates. We then explore the effects of stimulating the immature developing brain with information from exogenous sources, reviewing the available evidence concerning the characteristics of electromagnetic and hemodynamic responses. Once the characteristics of the correlates of neural dynamics have been determined, it will be essential to evaluate their possible modulation in the context of disease and in at-risk populations. Evidence can be collected with various neuroimaging techniques targeting both spontaneous and exogenously driven neural activity. A multimodal approach combining the neuromonitoring of different functional compartments (neuronal and vascular) is required to improve our understanding of the normal functioning and dysfunction of the brain and to identify neurobiomarkers for predicting the neurodevelopmental outcome of premature neonate and fetus. Such an approach would provide a framework for exploring early neurodevelopment, paving the way for the development of tools for earlier diagnosis in these vulnerable populations, thereby facilitating preventive, rescue and reparative neurotherapeutic interventions.

Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states

Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making.

Highlights

Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders.

eTOC

Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.

Infant neuroscience: how to measure brain activity in the youngest minds

The functional properties of the infant brain are poorly understood. Recent advances in cognitive neuroscience are opening new avenues for measuring brain activity in human infants. These include novel uses of existing technologies such as electroencephalography (EEG) and magnetoencephalography (MEG), the availability of newer technologies including functional near-infrared spectroscopy (fNIRS) and optically pumped magnetometry (OPM), and innovative applications of functional magnetic resonance imaging (fMRI) in awake infants during cognitive tasks. In this review article we catalog these available non-invasive methods, discuss the challenges and opportunities encountered when applying them to human infants, and highlight the potential they may ultimately hold for advancing our understanding of the youngest minds.

Twenty-month-olds categorically discriminate similar sounding vowels regardless of vocabulary level, an event related potentials (ERP) study

The current study investigated whether vocabulary relates to phonetic categorization at neural level in early childhood. Electoencephalogram (EEG) responses were collected from 53 Dutch 20-month-old children in a passive oddball paradigm, in which they were presented with two nonwords "giep" [ɣip] and "gip" [ɣɪp] that were contrasted solely by the vowel. In the multiple-speaker condition, both nonwords were produced by twelve different speakers; while, in the single-speaker condition, one single token of each word was used as stimuli. Infant positive mismatch responses (p-MMR) were elicited in both conditions without significant amplitude differences. When the infants were median split based on vocabulary level, the large and small vocabulary groups showed comparable p-MMR amplitudes yet different scalp distribution in both conditions. These results suggest successful phonetic categorization of native similar sounding vowels at 20 months, and a close relationship between speech categorization and vocabulary development.

Abstract processing of syllabic structures in early infancy

Syllables are one of the fundamental building blocks of early language acquisition. From birth onwards, infants preferentially segment, process and represent the speech into syllable-sized units, raising the question of what type of computations infants are able to perform on these perceptual units. Syllables are abstract units structured in a way that allows grouping phonemes into sequences. The goal of this research was to investigate 4-to-5-month-old infants' ability to encode the internal structure of syllables, at a target age when the language system is not yet specialized on the sounds and the phonotactics of native languages. We conducted two experiments in which infants were first familiarized to lists of syllables implementing either CVC (consonant-vowel-consonant) or CCV (consonant-consonant-vowel) structures, then presented with new syllables implementing both structures at test. Experiments differ in the degree of phonological similarity between the materials used at familiarization and test. Results show that infants were able to differentiate syllabic structures at test, even when test syllables were implemented by combinations of phonemes that infants did not hear before. Only infants familiarized with CVC syllables discriminated the structures at test, pointing to a processing advantage for CVC over CCV structures. This research shows that, in addition to preferentially processing the speech into syllable-sized units, during the first months of life, infants are also capable of performing fine-grained computations within such units.

Phonological acquisition depends on the timing of speech sounds: Deconvolution EEG modeling across the first five years

The late development of fast brain activity in infancy restricts initial processing abilities to slow information. Nevertheless, infants acquire the short-lived speech sounds of their native language during their first year of life. Here, we trace the early buildup of the infant phoneme inventory with naturalistic electroencephalogram. We apply the recent method of deconvolution modeling to capture the emergence of the feature-based phoneme representation that is known to govern speech processing in the mature brain. Our cross-sectional analysis uncovers a gradual developmental increase in neural responses to native phonemes. Critically, infants appear to acquire those phoneme features first that extend over longer time intervals-thus meeting infants' slow processing abilities. Shorter-lived phoneme features are added stepwise, with the shortest acquired last. Our study shows that the ontogenetic acceleration of electrophysiology shapes early language acquisition by determining the duration of the acquired units.

Phonemic mismatch negativity mediates the association between phoneme awareness and character reading ability in young Chinese children

Poor phonological awareness is associated with greater risk for reading disability. The underlying neural mechanism of such association may lie in the brain processing of phonological information. Lower amplitude of auditory mismatch negativity (MMN) has been associated with poor phonological awareness and with the presence of reading disability. The current study recorded auditory MMN to phoneme and lexical tone contrast with odd-ball paradigm and examined whether auditory MMN mediated the associations between phonological awareness and character reading ability through a three-year longitudinal study in 78 native Mandarin-speaking kindergarten children. Hierarchical linear regression and mediation analyses showed that the effect of phoneme awareness on the character reading ability was mediated by the phonemic MMN in young Chinese children. Findings underscore the key role of phonemic MMN as the underlying neurodevelopmental mechanism linking phoneme awareness and reading ability.

Sensorimotor foundations of speech perception in infancy

The perceptual system for speech is highly organized from early infancy. This organization bootstraps young human learners' ability to acquire their native speech and language from speech input. Here, we review behavioral and neuroimaging evidence that perceptual systems beyond the auditory modality are also specialized for speech in infancy, and that motor and sensorimotor systems can influence speech perception even in infants too young to produce speech-like vocalizations. These investigations complement existing literature on infant vocal development and on the interplay between speech perception and production systems in adults. We conclude that a multimodal speech and language network is present before speech-like vocalizations emerge.

Cognitive neuroscience: An abstract sense of number in the infant brain

The human infant brain automatically extracts number from the environment. A new study recovers an abstract code for number from the brain electrophysiology of sleeping infants.

Spontaneous supra-modal encoding of number in the infant brain

The core knowledge hypothesis postulates that infants automatically analyze their environment along abstract dimensions, including numbers. According to this view, approximate numbers should be encoded quickly, pre-attentively, and in a supra-modal manner by the infant brain. Here, we directly tested this idea by submitting the neural responses of sleeping 3-month-old infants, measured with high-density electroencephalography (EEG), to decoders designed to disentangle numerical and non-numerical information. The results show the emergence, in approximately 400 ms, of a decodable number representation, independent of physical parameters, that separates auditory sequences of 4 vs. 12 tones and generalizes to visual arrays of 4 vs. 12 objects. Thus, the infant brain contains a number code that transcends sensory modality, sequential or simultaneous presentation, and arousal state.

To be and not to be: wide-field Ca2+ imaging reveals neocortical functional segmentation combines stability and flexibility

The stability and flexibility of the functional parcellation of the cerebral cortex is fundamental to how familiar and novel information is both represented and stored. We leveraged new advances in Ca2+ sensors and microscopy to understand the dynamics of functional segmentation in the dorsal cerebral cortex. We performed wide-field Ca2+ imaging in head-fixed mice and used spatial independent component analysis (ICA) to identify independent spatial sources of Ca2+ fluorescence. The imaging data were evaluated over multiple timescales and discrete behaviors including resting, walking, and grooming. When evaluated over the entire dataset, a set of template independent components (ICs) were identified that were common across behaviors. Template ICs were present across a range of timescales, from days to 30 seconds, although with lower occurrence probability at shorter timescales, highlighting the stability of the functional segmentation. Importantly, unique ICs emerged at the shorter duration timescales that could act to transiently refine the cortical network. When data were evaluated by behavior, both common and behavior-specific ICs emerged. Each behavior is composed of unique combinations of common and behavior-specific ICs. These observations suggest that cerebral cortical functional segmentation exhibits considerable spatial stability over time and behaviors while retaining the flexibility for task-dependent reorganization.

Constraints on infants' ability to extract non-adjacent dependencies from vowels and consonants

Language acquisition requires infants' ability to track dependencies between distant speech elements. Infants as young as 3 months have been shown to successfully identify such non-adjacent dependencies between syllables, and this ability has been related to the maturity of infants' pitch processing. The present study tested whether 8- to 10-month-old infants (N = 68) can also learn dependencies at smaller segmental levels and whether the relation between dependency and pitch processing extends to other auditory features. Infants heard either syllable sequences encoding an item-specific dependency between non-adjacent vowels or between consonants. These frequent standard sequences were interspersed with infrequent intensity deviants and dependency deviants, which violated the non-adjacent relationship. Both vowel and consonant groups showed electrophysiological evidence for detection of the intensity manipulation. However, evidence for dependency learning was only found for infants hearing the dependencies across vowels, not consonants, and only in a subgroup of infants who had an above-average language score in a behavioral test. In a correlation analysis, we found no relation between intensity and dependency processing. We conclude that item-specific, segment-based non-adjacent dependencies are not easily learned by infants and if so, vowels are more accessible to the task, but only to infants who display advanced language skills.

The power of rhythms: how steady-state evoked responses reveal early neurocognitive development

Electroencephalography (EEG) is a non-invasive and painless recording of cerebral activity, particularly well-suited for studying young infants, allowing the inspection of cerebral responses in a constellation of different ways. Of particular interest for developmental cognitive neuroscientists is the use of rhythmic stimulation, and the analysis of steady-state evoked potentials (SS-EPs) - an approach also known as frequency tagging. In this paper we rely on the existing SS-EP early developmental literature to illustrate the important advantages of SS-EPs for studying the developing brain. We argue that (1) the technique is both objective and predictive: the response is expected at the stimulation frequency (and/or higher harmonics), (2) its high spectral specificity makes the computed responses particularly robust to artifacts, and (3) the technique allows for short and efficient recordings, compatible with infants' limited attentional spans. We additionally provide an overview of some recent inspiring use of the SS-EP technique in adult research, in order to argue that (4) the SS-EP approach can be implemented creatively to target a wide range of cognitive and neural processes. For all these reasons, we expect SS-EPs to play an increasing role in the understanding of early cognitive processes. Finally, we provide practical guidelines for implementing and analyzing SS-EP studies.

Sleeping neonates track transitional probabilities in speech but only retain the first syllable of words

Extracting statistical regularities from the environment is a primary learning mechanism that might support language acquisition. While it has been shown that infants are sensitive to transition probabilities between syllables in speech, it is still not known what information they encode. Here we used electrophysiology to study how full-term neonates process an artificial language constructed by randomly concatenating four pseudo-words and what information they retain after a few minutes of exposure. Neural entrainment served as a marker of the regularities the brain was tracking during learning. Then in a post-learning phase, evoked-related potentials (ERP) to different triplets explored which information was retained. After two minutes of familiarization with the artificial language, neural entrainment at the word rate emerged, demonstrating rapid learning of the regularities. ERPs in the test phase significantly differed between triplets starting or not with the correct first syllables, but no difference was associated with subsequent violations in transition probabilities. Thus, our results revealed a two-step learning process: neonates segmented the stream based on its statistical regularities, but memory encoding targeted during the word recognition phase entangled the ordinal position of the syllables but was still incomplete at that age.

A practical guide to applying machine learning to infant EEG data

Electroencephalography (EEG) has been widely adopted by the developmental cognitive neuroscience community, but the application of machine learning (ML) in this domain lags behind adult EEG studies. Applying ML to infant data is particularly challenging due to the low number of trials, low signal-to-noise ratio, high inter-subject variability, and high inter-trial variability. Here, we provide a step-by-step tutorial on how to apply ML to classify cognitive states in infants. We describe the type of brain attributes that are widely used for EEG classification and also introduce a Riemannian geometry based approach for deriving connectivity estimates that account for inter-trial and inter-subject variability. We present pipelines for learning classifiers using trials from a single infant and from multiple infants, and demonstrate the application of these pipelines on a standard infant EEG dataset of forty 12-month-old infants collected under an auditory oddball paradigm. While we classify perceptual states induced by frequent versus rare stimuli, the presented pipelines can be easily adapted for other experimental designs and stimuli using the associated code that we have made publicly available.

Time-resolved multivariate pattern analysis of infant EEG data: A practical tutorial

Time-resolved multivariate pattern analysis (MVPA), a popular technique for analyzing magneto- and electro-encephalography (M/EEG) neuroimaging data, quantifies the extent and time-course by which neural representations support the discrimination of relevant stimuli dimensions. As EEG is widely used for infant neuroimaging, time-resolved MVPA of infant EEG data is a particularly promising tool for infant cognitive neuroscience. MVPA has recently been applied to common infant imaging methods such as EEG and fNIRS. In this tutorial, we provide and describe code to implement time-resolved, within-subject MVPA with infant EEG data. An example implementation of time-resolved MVPA based on linear SVM classification is described, with accompanying code in Matlab and Python. Results from a test dataset indicated that in both infants and adults this method reliably produced above-chance accuracy for classifying stimuli images. Extensions of the classification analysis are presented including both geometric- and accuracy-based representational similarity analysis, implemented in Python. Common choices of implementation are presented and discussed. As the amount of artifact-free EEG data contributed by each participant is lower in studies of infants than in studies of children and adults, we also explore and discuss the impact of varying participant-level inclusion thresholds on resulting MVPA findings in these datasets.