Development of Print-Speech Integration in the Brain of Beginning Readers With Varying Reading Skills

The lateralization of reading

Reports in the 1890s described reading disorders from left hemisphere damage. Subsequent work converging from a variety of research approaches have confirmed a strong dependence of reading on the left ventral occipitotemporal cortex, though there is also evidence for some reading capacity of the right hemisphere. The development of this leftward bias parallels reading acquisition in children and adults and is blunted in developmental dyslexia. Several structural and functional hypotheses have been advanced to explain why reading lateralizes to the left. In the second half of this review we explore the extension of these findings to other forms of reading. Most reading studies used the alphabetic scripts of Europe but there are many writing systems. Comparisons with logographic scripts such as Chinese and kanji have revealed subtle differences. Also, while we often think of reading as the extraction of verbal language from written text, it can be broadened to other types of information extraction from symbols. Reading can occur with visual stimuli that are not written text, as with sign language in the deaf and lip-reading, and with non-visual stimuli that are textual, as with Braille. Musical notation and number reading are other text-based visual forms of reading that do not involve words. Overall, most studies show that the left ventral occipitotemporal cortex is involved in processing these diverse types of reading, with variable contributions from the right hemisphere.

Differential modulations of theta and beta oscillations by audiovisual congruency in letter-speech sound integration

The integration of visual letters and speech sounds is a crucial part of learning to read. Previous studies investigating this integration have revealed a modulation by audiovisual (AV) congruency, commonly known as the congruency effect. To investigate the cortical oscillations of the congruency effects across different oscillatory frequency bands, we conducted a Japanese priming task in which a visual letter was followed by a speech sound. We analyzed the power and phase properties of oscillatory activities in the theta and beta bands between congruent and incongruent letter-speech sound (L-SS) pairs. Our results revealed stronger theta-band (5-7 Hz) power in the congruent condition and cross-modal phase resetting within the auditory cortex, accompanied by enhanced inter-trial phase coherence (ITPC) in the auditory-related areas in response to the congruent condition. The observed congruency effect of theta-band power may reflect increased neural activities in the left auditory region during L-SS integration. Additionally, theta ITPC findings suggest that visual letters amplify neuronal responses to the following corresponding auditory stimulus, which may reflect the differential cross-modal influences in the primary auditory cortex. In contrast, decreased beta-band (20-35 Hz) oscillatory power was observed in the right centroparietal regions for the congruent condition. The reduced beta power seems to be unrelated to the processing of AV integration, but may be interpreted as the brain response to predicting auditory sounds during language processing. Our data provide valuable insights by indicating that oscillations in different frequency bands contribute to the disparate aspects of L-SS integration.

Music training is associated with better audio-visual integration in Chinese language

In the present study, we aimed to investigate whether long-term music training could improve audio-visual speech integration in Chinese, using event-related brain potential (ERP) measurements. Specifically, we recruited musicians and non-musicians to participate in our experiment where visual Chinese characters were presented simultaneously with congruent or incongruent speech sounds. In order to maintain participants' focus on both auditory and visual modalities, they were instructed to perform a probe detection task. Our study revealed that for the musicians, audiovisual incongruent stimuli elicited larger N1 and N400 amplitudes compared to audiovisual congruent stimuli. Conversely, for the non-musicians, only a larger N400 amplitude was observed for incongruent stimuli relative to congruent stimuli, without a significant difference in N1 amplitude. Furthermore, correlation analyses indicated that more years of music training was associated with a larger N1 effect for the musicians. These results suggest that musicians were capable of detecting character-speech sound incongruence at an earlier time window compared to non-musicians. Overall, our findings provide compelling evidence that music training is associated with better integration of visual characters and auditory speech sounds in language processing.

Letter-speech sound integration in typical reading development during the first years of formal education

This study investigated the neural basis of letter and speech sound (LS) integration in 53 typical readers (35 girls, all White) during the first 2 years of reading education (ages 7-9). Changes in both sensory (multisensory vs unisensory) and linguistic (congruent vs incongruent) aspects of LS integration were examined. The left superior temporal cortex and bilateral inferior frontal cortex showed increasing activation for multisensory over unisensory LS over time, driven by reduced activation to speech sounds. No changes were noted in the congruency effect. However, at age nine, heightened activation to incongruent over congruent LS pairs were observed, correlating with individual differences in reading development. This suggests that the incongruency effect evolves at varying rates depending on reading skills.

The Role of Letter-Speech Sound Integration in Native and Second Language Reading: A Study in Native Japanese Readers Learning English

The automatic activation of letter-speech sound (L-SS) associations is a vital step in typical reading acquisition. However, the contribution of L-SS integration during nonalphabetic native and alphabetic second language (L2) reading remains unclear. This study explored whether L-SS integration plays a similar role in a nonalphabetic language as in alphabetic languages and its contribution to L2 reading among native Japanese-speaking adults with varying English proficiency. A priming paradigm in Japanese and English was performed by presenting visual letters or symbols, followed by auditory sounds. We compared behavioral and event-related responses elicited by congruent letter-sound pairs, incongruent pairs, and baseline condition (symbol-sound pairs). The behavioral experiment revealed shorter RTs in the congruent condition for Japanese and English tasks, suggesting a facilitation effect of congruency. The ERP experiment results showed an increased early N1 response to Japanese congruent pairs compared to corresponding incongruent stimuli at the left frontotemporal electrodes. Interestingly, advanced English learners exhibited greater activities in bilateral but predominantly right-lateralized frontotemporal regions for the congruent condition within the N1 time window. Moreover, the enhancement of P2 response to congruent pairs was observed in intermediate English learners. These findings indicate that, despite deviations from native language processing, advanced speakers may successfully integrate letters and sounds during English reading, whereas intermediate learners may encounter difficulty in achieving L-SS integration when reading L2. Furthermore, our results suggest that L2 proficiency may affect the level of automaticity in L-SS integration, with the right P2 congruency effect playing a compensatory role for intermediate learners.

Unraveling individual differences in learning potential: A dynamic framework for the case of reading development

Children show an enormous capacity to learn during development, but with large individual differences in the time course and trajectory of learning and the achieved skill level. Recent progress in developmental sciences has shown the contribution of a multitude of factors including genetic variation, brain plasticity, socio-cultural context and learning experiences to individual development. These factors interact in a complex manner, producing children's idiosyncratic and heterogeneous learning paths. Despite an increasing recognition of these intricate dynamics, current research on the development of culturally acquired skills such as reading still has a typical focus on snapshots of children's performance at discrete points in time. Here we argue that this 'static' approach is often insufficient and limits advancements in the prediction and mechanistic understanding of individual differences in learning capacity. We present a dynamic framework which highlights the importance of capturing short-term trajectories during learning across multiple stages and processes as a proxy for long-term development on the example of reading. This framework will help explain relevant variability in children's learning paths and outcomes and fosters new perspectives and approaches to study how children develop and learn.

Similarities and differences in the neural correlates of letter and speech sound integration in blind and sighted readers

Learning letter and speech sound (LS) associations is a major step in reading acquisition common for all alphabetic scripts, including Braille used by blind readers. The left superior temporal cortex (STC) plays an important role in audiovisual LS integration in sighted people, but it is still unknown what neural mechanisms are responsible for audiotactile LS integration in blind individuals. Here, we investigated the similarities and differences between LS integration in blind Braille (N = 42, age range: 9-60 y.o.) and sighted print (N = 47, age range: 9-60 y.o.) readers who acquired reading using different sensory modalities. In both groups, the STC responded to both isolated letters and isolated speech sounds, showed enhanced activation when they were presented together, and distinguished between congruent and incongruent letter and speech sound pairs. However, the direction of the congruency effect was different between the groups. Sighted subjects showed higher activity for incongruent LS pairs in the bilateral STC, similarly to previously studied typical readers of transparent orthographies. In the blind, congruent pairs resulted in an increased response in the right STC. These differences may be related to more sequential processing of Braille as compared to print reading. At the same time, behavioral efficiency in LS discrimination decisions and the congruency effect were found to be related to age and reading skill only in sighted participants, suggesting potential differences in the developmental trajectories of LS integration between blind and sighted readers.

Neurodevelopmental trajectories of letter and speech sound processing from preschool to the end of elementary school

Learning to read alphabetic languages starts with learning letter-speech-sound associations. How this process changes brain function during development is still largely unknown. We followed 102 children with varying reading skills in a mixed-longitudinal/cross-sectional design from the prereading stage to the end of elementary school over five time points (n = 46 with two and more time points, of which n = 16 fully-longitudinal) to investigate the neural trajectories of letter and speech sound processing using fMRI. Children were presented with letters and speech sounds visually, auditorily, and audiovisually in kindergarten (6.7yo), at the middle (7.3yo) and end of first grade (7.6yo), and in second (8.4yo) and fifth grades (11.5yo). Activation of the ventral occipitotemporal cortex for visual and audiovisual processing followed a complex trajectory, with two peaks in first and fifth grades. The superior temporal gyrus (STG) showed an inverted U-shaped trajectory for audiovisual letter processing, a development that in poor readers was attenuated in middle STG and absent in posterior STG. Finally, the trajectories for letter-speech-sound integration were modulated by reading skills and showed differing directionality in the congruency effect depending on the time point. This unprecedented study captures the development of letter processing across elementary school and its neural trajectories in children with varying reading skills.

Disentangling influences of dyslexia, development, and reading experience on effective brain connectivity in children

Altered brain connectivity between regions of the reading network has been associated with reading difficulties. However, it remains unclear whether connectivity differences between children with dyslexia (DYS) and those with typical reading skills (TR) are specific to reading impairments or to reading experience. In this functional MRI study, 132 children (M = 10.06 y, SD = 1.46) performed a phonological lexical decision task. We aimed to disentangle (1) disorder-specific from (2) experience-related differences in effective connectivity and to (3) characterize the development of DYS and TR. We applied dynamic causal modeling to age-matched (n_dys = 25, n_TR = 35) and reading-level-matched (n_dys = 25, n_TR = 22) groups. Developmental effects were assessed in beginning and advanced readers (TR: n_beg = 48, n_adv = 35, DYS: n_beg = 24, n_adv = 25). We show that altered feedback connectivity between the inferior parietal lobule and the visual word form area (VWFA) during print processing can be specifically attributed to reading impairments, because these alterations were found in DYS compared to both the age-matched and reading-level-matched TR. In contrast, feedforward connectivity from the VWFA to parietal and frontal regions characterized experience in TR and increased with age and reading skill. These directed connectivity findings pinpoint disorder-specific and experience-dependent alterations in the brain's reading network.

Differences in brain functional networks for audiovisual integration during reading between children and adults

Building robust letter-to-sound correspondences is a prerequisite for developing reading capacity. However, the neural mechanisms underlying the development of audiovisual integration for reading are largely unknown. This study used functional magnetic resonance imaging in a lexical decision task to investigate functional brain networks that support audiovisual integration during reading in developing child readers (10-12 years old) and skilled adult readers (20-28 years old). The results revealed enhanced connectivity in a prefrontal-superior temporal network (including the right medial frontal gyrus, right superior frontal gyrus, and left superior temporal gyrus) in adults relative to children, reflecting the development of attentional modulation of audiovisual integration involved in reading processing. Furthermore, the connectivity strength of this brain network was correlated with reading accuracy. Collectively, this study, for the first time, elucidates the differences in brain networks of audiovisual integration for reading between children and adults, promoting the understanding of the neurodevelopment of multisensory integration in high-level human cognition.

Longitudinal changes in brain activation underlying reading fluency

Reading fluency-the speed and accuracy of reading connected text-is foundational to educational success. The current longitudinal study investigates the neural correlates of fluency development using a connected-text paradigm with an individualized presentation rate. Twenty-six children completed a functional MRI task in 1st/2nd grade (time 1) and again 1-2 years later (time 2). There was a longitudinal increase in activation in the ventral occipito-temporal (vOT) cortex from time 1 to time 2. This increase was also associated with improvements in reading fluency skills and modulated by individual speed demands. These findings highlight the reciprocal relationship of the vOT region with reading proficiency and its importance for supporting the developmental transition to fluent reading. These results have implications for developing effective interventions to target increased automaticity in reading.

Visual Occipito-Temporal N1 Sensitivity to Digits Across Elementary School

Number processing abilities are important for academic and personal development. The course of initial specialization of ventral occipito-temporal cortex (vOTC) sensitivity to visual number processing is crucial for the acquisition of numeric and arithmetic skills. We examined the visual N1, the electrophysiological correlate of vOTC activation across five time points in kindergarten (T1, mean age 6.60 years), middle and end of first grade (T2, 7.38 years; T3, 7.68 years), second grade (T4, 8.28 years), and fifth grade (T5, 11.40 years). A combination of cross-sectional and longitudinal EEG data of a total of 62 children (35 female) at varying familial risk for dyslexia were available to form groups of 23, 22, 27, 27, and 42 participants for each of the five time points. The children performed a target detection task which included visual presentation of single digits (DIG), false fonts (FF), and letters (LET) to derive measures for coarse (DIG vs. FF) and fine (DIG vs. LET) digit sensitive processing across development. The N1 amplitude analyses indicated coarse and fine sensitivity characterized by a stronger N1 to digits than false fonts across all five time points, and stronger N1 to digits than letters at all but the second (T2) time point. In addition, lower arithmetic skills were associated with stronger coarse N1 digit sensitivity over the left hemisphere in second grade (T4), possibly reflecting allocation of more attentional resources or stronger reliance on the verbal system in children with poorer arithmetic skills. To summarize, our results show persistent visual N1 sensitivity to digits that is already present early on in pre-school and remains stable until fifth grade. This pattern of digit sensitivity development clearly differs from the relatively sharp rise and fall of the visual N1 sensitivity to words or letters between kindergarten and middle of elementary school and suggests unique developmental trajectories for visual processing of written characters that are relevant to numeracy and literacy.

Atypical Relationships Between Neurofunctional Features of Print-Sound Integration and Reading Abilities in Chinese Children With Dyslexia

Conquering print-sound mappings (e.g., grapheme-phoneme correspondence rules) is vital for developing fluent reading skills. In neuroimaging research, this ability can be indexed by activation differences between audiovisual congruent against incongruent conditions in brain areas such as the left superior temporal cortex. In line with it, individuals with dyslexia have difficulty in tasks requiring print-sound processing, accompanied by a reduced neural integration. However, existing evidence is almost restricted to alphabetic languages. Whether and how multisensory processing of print and sound is impaired in Chinese dyslexia remains underexplored. In this study, we applied a passive audiovisual integration paradigm with functional magnetic resonance imaging to investigate the possible dysfunctions in processing character-sound (opaque; semantics can be automatically accessed) and pinyin-sound associations (transparent; no particular meaning can be confirmed) in Chinese dyslexic children. Unexpectedly, the dyslexic group did not show reduced neural integration compared with typically developing readers in either character or pinyin experiment. However, the results revealed atypical correlations between neural integration and different reading abilities in dyslexia. Specifically, while the neural integration in the left inferior frontal cortex in processing character-sound pairs correlated with silent reading comprehension in both children with and without dyslexia, it was associated with morphological awareness (semantic-related) in controls but with rapid naming (phonological-related) in dyslexics. This result indicates Chinese dyslexic children may not use the same grapho-semantic processing strategy as their typical peers do. As for pinyin-sound processing, while a stronger neural integration in the direction of "congruent > incongruent" in the left occipito-temporal cortex and bilateral superior temporal cortices was associated with better oral reading fluency in the control group, an opposite pattern was found in dyslexia. This finding may reflect dyslexia's dysfunctional recruitment of the regions in grapho-phonological processing, which further impedes character learning.

Neurofunctional basis underlying audiovisual integration of print and speech sound in Chinese children

Effortless print-sound integration is essential to reading development, and the superior temporal cortex (STC) is the most critical brain region. However, to date, the conclusion is almost restricted to alphabetic orthographies. To examine the neural basis in non-alphabetic languages and its relationship with reading abilities, we conducted a functional magnetic resonance imaging study in typically developing Chinese children. Two neuroimaging-based indicators of audiovisual processing-additive enhancement (higher activation in the congruent than the average activation of unimodal conditions) and neural integration (different activations between the congruent versus incongruent conditions)-were used to investigate character-sounds (opaque) and pinyin-sounds (transparent) processing. We found additive enhancement in bilateral STCs processing both character and pinyin stimulations. Moreover, the neural integrations in the left STC for the two scripts were strongly correlated. In terms of differentiation, first, areas beyond the STCs showed additive enhancement in processing pinyin-sounds. Second, while the bilateral STCs, left inferior/middle frontal and parietal regions manifested a striking neural integration (incongruent > congruent) for character-sounds, no significant clusters were revealed for pinyin-sounds. Finally, the neural integration in the left middle frontal gyrus for characters was specifically associated with silent reading comprehension proficiency, indicating automatic semantic processing during implicit character-sound integration. In contrast, the neural integration in the left STC for pinyin was specifically associated with oral reading fluency that relies on grapho-phonological mapping. To summarize, this study revealed both script-universal and script-specific neurofunctional substrates of print-sound integration as well as their processing- and region-dependent associations with reading abilities in typical Chinese children.

How Learning to Read Changes the Listening Brain

Reading acquisition reorganizes existing brain networks for speech and visual processing to form novel audio-visual language representations. This requires substantial cortical plasticity that is reflected in changes in brain activation and functional as well as structural connectivity between brain areas. The extent to which a child's brain can accommodate these changes may underlie the high variability in reading outcome in both typical and dyslexic readers. In this review, we focus on reading-induced functional changes of the dorsal speech network in particular and discuss how its reciprocal interactions with the ventral reading network contributes to reading outcome. We discuss how the dynamic and intertwined development of both reading networks may be best captured by approaching reading from a skill learning perspective, using audio-visual learning paradigms and longitudinal designs to follow neuro-behavioral changes while children's reading skills unfold.

Developmental Trajectories of Letter and Speech Sound Integration During Reading Acquisition

Reading acquisition in alphabetic languages starts with learning the associations between speech sounds and letters. This learning process is related to crucial developmental changes of brain regions that serve visual, auditory, multisensory integration, and higher cognitive processes. Here, we studied the development of audiovisual processing and integration of letter-speech sound pairs with an audiovisual target detection functional MRI paradigm. Using a longitudinal approach, we tested children with varying reading outcomes before the start of reading acquisition (T1, 6.5 yo), in first grade (T2, 7.5 yo), and in second grade (T3, 8.5 yo). Early audiovisual integration effects were characterized by higher activation for incongruent than congruent letter-speech sound pairs in the inferior frontal gyrus and ventral occipitotemporal cortex. Audiovisual processing in the left superior temporal gyrus significantly increased from the prereading (T1) to early reading stages (T2, T3). Region of interest analyses revealed that activation in left superior temporal gyrus (STG), inferior frontal gyrus and ventral occipitotemporal cortex increased in children with typical reading fluency skills, while poor readers did not show the same development in these regions. The incongruency effect bilaterally in parts of the STG and insular cortex at T1 was significantly associated with reading fluency skills at T3. These findings provide new insights into the development of the brain circuitry involved in audiovisual processing of letters, the building blocks of words, and reveal early markers of audiovisual integration that may be predictive of reading outcomes.

ERP Correlates of Altered Orthographic-Phonological Processing in Dyslexia

Automatic visual word recognition requires not only well-established phonological and orthographic representations but also efficient audio-visual integration of these representations. One possibility is that in developmental dyslexia, inefficient orthographic processing might underlie poor reading. Alternatively, reading deficit could be due to inefficient phonological processing or inefficient integration of orthographic and phonological information. In this event-related potential study, participants with dyslexia (N = 25) and control readers (N = 27) were presented with pairs of words and pseudowords in an implicit same-different task. The reference-target pairs could be identical, or different in the identity or the position of the letters. To test the orthographic-phonological processing, target stimuli were presented in visual-only and audiovisual conditions. Participants with and without dyslexia processed the reference stimuli similarly; however, group differences emerged in the processing of target stimuli, especially in the audiovisual condition where control readers showed greater N1 responses for words than for pseudowords, but readers with dyslexia did not show such difference. Moreover, after 300 ms lexicality effect exhibited a more focused frontal topographic distribution in readers with dyslexia. Our results suggest that in developmental dyslexia, phonological processing and audiovisual processing deficits are more pronounced than orthographic processing deficits.

Reading-Related Brain Changes in Audiovisual Processing: Cross-Sectional and Longitudinal MEG Evidence

The ability to establish associations between visual objects and speech sounds is essential for human reading. Understanding the neural adjustments required for acquisition of these arbitrary audiovisual associations can shed light on fundamental reading mechanisms and help reveal how literacy builds on pre-existing brain circuits. To address these questions, the present longitudinal and cross-sectional MEG studies characterize the temporal and spatial neural correlates of audiovisual syllable congruency in children (age range, 4-9 years; 22 males and 20 females) learning to read. Both studies showed that during the first years of reading instruction children gradually set up audiovisual correspondences between letters and speech sounds, which can be detected within the first 400 ms of a bimodal presentation and recruit the superior portions of the left temporal cortex. These findings suggest that children progressively change the way they treat audiovisual syllables as a function of their reading experience. This reading-specific brain plasticity implies (partial) recruitment of pre-existing brain circuits for audiovisual analysis.SIGNIFICANCE STATEMENT Linking visual and auditory linguistic representations is the basis for the development of efficient reading, while dysfunctional audiovisual letter processing predicts future reading disorders. Our developmental MEG project included a longitudinal and a cross-sectional study; both studies showed that children's audiovisual brain circuits progressively change as a function of reading experience. They also revealed an exceptional degree of neuroplasticity in audiovisual neural networks, showing that as children develop literacy, the brain progressively adapts so as to better detect new correspondences between letters and speech sounds.

The rise and fall of rapid occipito-temporal sensitivity to letters: Transient specialization through elementary school

Letters, foundational units of alphabetic writing systems, are quintessential to human culture. The ability to read, indispensable to perform in today’s society, necessitates a reorganization of visual cortex for fast letter recognition, but the developmental course of this process has not yet been characterized. Here, we show the emergence of visual sensitivity to letters across five electroencephalography measurements from kindergarten and throughout elementary school and relate this development to emerging reading skills. We examined the visual N1, the electrophysiological correlate of ventral occipito-temporal cortex activation in 65 children at varying familial risk for dyslexia. N1 letter sensitivity emerged in first grade, when letter sound knowledge gains were most pronounced and decayed shortly after when letter knowledge is consolidated, showing an inverted U-shaped development. This trajectory can be interpreted within an interactive framework that underscores the influence of top-down predictions. While the N1 amplitudes to letters correlated with letter sound knowledge at the beginning of learning, no association between the early N1 letter response and later reading skills was found. In summary, the current findings provide an important reference point for our neuroscientific understanding of learning trajectories and the process of visual specialization during skill learning.

Brain dynamics of (a)typical reading development-a review of longitudinal studies

Literacy development is a process rather than a single event and thus should be studied at multiple time points. A longitudinal design employing neuroimaging methods offers the possibility to identify neural changes associated with reading development, and to reveal early markers of dyslexia. The core of this review is a summary of findings from longitudinal neuroimaging studies on typical and atypical reading development. Studies focused on the prediction of reading gains with a single neuroimaging time point complement this review. Evidence from structural studies suggests that reading development results in increased structural integrity and functional specialization of left-hemispheric language areas. Compromised integrity of some of these tracts in children at risk for dyslexia might be compensated by higher anatomical connectivity in the homologous right hemisphere tracts. Regarding function, activation in phonological and audiovisual integration areas and growing sensitivity to print in the ventral occipito-temporal cortex (vOT) seem to be relevant neurodevelopmental markers of successful reading acquisition. Atypical vOT responses at the beginning of reading training and infant auditory brain potentials have been proposed as neuroimaging predictors of dyslexia that can complement behavioral measures. Besides these insights, longitudinal neuroimaging studies on reading and dyslexia are still relatively scarce and small sample sizes raise legitimate concerns about the reliability of the results. This review discusses the challenges of these studies and provides recommendations to improve this research area. Future longitudinal research with larger sample sizes are needed to improve our knowledge of typical and atypical reading neurodevelopment.