Evidence for the Influence of Syntax on Prosodic Parsing

What Silent Pauses Can 'Tell' Us About the Storytelling Skills of Autistic Children: Relations Between Pausing, Language Skills and Executive Functions

Silent pauses may serve communicative purposes such as demarcating boundaries between discourse units in language production. Previous research has shown that autistic children differ in their pausing behavior from typically-developing (TD) peers, however, the factors behind this difference remain underexplored. The current study was aimed at comparing the use of silent pauses in the narrative production of autistic children and age-matched TD children, and also to identify possible relations between pausing behavior and the children's language and executive function abilities. According to the study's findings, the autistic children did not differ from their TD peers in the use of grammatical pauses, however, the former tended to produce significantly less syntactically complex narratives than the TD group, which increased the likelihood that the autistic group would pause appropriately at phrasal boundaries. Though we have found low rates of ungrammatical silent pauses and omitted pauses in obligatory discourse contexts across both groups, autistic children with lower cognitive flexibility tended to use more ungrammatical pauses than their peers with higher cognitive flexibility scores. Also, the autistic group tended to omit obligatory silent pauses more often as their narration became more complex. The results demonstrate that syntactic complexity in narrative production modulated autistic children's pausing behavior, and that structurally simple narrations boosted the autistic group's appropriate use of grammatical pauses. The overall findings also demonstrate the importance of studying silent pauses in the narrative discourse of autistic children, and also highlight the links between silent pauses and the children's syntactic and cognitive skills.

Intonation Units in Spontaneous Speech Evoke a Neural Response

Spontaneous speech is produced in chunks called intonation units (IUs). IUs are defined by a set of prosodic cues and presumably occur in all human languages. Recent work has shown that across different grammatical and sociocultural conditions IUs form rhythms of ∼1 unit per second. Linguistic theory suggests that IUs pace the flow of information in the discourse. As a result, IUs provide a promising and hitherto unexplored theoretical framework for studying the neural mechanisms of communication. In this article, we identify a neural response unique to the boundary defined by the IU. We measured the EEG of human participants (of either sex), who listened to different speakers recounting an emotional life event. We analyzed the speech stimuli linguistically and modeled the EEG response at word offset using a GLM approach. We find that the EEG response to IU-final words differs from the response to IU-nonfinal words even when equating acoustic boundary strength. Finally, we relate our findings to the body of research on rhythmic brain mechanisms in speech processing. We study the unique contribution of IUs and acoustic boundary strength in predicting delta-band EEG. This analysis suggests that IU-related neural activity, which is tightly linked to the classic Closure Positive Shift (CPS), could be a time-locked component that captures the previously characterized delta-band neural speech tracking.SIGNIFICANCE STATEMENT Linguistic communication is central to human experience, and its neural underpinnings are a topic of much research in recent years. Neuroscientific research has benefited from studying human behavior in naturalistic settings, an endeavor that requires explicit models of complex behavior. Usage-based linguistic theory suggests that spoken language is prosodically structured in intonation units. We reveal that the neural system is attuned to intonation units by explicitly modeling their impact on the EEG response beyond mere acoustics. To our understanding, this is the first time this is demonstrated in spontaneous speech under naturalistic conditions and under a theoretical framework that connects the prosodic chunking of speech, on the one hand, with the flow of information during communication, on the other.

Periodic fluctuations in reading times reflect multi-word-chunking

Memory is fleeting. To avoid information loss, humans need to recode verbal stimuli into chunks of limited duration, each containing multiple words. Chunk duration may also be limited neurally by the wavelength of periodic brain activity, so-called neural oscillations. While both cognitive and neural constraints predict some degree of behavioral regularity in processing, this remains to be shown. Our analysis of self-paced reading data from 181 participants reveals periodic patterns at a frequency of [Formula: see text] 2 Hz. We defined multi-word chunks by using a computational formalization based on dependency annotations and part-of-speech tags. Potential chunk outputs were first generated from the computational formalization and the final chunk outputs were selected based on normalized pointwise mutual information. We show that behavioral periodicity is time-aligned to multi-word chunks, suggesting that the multi-word chunks generated from local dependency clusters may minimize memory demands. This is the first evidence that sentence processing behavior is periodic, consistent with a role of both memory constraints and endogenous electrophysiological rhythms in the formation of chunks during language comprehension.

Event-related responses reflect chunk boundaries in natural speech

Chunking language has been proposed to be vital for comprehension enabling the extraction of meaning from a continuous stream of speech. However, neurocognitive mechanisms of chunking are poorly understood. The present study investigated neural correlates of chunk boundaries intuitively identified by listeners in natural speech drawn from linguistic corpora using magneto- and electroencephalography (MEEG). In a behavioral experiment, subjects marked chunk boundaries in the excerpts intuitively, which revealed highly consistent chunk boundary markings across the subjects. We next recorded brain activity to investigate whether chunk boundaries with high and medium agreement rates elicit distinct evoked responses compared to non-boundaries. Pauses placed at chunk boundaries elicited a closure positive shift with the sources over bilateral auditory cortices. In contrast, pauses placed within a chunk were perceived as interruptions and elicited a biphasic emitted potential with sources located in the bilateral primary and non-primary auditory areas with right-hemispheric dominance, and in the right inferior frontal cortex. Furthermore, pauses placed at stronger boundaries elicited earlier and more prominent activation over the left hemisphere suggesting that brain responses to chunk boundaries of natural speech can be modulated by the relative strength of different linguistic cues, such as syntactic structure and prosody.

Context Matters for Tone and Intonation Processing in Mandarin

In tonal languages such as Mandarin, both lexical tone and sentence intonation are primarily signaled by F0. Their F0 encodings are sometimes in conflict and sometimes in congruency. The present study investigated how tone and intonation, with F0 encodings in conflict or in congruency, are processed and how semantic context may affect their processing. To this end, tone and intonation identification experiments were conducted in both semantically neutral and constraining contexts. Results showed that the overall performance of tone identification was better than that of intonation. Specifically, tone identification was seldom affected by intonation information irrespective of semantic contexts. However, intonation identification, particularly question intonation, was susceptible to the final lexical tone identity and affected by the semantic context. In the semantically neutral context, questions ending with a rising tone and a falling tone were equally difficult to identify. In the semantically constraining context, questions ending with a falling tone were much better identified than those ending with a rising tone. This perceptual asymmetry suggests that top-down information provided by the semantically constraining context can play a facilitating role for listeners to disentangle intonational information from tonal information, but mainly in sentences with the lexical falling tone in the final position.

Linguistic syncopation: Meter-syntax alignment affects sentence comprehension and sensorimotor synchronization

The hierarchical organization of speech rhythm into meter putatively confers cognitive affordances for perception, memory, and motor coordination. Meter also aligns with phrasal structure in systematic ways. In this paper, we show that this alignment affects the robustness of syntactic comprehension and discuss possible underlying mechanisms. In two experiments, we manipulated meter-syntax alignment while sentences with relative clause structures were either read as text (experiment 1, n = 40) or listened to as speech (experiment 2, n = 40). In experiment 2, we also measured the stability with which participants could tap in time with the metrical accents in the sentences they were comprehending. In addition to making more mistakes, sensorimotor synchronization was disrupted when syntactic cues clashed with the metrical context. We suggest that this reflects a tight coordination of top-down linguistic knowledge with the sensorimotor system to optimize comprehension.

Endogenous Oscillations Time-Constrain Linguistic Segmentation: Cycling the Garden Path

Speech is transient. To comprehend entire sentences, segments consisting of multiple words need to be memorized for at least a while. However, it has been noted previously that we struggle to memorize segments longer than approximately 2.7 s. We hypothesized that electrophysiological processing cycles within the delta band (<4 Hz) underlie this time constraint. Participants’ EEG was recorded while they listened to temporarily ambiguous sentences. By manipulating the speech rate, we aimed at biasing participants’ interpretation: At a slow rate, segmentation after 2.7 s would trigger a correct interpretation. In contrast, at a fast rate, segmentation after 2.7 s would trigger a wrong interpretation and thus an error later in the sentence. In line with the suggested time constraint, the phase of the delta-band oscillation at the critical point in the sentence mirrored segmentation on the level of single trials, as indicated by the amplitude of the P600 event-related brain potential (ERP) later in the sentence. The correlation between upstream delta-band phase and downstream P600 amplitude implies that segmentation took place when an underlying neural oscillator had reached a specific angle within its cycle, determining comprehension. We conclude that delta-band oscillations set an endogenous time constraint on segmentation.

Cortical encoding of acoustic and linguistic rhythms in spoken narratives

Speech contains rich acoustic and linguistic information. Using highly controlled speech materials, previous studies have demonstrated that cortical activity is synchronous to the rhythms of perceived linguistic units, for example, words and phrases, on top of basic acoustic features, for example, the speech envelope. When listening to natural speech, it remains unclear, however, how cortical activity jointly encodes acoustic and linguistic information. Here we investigate the neural encoding of words using electroencephalography and observe neural activity synchronous to multi-syllabic words when participants naturally listen to narratives. An amplitude modulation (AM) cue for word rhythm enhances the word-level response, but the effect is only observed during passive listening. Furthermore, words and the AM cue are encoded by spatially separable neural responses that are differentially modulated by attention. These results suggest that bottom-up acoustic cues and top-down linguistic knowledge separately contribute to cortical encoding of linguistic units in spoken narratives.

Sequences of Intonation Units form a ~ 1 Hz rhythm

Studies of speech processing investigate the relationship between temporal structure in speech stimuli and neural activity. Despite clear evidence that the brain tracks speech at low frequencies (~ 1 Hz), it is not well understood what linguistic information gives rise to this rhythm. In this study, we harness linguistic theory to draw attention to Intonation Units (IUs), a fundamental prosodic unit of human language, and characterize their temporal structure as captured in the speech envelope, an acoustic representation relevant to the neural processing of speech. IUs are defined by a specific pattern of syllable delivery, together with resets in pitch and articulatory force. Linguistic studies of spontaneous speech indicate that this prosodic segmentation paces new information in language use across diverse languages. Therefore, IUs provide a universal structural cue for the cognitive dynamics of speech production and comprehension. We study the relation between IUs and periodicities in the speech envelope, applying methods from investigations of neural synchronization. Our sample includes recordings from every-day speech contexts of over 100 speakers and six languages. We find that sequences of IUs form a consistent low-frequency rhythm and constitute a significant periodic cue within the speech envelope. Our findings allow to predict that IUs are utilized by the neural system when tracking speech. The methods we introduce here facilitate testing this prediction in the future (i.e., with physiological data).

Prosody indexes both competence and performance

Prosody is an important feature of language that conveys a wide range of information. However, prosody is widely considered to be a difficult domain of study within the language sciences. One consequence of this is that existing grammatical theories of prosody fail to explain prosodic choices that seem to arise from nonlinguistic cognitive demands, such as communicative context, top-down expectations, and recent articulatory and acoustic experience. We provide an account of some of these phenomena and argue that linguistic theories that do not incorporate these factors into models of prosody are likely to mischaracterize its role in language. This article is categorized under: Linguistics > Language in Mind and Brain Psychology > Language Linguistics > Linguistic Theory.

Constraints on learning disjunctive, unidimensional auditory and phonetic categories

Phonetic categories must be learned, but the processes that allow that learning to unfold are still under debate. The current study investigates constraints on the structure of categories that can be learned and whether these constraints are speech-specific. Category structure constraints are a key difference between theories of category learning, which can roughly be divided into instance-based learning (i.e., exemplar only) and abstractionist learning (i.e., at least partly rule-based or prototype-based) theories. Abstractionist theories can relatively easily accommodate constraints on the structure of categories that can be learned, whereas instance-based theories cannot easily include such constraints. The current study included three groups to investigate these possible constraints as well as their speech specificity: English speakers learning German speech categories, German speakers learning German speech categories, and English speakers learning musical instrument categories, with each group including participants who learned different sets of categories. Both speech groups had greater difficulty learning disjunctive categories (ones that require an "or" statement) than nondisjunctive categories, which suggests that instance-based learning alone is insufficient to explain the learning of the participants learning phonetic categories. This fact was true for both novices (English speakers) and experts (German speakers), which implies that expertise with the materials used cannot explain the patterns observed. However, the same was not true for the musical instrument categories, suggesting a degree of domain-specificity in these constraints that cannot be explained through recourse to expertise alone.

Domain-Specific Expectations in Music Segmentation

The acoustic cues that guide the assignment of phrase boundaries in music (pauses and pitch movements) overlap with those that are known for speech prosody. Based on this, researchers have focused on highlighting the similarities and neural resources shared between music and speech prosody segmentation. The possibility that music-specific expectations add to acoustic cues in driving the segmentation of music into phrases could weaken this bottom-up view, but it remains underexplored. We tested for domain-specific expectations in music segmentation by comparing the segmentation of the same set of ambiguous stimuli under two different instructions: stimuli were either presented as speech prosody or as music. We measured how segmentation differed, in each instruction group, from a common reference (natural speech); thus, focusing on how instruction affected delexicalization effects (natural speech vs. transformed versions with no phonetic content) on segmentation. We saw interactions between delexicalization and instruction on most segmentation indices, suggesting that there is a music mode, different from a speech prosody mode in segmentation. Our findings highlight the importance of top-down influences in segmentation, and they contribute to rethinking the analogy between music and speech prosody.

White matter pathways for prosodic structure building: A case study

The relevance of left dorsal and ventral fiber pathways for syntactic and semantic comprehension is well established, while pathways for prosody are little explored. The present study examined linguistic prosodic structure building in a patient whose right arcuate/superior longitudinal fascicles and posterior corpus callosum were transiently compromised by a vasogenic peritumoral edema. Compared to ten matched healthy controls, the patient's ability to detect irregular prosodic structure significantly improved between pre- and post-surgical assessment. This recovery was accompanied by an increase in average fractional anisotropy (FA) in right dorsal and posterior transcallosal fiber tracts. Neither general cognitive abilities nor (non-prosodic) syntactic comprehension nor FA in right ventral and left dorsal fiber tracts showed a similar pre-post increase. Together, these findings suggest a contribution of right dorsal and inter-hemispheric pathways to prosody perception, including the right-dorsal tracking and structuring of prosodic pitch contours that is transcallosally informed by concurrent syntactic information.

Perturbation of left posterior prefrontal cortex modulates top-down processing in sentence comprehension

Communication is an inferential process. In particular, language comprehension constantly requires top-down efforts, as often multiple interpretations are compatible with a given sentence. To assess top-down processing in the language domain, our experiment employed ambiguous sentences that allow for multiple interpretations (e.g., The client sued the murderer with the corrupt lawyer., where the corrupt lawyer could either belong to The client or the murderer). Interpretation thus depended on whether participants chunk the words of the sentence into short or long syntactic phrases. In principle, bottom-up acoustic information (i.e., the presence or absence of an intonational phrase boundary at the offset of the murderer) indicates one of the two possible interpretations. Yet, acoustic information often indicates interpretations that require words to be chunked into overly long phrases that would overburden working memory. Processing is biased against these demands, reflected in a top-down preference to chunk words into short rather than long phrases. It is often proposed, but also hotly debated, that the ability to chunk words into short phrases is subserved by the left inferior frontal gyrus (IFG). Here, we employed focal repetitive transcranial magnetic stimulation to perturb the left IFG, which resulted in a further decrease of the aptitude to tolerate long phrases, indicating the inability of the left IFG to assist the chunking of words into phrases. In contrast, the processing of auditory information was not affected. Our findings support a causal top-down role of the left inferior frontal gyrus in the chunking of words into phrases.

Linguistic Bias Modulates Interpretation of Speech via Neural Delta-Band Oscillations

Language comprehension requires that single words be grouped into syntactic phrases, as words in sentences are too many to memorize individually. In speech, acoustic and syntactic grouping patterns mostly align. However, when ambiguous sentences allow for alternative grouping patterns, comprehenders may form phrases that contradict speech prosody. While delta-band oscillations are known to track prosody, we hypothesized that linguistic grouping bias can modulate the interpretational impact of speech prosody in ambiguous situations, which should surface in delta-band oscillations when grouping patterns chosen by comprehenders differ from those indicated by prosody. In our auditory electroencephalography study, the interpretation of ambiguous sentences depended on whether an identical word was either followed by a prosodic boundary or not, thereby signaling the ending or continuation of the current phrase. Delta-band oscillatory phase at the critical word should reflect whether participants terminate a phrase despite a lack of acoustic boundary cues. Crossing speech prosody with participants' grouping choice, we observed a main effect of grouping choice-independent of prosody. An internal linguistic bias for grouping words into phrases can thus modulate the interpretational impact of speech prosody via delta-band oscillatory phase.

Is infant-directed speech interesting because it is surprising? - Linking properties of IDS to statistical learning and attention at the prosodic level

The exaggerated intonation and special rhythmic properties of infant-directed speech (IDS) have been hypothesized to attract infants' attention to the speech stream. However, there has been little work actually connecting the properties of IDS to models of attentional processing or perceptual learning. A number of such attention models suggest that surprising or novel perceptual inputs attract attention, where novelty can be operationalized as the statistical (un)predictability of the stimulus in the given context. Since prosodic patterns such as F0 contours are accessible to young infants who are also known to be adept statistical learners, the present paper investigates a hypothesis that F0 contours in IDS are less predictable than those in adult-directed speech (ADS), given previous exposure to both speaking styles, thereby potentially tapping into basic attentional mechanisms of the listeners in a similar manner that relative probabilities of other linguistic patterns are known to modulate attentional processing in infants and adults. Computational modeling analyses with naturalistic IDS and ADS speech from matched speakers and contexts show that IDS intonation has lower overall temporal predictability even when the F0 contours of both speaking styles are normalized to have equal means and variances. A closer analysis reveals that there is a tendency of IDS intonation to be less predictable at the end of short utterances, whereas ADS exhibits more stable average predictability patterns across the full extent of the utterances. The difference between IDS and ADS persists even when the proportion of IDS and ADS exposure is varied substantially, simulating different relative amounts of IDS heard in different family and cultural environments. Exposure to IDS is also found to be more efficient for predicting ADS intonation contours in new utterances than exposure to the equal amount of ADS speech. This indicates that the more variable prosodic contours of IDS also generalize to ADS, and may therefore enhance prosodic learning in infancy. Overall, the study suggests that one reason behind infant preference for IDS could be its higher information value at the prosodic level, as measured by the amount of surprisal in the F0 contours. This provides the first formal link between the properties of IDS and the models of attentional processing and statistical learning in the brain. However, this finding does not rule out the possibility that other differences between the IDS and ADS also play a role.

Making predictable unpredictable with style - Behavioral and electrophysiological evidence for the critical role of prosodic expectations in the perception of prominence in speech

Perceptual prominence of linguistic units such as words has been earlier connected to the concepts of predictability and attentional orientation. One hypothesis is that low-probability prosodic or lexical content is perceived as prominent due to the surprisal and high information value associated with the stimulus. However, the existing behavioral studies have used stimulus manipulations that follow or violate typical linguistic patterns present in the listeners' native language, i.e., assuming that the listeners have already established a model for acceptable prosodic patterns in the language. In the present study, we investigated whether prosodic expectations and the resulting subjective impression of prominence is affected by brief statistical adaptation to suprasegmental acoustic features in speech, also in the case where the prosodic patterns do not necessarily follow language-typical marking for prominence. We first exposed listeners to five minutes of speech with uneven distributions of falling and rising fundamental frequency (F0) trajectories on sentence-final words, and then tested their judgments of prominence on a set of new utterances. The results show that the probability of the F0 trajectory affects the perception of prominence, a less frequent F0 trajectory making a word more prominent independently of the absolute direction of F0 change. In the second part of the study, we conducted EEG-measurements on a set of new subjects listening to similar utterances with predominantly rising or falling F0 on sentence-final words. Analysis of the resulting event-related potentials (ERP) reveals a significant difference in N200 and N400 ERP-component amplitudes between standard and deviant prosody, again independently of the F0 direction and the underlying lexical content. Since N400 has earlier been associated with semantic processing of stimuli, this suggests that listeners implicitly track probabilities at the suprasegmental level and that predictability of a prosodic pattern during a word has an impact to the semantic processing of the word. Overall, the study suggests that prosodic markers for prominence are at least partially driven by the statistical structure of recently perceived speech, and therefore prominence perception could be based on statistical learning mechanisms similar to those observed in early word learning, but in this case operating at the level of suprasegmental acoustic features.

The neural oscillations of speech processing and language comprehension: state of the art and emerging mechanisms

Neural oscillations subserve a broad range of functions in speech processing and language comprehension. On the one hand, speech contains-somewhat-repetitive trains of air pressure bursts that occur at three dominant amplitude modulation frequencies, physically marking the linguistically meaningful progressions of phonemes, syllables and intonational phrase boundaries. To these acoustic events, neural oscillations of isomorphous operating frequencies are thought to synchronise, presumably resulting in an implicit temporal alignment of periods of neural excitability to linguistically meaningful spectral information on the three low-level linguistic description levels. On the other hand, speech is a carrier signal that codes for high-level linguistic meaning, such as syntactic structure and semantic information-which cannot be read from stimulus acoustics, but must be acquired during language acquisition and decoded for language comprehension. Neural oscillations subserve the processing of both syntactic structure and semantic information. Here, I synthesise a mapping from each linguistic processing domain to a unique set of subserving oscillatory mechanisms-the mapping is plausible given the role ascribed to different oscillatory mechanisms in different subfunctions of cortical information processing and faithful to the underlying electrophysiology. In sum, the present article provides an accessible and extensive review of the functional mechanisms that neural oscillations subserve in speech processing and language comprehension.

Stress in Context: Morpho-Syntactic Properties Affect Lexical Stress Assignment in Reading Aloud

Recent findings from English and Russian have shown that grammatical category plays a key role in stress assignment. In these languages, some grammatical categories have a typical stress pattern and this information is used by readers. However, whether readers are sensitive to smaller distributional differences and other morpho-syntactic properties (e.g., gender, number, person) remains unclear. We addressed this issue in word and non-word reading in Italian, a language in which: (1) nouns and verbs differ in the proportion of words with a dominant stress pattern; (2) information specified by words sharing morpho-syntactic properties may contrast with other sources of information, such as stress neighborhood. Both aspects were addressed in two experiments in which context words were used to induce the desired morpho-syntactic properties. Experiment 1 showed that the relatively different proportions of stress patterns between grammatical categories do not affect stress processing in word reading. In contrast, Experiment 2 showed that information specified by words sharing morpho-syntactic properties outweighs stress neighborhood in non-word reading. Thus, while general information specified by grammatical categories may not be used by Italian readers, stress neighbors with morpho-syntactic properties congruent with those of the target stimulus have a primary role in stress assignment. These results underscore the importance of expanding investigations of stress assignment beyond single words, as current models of single-word reading seem unable to account for our results.