Effects of event knowledge in processing verbal arguments

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward-presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing.

Flexible and fine-grained simulation of speed in language processing

According to the embodied cognition theory, language comprehension is achieved through mental simulation. This account is supported by a number of studies reporting action simulations during language comprehension. However, which details of sensory-motor experience are included in these simulations is still controversial. Here, three experiments were carried out to examine the simulation of speed in action language comprehension. Experiment 1 adopted a lexical decision task and a semantic similarity judgment task on isolated fast and slow action verbs. It has been shown that fast action verbs were processed significantly faster than slow action verbs when deep semantic processing is required. Experiment 2 and Experiment 3 investigated the contextual influence on the simulation of speed, showing that the processing of verbs, either depicting fast actions or neutral actions, would be slowed down when embedded in the slow action sentences. These experiments together demonstrate that the fine-gained information, speed, is an important part of action representation and can be simulated but may not in an automatic way. Moreover, the speed simulation is flexible and can be modulated by the context.

Finding structure during incremental speech comprehension

A core aspect of human speech comprehension is the ability to incrementally integrate consecutive words into a structured and coherent interpretation, aligning with the speaker's intended meaning. This rapid process is subject to multidimensional probabilistic constraints, including both linguistic knowledge and non-linguistic information within specific contexts, and it is their interpretative coherence that drives successful comprehension. To study the neural substrates of this process, we extract word-by-word measures of sentential structure from BERT, a deep language model, which effectively approximates the coherent outcomes of the dynamic interplay among various types of constraints. Using representational similarity analysis, we tested BERT parse depths and relevant corpus-based measures against the spatiotemporally resolved brain activity recorded by electro-/magnetoencephalography when participants were listening to the same sentences. Our results provide a detailed picture of the neurobiological processes involved in the incremental construction of structured interpretations. These findings show when and where coherent interpretations emerge through the evaluation and integration of multifaceted constraints in the brain, which engages bilateral brain regions extending beyond the classical fronto-temporal language system. Furthermore, this study provides empirical evidence supporting the use of artificial neural networks as computational models for revealing the neural dynamics underpinning complex cognitive processes in the brain.

Lexical-Semantic Content, Not Syntactic Structure, Is the Main Contributor to ANN-Brain Similarity of fMRI Responses in the Language Network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI data set of responses to n = 627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we (i) perturbed sentences' word order, (ii) removed different subsets of words, or (iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical-semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust as to whether the mapping model is trained on intact or perturbed stimuli and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

Nuances of knowing: Brain potentials reveal implicit effects of domain knowledge on word processing in the absence of sentence-level knowledge

In previous work investigating the relationship between domain knowledge (of the fictional world of Harry Potter) and sentence comprehension, domain knowledge had a greater impact on electrical brain potentials to words which completed sentences about fictional "facts" participants reported they did not know compared to facts they did. This suggests that individuals use domain knowledge continuously to activate relevant/related concepts as they process sentences, even with only partial knowledge. As that study relied on subjective reports, it may have resulted in response bias related to an individual's overall domain knowledge. In the present study, we therefore asked participants with varying degrees of domain knowledge to complete sentences describing fictional "facts" as an objective measure of sentence-level knowledge. We then recorded EEG as the same individuals (re-)read the same sentences, including their appropriate final words, and sorted these according to their objective knowledge scores. Replicating and extending Troyer et al., domain knowledge immediately facilitated access to meaning for unknown words; greater domain knowledge was associated with reduced N400 amplitudes for unknown words. These findings constitute novel evidence for graded preactivation of conceptual knowledge (e.g., at the level of semantic features and/or relations) in the absence of lexical prediction. Knowledge also influenced post-N400 memory/integration processes for these same unknown words; greater domain knowledge was associated with enhanced late positive components (LPCs), suggesting that deeper encoding during language processing may be engendered when knowledgeable individuals encounter an apparent gap in their knowledge.

Does negation influence the choice of sentence continuations? Evidence from a four-choice cloze task

Event plausibility facilitates the processing of affirmative sentences, but little is known about how it affects negative sentences. In six behavioural experiments, we investigated negation's impact on the choice of sentence continuations that differ with respect to event plausibility. In a four-choice cloze task, participants saw affirmative and negative sentence fragments (The child will [not] eat the . . .) in combination with four potential continuations: yoghurt (a plausible word), shellfish (a weak world knowledge violating word), branch (a severe world knowledge violating word), and minivan (a word resulting in a semantic violation). Across all experiments the plausible word was highly preferred in both affirmative and negative sentences. Experiment 2 replicated Experiment 1 while ruling out the possibility that the lack of effect of negation in Experiment 1 stemmed from participants not fully processing the negation. Experiment 3 showed that the observed plausibility effects can be generalised to other aspectual forms (The child has [not] eaten the yoghurt). Experiment 4 ruled out the possibility that the choices were mainly driven by lexical associations and additionally suggested a role for informativity. Experiment 5 replicated Experiment 4 and reinforced the general pattern according to which negative sentences express the denial of plausible positive events. Experiment 6 provided evidence that informativity might be driving patterns of choices in the negative sentences. All in all, these findings suggest that upcoming continuations are chosen to maximise the plausibility of the event in the affirmative sentences and to deny that event in the negative sentences. The observed plausibility effects do not seem to be modulated by the internal representation of events, but they can be modulated by changes to the expected informativity of the sentence.

Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely

Word co-occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs' semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pretrained LLMs (from 2018's BERT to 2023's MPT) assign a higher likelihood to plausible descriptions of agent-patient interactions than to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n = 1215), we found that pretrained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign a higher likelihood to possible versus impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely versus unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i) LLM scores are driven by both plausibility and surface-level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.

Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI dataset of responses to n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we i) perturbed sentences' word order, ii) removed different subsets of words, or iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical semantic content of the sentence (largely carried by content words) rather than the sentence's syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN's embedding space and decrease the ANN's ability to predict upcoming tokens in those stimuli. Further, results are robust to whether the mapping model is trained on intact or perturbed stimuli, and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result-that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones-aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

Predictive language comprehension in Parkinson's disease

Verb and action knowledge deficits are reported in persons with Parkinson's disease (PD), even in the absence of dementia or mild cognitive impairment. However, the impact of these deficits on combinatorial semantic processing is less well understood. Following on previous verb and action knowledge findings, we tested the hypothesis that PD impairs the ability to integrate event-based thematic fit information during online sentence processing. Specifically, we anticipated persons with PD with age-typical cognitive abilities would perform more poorly than healthy controls during a visual world paradigm task requiring participants to predict a target object constrained by the thematic fit of the agent-verb combination. Twenty-four PD and 24 healthy age-matched participants completed comprehensive neuropsychological assessments. We recorded participants' eye movements as they heard predictive sentences (The fisherman rocks the boat) alongside target, agent-related, verb-related, and unrelated images. We tested effects of group (PD/control) on gaze using growth curve models. There were no significant differences between PD and control participants, suggesting that PD participants successfully and rapidly use combinatory thematic fit information to predict upcoming language. Baseline sentences with no predictive information (e.g., Look at the drum) confirmed that groups showed equivalent sentence processing and eye movement patterns. Additionally, we conducted an exploratory analysis contrasting PD and controls' performance on low-motion-content versus high-motion-content verbs. This analysis revealed fewer predictive fixations in high-motion sentences only for healthy older adults. PD participants may adapt to their disease by relying on spared, non-action-simulation-based language processing mechanisms, although this conclusion is speculative, as the analyses of high- vs. low-motion items was highly limited by the study design. These findings provide novel evidence that individuals with PD match healthy adults in their ability to use verb meaning to predict upcoming nouns despite previous findings of verb semantic impairment in PD across a variety of tasks.

Effects of world knowledge on the prediction of upcoming verbs: an eye-tracking study

Some theories of sentence processing make a distinction between two kinds of meaning: a linguistic meaning encoded at the lexicon (i.e., selectional restrictions), and an extralinguistic knowledge derived from our everyday experiences (i.e., world knowledge). According to such theories, the former meaning is privileged over the latter in terms of the time-course of its access and influence during on-line language comprehension. The present study aims to examine whether world knowledge anomalies (that do not violate selectional restrictions) are rapidly detected during online sentence processing. In an eye-tracking experiment, we used materials in which the likelihood of a specific verb (entrevistar or secuestrar, the Spanish translations for to interview and to kidnap) depended on the agent of the event (periodista or terrorista, the Spanish translations for journalist and terrorist). The results showed an effect of typicality in regression path duration and total reading times at both the verb region and the spillover region, thus providing evidence that world knowledge is rapidly accessed and used during on-line sentence comprehension.

The social hierarchical restrictions of Chinese verbs rapidly guide online thematic role assignment in comprehension

The information about social hierarchical relationship has been intrinsically embedded into semantic restrictions of some Chinese verbs theoretically. For example, the Chinese verb ''(shanyang, support: provide for the needs and comfort of one's elders) only allows its Agent role to have a lower social status compared to the Patient role. However, whether this hierarchical restriction can be rapidly activated and how it impacts online thematic role assignment in reading remains to be seen. To answer this question, a 2 (Verb Type: hierarchical vs non-hierarchical verbs) × 2 (Social Hierarchy Sequence: match vs mismatch) design was constructed in the present study to investigate the interaction between the two factors. ERPs showed that hierarchical restriction violations evoked a stronger anterior negativity to the sentence-final noun (NP2). This effect was absent under two non-hierarchical conditions. To our knowledge, this study is the first to reveal that social hierarchical restrictions of Chinese hierarchical verbs can be rapidly available to guide online thematic role assignment and this process might be closely related to readers' thematic role knowledge.

Are Second Person Masculine Generics Easier to Process for Men than for Women? Evidence from Polish

In Polish, it is obligatory to mark feminine or masculine grammatical gender on second-person singular past tense verbs (e.g., Dostałaś list 'You received-F a letter'). When the addressee's gender is unknown or unspecified, masculine but never feminine gender marking may be used. The present self-paced reading experiment aims to determine whether this practice creates a processing disadvantage for female addressees in such contexts. We further investigated how men process being addressed with feminine-marked verbs, which constitutes a pragmatic violation. To this end, we presented Polish native speakers with short narratives. Each narrative contained either a second-person singular past tense verb with masculine or feminine gender marking, or a gerund verb with no gender marking as a baseline. We hypothesised that both men and women would read the verbs with gender marking mismatching their own gender more slowly than the gender-unmarked gerund verbs. The results revealed that the gender-mismatching verbs were read equally fast as the gerund verbs, and that the verbs with gender marking matching participant gender were read faster. While the relatively high reading time of the gender-unmarked baseline was unexpected, the pattern of results nevertheless shows that verbs with masculine marking were more difficult to process for women compared to men, and vice versa. In conclusion, even though masculine gender marking in the second person is commonly used with a gender-unspecific intention, it created similar processing difficulties for women as the ones that men experienced when addressed through feminine gender marking. This study is the first one, as far as we are aware, to provide evidence for the male bias of second-person masculine generics during language processing.

This Better Be Interesting: A Speaker’s Decision to Speak Cues Listeners to Expect Informative Content

In anticipating upcoming content, comprehenders are known to rely on real-world knowledge. This knowledge can be deployed directly in favor of upcoming content about typical situations (implying a transparent mapping between the world and what speakers say about the world). Such knowledge can also be used to estimate the likelihood of speech, whereby atypical situations are the ones newsworthy enough to merit reporting (i.e., a nontransparent mapping in which improbable situations yield likely utterances). We report four forced-choice studies (three preregistered) testing this distinction between situation knowledge and speech production likelihood. Comprehenders are shown to anticipate situation-atypical meanings more when guessing content (a) that a speaker announces (rather than thinks), (b) that is said out of the blue (rather than produced when prompted), and (c) that is addressed to a large audience (rather than a single listener). The findings contrast with prior work that emphasizes a comprehension bias in favor of typicality, and they highlight the need for comprehension models that incorporate expectations for informativity (as one of a set of inferred speaker goals) alongside expectations for content plausibility.

Mandarin and English Event Cognitive Alignment From Corpus-Based Semantic Fusion Model Perspective

The study explores the fusion of semantic roles and the different semantic fusion types, aiming at establishing a semantic fusion model to explain the cognitive alignment of events in Chinese and English simple sentence constructions containing two verbs. In total, 20,280 simple sentence constructions containing two verbs are collected from Chinese literary works, Peking University Chinese Corpus, and English classic literary works. The semantic fusion in the collected simple sentence constructions containing two verbs is classified into five major semantic fusion categories, which appear with different occurrence frequencies in the two languages. The semantic fusion model of event alignment is comprehensively supported by linguistic research in Chinese and English. From a cognitive linguistic perspective, it is found that the double semantic profiles of the same syntactic element N (noun) make N psychologically activated twice and enable it to enter two processes profiled by the two verbs as a participant. The two processes are combined into one event, which designates a cognitive occurrence of any degree of complexity. N's entry into the two subevents is realized by its double semantic profiles that enable it to fuse two semantic roles into one syntactic element and explain the relationship between N's double syntactic identities and double semantic roles. The semantic fusion model was used to explore event alignment in simple sentence constructions containing two verbs, and it was discovered that the fusion of two semantic roles is universal in languages and is a common psychological and cognitive behavior deeply rooted in the mental conceptualization of language users. The empirical discussion of simple sentence constructions containing two verbs proves that semantic fusion as an important psychological passage in event alignment has solid psychological reality and verifies the applicability of the semantic fusion model in the explanation of event alignment.

Predictable Words Are More Likely to Be Omitted in Fragments-Evidence From Production Data

Instead of a full sentence like Bring me to the university (uttered by the passenger to a taxi driver) speakers often use fragments like To the university to get their message across. So far there is no comprehensive and empirically supported account of why and under which circumstances speakers sometimes prefer a fragment over the corresponding full sentence. We propose an information-theoretic account to model this choice: A speaker chooses the encoding that distributes information most uniformly across the utterance in order to make the most efficient use of the hearer's processing resources (Uniform Information Density, Levy and Jaeger, 2007). Since processing effort is related to the predictability of words (Hale, 2001) our account predicts two effects of word probability on omissions: First, omitting predictable words (which are more easily processed), avoids underutilizing processing resources. Second, inserting words before very unpredictable words distributes otherwise excessively high processing effort more uniformly. We test these predictions with a production study that supports both of these predictions. Our study makes two main contributions: First we develop an empirically motivated and supported account of fragment usage. Second, we extend previous evidence for information-theoretic processing constraints on language in two ways: We find predictability effects on omissions driven by extralinguistic context, whereas previous research mostly focused on effects of local linguistic context. Furthermore, we show that omissions of content words are also subject to information-theoretic well-formedness considerations. Previously, this has been shown mostly for the omission of function words.

Zombie intuitions

In philosophical thought experiments, as in ordinary discourse, our understanding of verbal case descriptions is enriched by automatic comprehension inferences. Such inferences have us routinely infer what else is also true of the cases described. We consider how such routine inferences from polysemous words can generate zombie intuitions: intuitions that are 'killed' (defeated) by contextual information but kept cognitively alive by the psycholinguistic phenomenon of linguistic salience bias. Extending 'evidentiary' experimental philosophy, this paper examines whether the 'zombie argument' against materialism is built on zombie intuitions. We examine the hypothesis that contextually defeated stereotypical inferences from the noun 'zombie' influence intuitions about 'philosophical zombies'. We document framing effects ('zombie' vs 'duplicate') predicted by the hypothesis. Findings undermine intuitions about the conceivability of 'philosophical zombies' and address the philosophical 'hard problem of consciousness'. Findings support a deflationary response: The impression that principled obstacles prevent scientific explanation of how physical processes give rise to conscious experience is generated by philosophical arguments that rely on epistemically deficient intuitions.

Thematic and other semantic relations central to abstract (and concrete) concepts

In this article, we discuss multiple types of meaningful (semantic) relations underlying abstract (as compared to concrete) concepts. We adopt the viewpoint that words act as cues to meaning (Elman in Ment Lexicon 6(1):1-34, 2011; Lupyan and Lewis in Lang Cogn Neurosci 34(10):1319-1337, 2019), which is dependent on the dynamic contents of a comprehender's mental model of the situation. This view foregrounds the importance of both linguistic and real-world context as individuals make sense of words, flexibly access relevant knowledge, and understand described events and situations. We discuss theories of, and experimental work on, abstract concepts through the lens of the importance of thematic and other semantic relations. We then tie these findings to the sentence processing literature in which such meaningful relations within sentential contexts are often experimentally manipulated. In this literature, some specific classes/types of abstract words have been studied, although not comprehensively, and with limited connection to the literature on knowledge underlying abstract concepts reviewed herein. We conclude by arguing that the ways in which humans understand relatively more abstract concepts, in particular, can be informed by the careful study of words presented not in isolation, but rather in situational and linguistic contexts, and as a function of individual differences in knowledge, goals, and beliefs.

Grounding Language Processing: The Added Value of Specifying Linguistic/Compositional Representations and Processes

Abundant empirical evidence suggests that visual perception and motor responses are involved in language comprehension ('grounding'). However, when modeling the grounding of sentence comprehension on a word-by-word basis, linguistic representations and cognitive processes are rarely made fully explicit. This article reviews representational formalisms and associated (computational) models with a view to accommodating incremental and compositional grounding effects. Are different representation formats equally suitable and what mechanisms and representations do models assume to accommodate grounding effects? I argue that we must minimally specify compositional semantic representations, a set of incremental processes/mechanisms, and an explicit link from the assumed processes to measured behavior. Different representational formats can be contrasted in psycholinguistic modeling by holding the set of processes/mechanisms constant; contrasting different processes/mechanisms is possible by holding representations constant. Such psycholinguistic modeling could be applied across a wide range of experimental investigations and complement computational modeling.

Modeling the predictive potential of extralinguistic context with script knowledge: The case of fragments

We describe a novel approach to estimating the predictability of utterances given extralinguistic context in psycholinguistic research. Predictability effects on language production and comprehension are widely attested, but so far predictability has mostly been manipulated through local linguistic context, which is captured with n-gram language models. However, this method does not allow to investigate predictability effects driven by extralinguistic context. Modeling effects of extralinguistic context is particularly relevant to discourse-initial expressions, which can be predictable even if they lack linguistic context at all. We propose to use script knowledge as an approximation to extralinguistic context. Since the application of script knowledge involves the generation of prediction about upcoming events, we expect that scrips can be used to manipulate the likelihood of linguistic expressions referring to these events. Previous research has shown that script-based discourse expectations modulate the likelihood of linguistic expressions, but script knowledge has often been operationalized with stimuli which were based on researchers’ intuitions and/or expensive production and norming studies. We propose to quantify the likelihood of an utterance based on the probability of the event to which it refers. This probability is calculated with event language models trained on a script knowledge corpus and modulated with probabilistic event chains extracted from the corpus. We use the DeScript corpus of script knowledge to obtain empirically founded estimates of the likelihood of an event to occur in context without having to resort to expensive pre-tests of the stimuli. We exemplify our method at a case study on the usage of nonsentential expressions (fragments), which shows that utterances that are predictable given script-based extralinguistic context are more likely to be reduced.

What's new? A comprehension bias in favor of informativity

Language is used as a channel by which speakers convey, among other things, newsworthy and informative messages, i.e., content that is otherwise unpredictable to the comprehender. We therefore might expect comprehenders to show a preference for such messages. However, comprehension studies tend to emphasize the opposite: i.e., processing ease for situation-predictable content (e.g., chopping carrots with a knife). Comprehenders are known to deploy knowledge about situation plausibility during processing in fine-grained context-sensitive ways. Using self-paced reading, we test whether comprehenders can also deploy this knowledge in favor of newsworthy content to yield informativity-driven effects alongside, or instead of, plausibility-driven effects. We manipulate semantic context (unusual protagonists), syntactic construction (wh- clefts), and the communicative environment (text messages). Reading times (primarily sentence-finally) show facilitation for sentences containing newsworthy content (e.g., chopping carrots with a shovel), where the content is both unpredictable at the situation level because of its atypicality and also unpredictable at the word level because of the large number of atypical elements a speaker could potentially mention. Our studies are the first to show that informativity-driven effects are observable at all, and the results highlight the need for models that distinguish between comprehenders' estimate of content plausibility and their estimate of a speaker's decision to talk about that content.

Decoding the Real-Time Neurobiological Properties of Incremental Semantic Interpretation

Communication through spoken language is a central human capacity, involving a wide range of complex computations that incrementally interpret each word into meaningful sentences. However, surprisingly little is known about the spatiotemporal properties of the complex neurobiological systems that support these dynamic predictive and integrative computations. Here, we focus on prediction, a core incremental processing operation guiding the interpretation of each upcoming word with respect to its preceding context. To investigate the neurobiological basis of how semantic constraints change and evolve as each word in a sentence accumulates over time, in a spoken sentence comprehension study, we analyzed the multivariate patterns of neural activity recorded by source-localized electro/magnetoencephalography (EMEG), using computational models capturing semantic constraints derived from the prior context on each upcoming word. Our results provide insights into predictive operations subserved by different regions within a bi-hemispheric system, which over time generate, refine, and evaluate constraints on each word as it is heard.

Implicit Gender Bias in Linguistic Descriptions for Expected Events: The Cases of the 2016 United States and 2017 United Kingdom Elections

Gender stereotypes influence subjective beliefs about the world, and this is reflected in our use of language. But do gender biases in language transparently reflect subjective beliefs? Or is the process of translating thought to language itself biased? During the 2016 United States (N = 24,863) and 2017 United Kingdom (N = 2,609) electoral campaigns, we compared participants' beliefs about the gender of the next head of government with their use and interpretation of pronouns referring to the next head of government. In the United States, even when the female candidate was expected to win, she pronouns were rarely produced and induced substantial comprehension disruption. In the United Kingdom, where the incumbent female candidate was heavily favored, she pronouns were preferred in production but yielded no comprehension advantage. These and other findings suggest that the language system itself is a source of implicit biases above and beyond previously known biases, such as those measured by the Implicit Association Test.

Thematic roles: Core knowledge or linguistic construct?

The status of thematic roles such as Agent and Patient in cognitive science is highly controversial: To some they are universal components of core knowledge, to others they are scholarly fictions without psychological reality. We address this debate by posing two critical questions: to what extent do humans represent events in terms of abstract role categories, and to what extent are these categories shaped by universal cognitive biases? We review a range of literature that contributes answers to these questions: psycholinguistic and event cognition experiments with adults, children, and infants; typological studies grounded in cross-linguistic data; and studies of emerging sign languages. We pose these questions for a variety of roles and find that the answers depend on the role. For Agents and Patients, there is strong evidence for abstract role categories and a universal bias to distinguish the two roles. For Goals and Recipients, we find clear evidence for abstraction but mixed evidence as to whether there is a bias to encode Goals and Recipients as part of one or two distinct categories. Finally, we discuss the Instrumental role and do not find clear evidence for either abstraction or universal biases to structure instrumental categories.

The use of context in resolving syntactic ambiguity: Structural and semantic influences

Verb bias facilitates parsing of temporarily ambiguous sentences, but it is unclear when and how comprehenders use probabilistic knowledge about the combinatorial properties of verbs in context. In a self-paced reading experiment, participants read direct object/sentential complement sentences. Reading time in the critical region was investigated as a function of three forms of bias: structural bias (the frequency with which a verb appears in direct object/sentential complement sentences), lexical bias (the simple co-occurrence of verbs and other lexical items), and global bias (obtained from norming data about the use of verbs with specific noun phrases). For reading times at the critical word, structural bias was the only reliable predictor. However, global bias was superior to structural and lexical bias at the post-critical word and for offline acceptability ratings. The results suggest that structural information about verbs is available immediately, but that context-specific, semantic information becomes increasingly informative as processing proceeds.

On the predictability of event boundaries in discourse: An ERP investigation

When reading a text describing an everyday activity, comprehenders build a model of the situation described that includes prior knowledge of the entities, locations, and sequences of actions that typically occur within the event. Previous work has demonstrated that such knowledge guides the processing of incoming information by making event boundaries more or less expected. In the present ERP study, we investigated whether comprehenders' expectations about event boundaries are influenced by how elaborately common events are described in the context. Participants read short stories in which a common activity (e.g., washing the dishes) was described either in brief or in an elaborate manner. The final sentence contained a target word referring to a more predictable action marking a fine event boundary (e.g., drying) or a less predictable action, marking a coarse event boundary (e.g., jogging). The results revealed a larger N400 effect for coarse event boundaries compared to fine event boundaries, but no interaction with description length. Between 600 and 1000 ms, however, elaborate contexts elicited a larger frontal positivity compared to brief contexts. This effect was largely driven by less predictable targets, marking coarse event boundaries. We interpret the P600 effect as indexing the updating of the situation model at event boundaries, consistent with Event Segmentation Theory (EST). The updating process is more demanding with coarse event boundaries, which presumably require the construction of a new situation model.

Complement Coercion: The Joint Effects of Type and Typicality

Complement coercion (begin a book →reading) involves a type clash between an event-selecting verb and an entity-denoting object, triggering a covert event (reading). Two main factors involved in complement coercion have been investigated: the semantic type of the object (event vs. entity), and the typicality of the covert event (the author began a book →writing). In previous research, reading times have been measured at the object. However, the influence of the typicality of the subject–object combination on processing an aspectual verb such as begin has not been studied. Using a self-paced reading study, we manipulated semantic type and subject–object typicality, exploiting German word order to measure reading times at the aspectual verb. These variables interacted at the target verb. We conclude that both type and typicality probabilistically guide expectations about upcoming input. These results are compatible with an expectation-based view of complement coercion and language comprehension more generally in which there is rapid interaction between what is typically viewed as linguistic knowledge, and what is typically viewed as domain general knowledge about how the world works.

Looking for a Location: Dissociated Effects of Event-Related Plausibility and Verb-Argument Information on Predictive Processing in Aphasia

PURPOSE

This study examined the influence of verb-argument information and event-related plausibility on prediction of upcoming event locations in people with aphasia, as well as older and younger, neurotypical adults. It investigated how these types of information interact during anticipatory processing and how the ability to take advantage of the different types of information is affected by aphasia.

METHOD

This study used a modified visual-world task to examine eye movements and offline photo selection. Twelve adults with aphasia (aged 54-82 years) as well as 44 young adults (aged 18-31 years) and 18 older adults (aged 50-71 years) participated.

RESULTS

Neurotypical adults used verb argument status and plausibility information to guide both eye gaze (a measure of anticipatory processing) and image selection (a measure of ultimate interpretation). Argument status did not affect the behavior of people with aphasia in either measure. There was only limited evidence of interaction between these 2 factors in eye gaze data.

CONCLUSIONS

Both event-related plausibility and verb-based argument status contributed to anticipatory processing of upcoming event locations among younger and older neurotypical adults. However, event-related likelihood had a much larger role in the performance of people with aphasia than did verb-based knowledge regarding argument structure.

The amount and structure of prior event experience affects anticipatory sentence interpretation

Listeners easily interpret speech about novel events in everyday conversation; however, much of research on mechanisms of spoken language comprehension, by design, capitalises on event knowledge that is familiar to most listeners. This paper explores how listeners generalise from previous experience during incremental processing of novel spoken sentences. In two studies, participants initially heard stories that conveyed novel event mappings between agents, actions and objects, and their ability to interpret a novel, related event in real-time was measured via eye-tracking. A single exposure to a novel event was not sufficient to support generalisation in real-time sentence processing. When each story event was repeated with either the same agent or a different, related agent, listeners generalised in the repetition condition, but not in the multiple agent condition. These findings shed light on the conditions under which listeners leverage prior event experience while interpreting novel linguistic signals in everyday speech.

Salience and Attention in Surprisal-Based Accounts of Language Processing

The notion of salience has been singled out as the explanatory factor for a diverse range of linguistic phenomena. In particular, perceptual salience (e.g., visual salience of objects in the world, acoustic prominence of linguistic sounds) and semantic-pragmatic salience (e.g., prominence of recently mentioned or topical referents) have been shown to influence language comprehension and production. A different line of research has sought to account for behavioral correlates of cognitive load during comprehension as well as for certain patterns in language usage using information-theoretic notions, such as surprisal. Surprisal and salience both affect language processing at different levels, but the relationship between the two has not been adequately elucidated, and the question of whether salience can be reduced to surprisal / predictability is still open. Our review identifies two main challenges in addressing this question: terminological inconsistency and lack of integration between high and low levels of representations in salience-based accounts and surprisal-based accounts. We capitalize upon work in visual cognition in order to orient ourselves in surveying the different facets of the notion of salience in linguistics and their relation with models of surprisal. We find that work on salience highlights aspects of linguistic communication that models of surprisal tend to overlook, namely the role of attention and relevance to current goals, and we argue that the Predictive Coding framework provides a unified view which can account for the role played by attention and predictability at different levels of processing and which can clarify the interplay between low and high levels of processes and between predictability-driven expectation and attention-driven focus.

Hemispheric asymmetry in event knowledge activation during incremental language comprehension: A visual half-field ERP study

During incremental language comprehension, the brain activates knowledge of described events, including knowledge elements that constitute semantic anomalies in their linguistic context. The present study investigates hemispheric asymmetries in this process, with the aim of advancing our understanding of the neural basis and functional properties of event knowledge activation during incremental comprehension. In a visual half-field event-related brain potential (ERP) experiment, participants read brief discourses in which the third sentence contained a word that was either highly expected, semantically anomalous but related to the described event (Event-Related), or semantically anomalous but unrelated to the described event (Event-Unrelated). For both visual fields of target word presentation, semantically anomalous words elicited N400 ERP components of greater amplitude than did expected words. Crucially, Event-Related anomalous words elicited a reduced N400 relative to Event-Unrelated anomalous words only with left visual field/right hemisphere presentation. This result suggests that right hemisphere processes are critical to the activation of event knowledge elements that violate the linguistic context, and in doing so informs existing theories of hemispheric asymmetries in semantic processing during language comprehension. Additionally, this finding coincides with past research suggesting a crucial role for the right hemisphere in elaborative inference generation, raises interesting questions regarding hemispheric coordination in generating event-specific linguistic expectancies, and more generally highlights the possibility of functional dissociation of event knowledge activation for the generation of elaborative inferences and for linguistic expectancies.

Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events

Since the early 2000s, several ERP studies have challenged the assumption that we always use syntactic contextual information to influence semantic processing of incoming words, as reflected by the N400 component. One approach for explaining these findings is to posit distinct semantic and syntactic processing mechanisms, each with distinct time courses. While this approach can explain specific datasets, it cannot account for the wider body of findings. I propose an alternative explanation: a dynamic generative framework in which our goal is to infer the underlying event that best explains the set of inputs encountered at any given time. Within this framework, combinations of semantic and syntactic cues with varying reliabilities are used as evidence to weight probabilistic hypotheses about this event. I further argue that the computational principles of this framework can be extended to understand how we infer situation models during discourse comprehension, and intended messages during spoken communication.

World knowledge affects prediction as quickly as selectional restrictions: Evidence from the visual world paradigm

There has been considerable debate regarding the question of whether linguistic knowledge and world knowledge are separable and used differently during processing or not (Hagoort, Hald, Bastiaansen, & Petersson, 2004; Matsuki et al., 2011; Paczynski & Kuperberg, 2012; Warren & McConnell, 2007; Warren, McConnell, & Rayner, 2008). Previous investigations into this question have provided mixed evidence as to whether violations of selectional restrictions are detected earlier than violations of world knowledge. We report a visual-world eye-tracking study comparing the timing of facilitation contributed by selectional restrictions versus world knowledge. College-aged adults (n=36) viewed photographs of natural scenes while listening to sentences. Participants anticipated upcoming direct objects similarly regardless of whether facilitation was provided by only world knowledge or a combination of selectional restrictions and world knowledge. These results suggest that selectional restrictions are not available earlier in comprehension than world knowledge.

The Truth Before and After: Brain Potentials Reveal Automatic Activation of Event Knowledge during Sentence Comprehension

How does knowledge of real-world events shape our understanding of incoming language? Do temporal terms like “before” and “after” impact the online recruitment of real-world event knowledge? These questions were addressed in two ERP experiments, wherein participants read sentences that started with “before” or “after” and contained a critical word that rendered each sentence true or false (e.g., “Before/After the global economic crisis, securing a mortgage was easy/harder”). The critical words were matched on predictability, rated truth value, and semantic relatedness to the words in the sentence. Regardless of whether participants explicitly verified the sentences or not, false-after-sentences elicited larger N400s than true-after-sentences, consistent with the well-established finding that semantic retrieval of concepts is facilitated when they are consistent with real-world knowledge. However, although the truth judgments did not differ between before- and after-sentences, no such sentence N400 truth value effect occurred in before-sentences, whereas false-before-sentences elicited an enhanced subsequent positive ERPs. The temporal term “before” itself elicited more negative ERPs at central electrode channels than “after.” These patterns of results show that, irrespective of ultimate sentence truth value judgments, semantic retrieval of concepts is momentarily facilitated when they are consistent with the known event outcome compared to when they are not. However, this inappropriate facilitation incurs later processing costs as reflected in the subsequent positive ERP deflections. The results suggest that automatic activation of event knowledge can impede the incremental semantic processes required to establish sentence truth value.

Comprehending the impossible: what role do selectional restriction violations play?

To elucidate how different kinds of knowledge are used during comprehension, readers' eye movements were monitored as they read sentences that were: plausible, impossible because of a selectional restriction violation, or impossible because of a violation of general world knowledge. Eye movements on the pre-critical, critical, and post-critical words evidenced disruption in the selectional restriction violation condition compared to the other two conditions. These findings suggest that disruption associated with reading about impossible events is not directly determined by how impossible the event seems. Rather, the relationship between the verb and arguments in the sentence seems to matter. These findings are the strongest evidence to date that processing effects associated with selectional restrictions can dissociate from those associated with general world knowledge about events.

Grammatical context constrains lexical competition in spoken word recognition

When perceiving spoken language, listeners must match the incoming acoustic phonetic input to lexical representations in memory. Models that quantify this process propose that the input activates multiple lexical representations in parallel and that these activated representations compete for recognition (Weber & Scharenborg, 2012). In two experiments, we assessed how grammatically constraining contexts alter the process of lexical competition. The results suggest that grammatical context constrains the lexical candidates that are activated to grammatically appropriate competitors. Stimulus words with little competition from items of the same grammatical class benefit more from the addition of grammatical context than do words with more within-class competition. The results provide evidence that top-down contextual information is integrated in the early stages of word recognition. We propose adding a grammatical class level of analysis to existing models of word recognition to account for these findings.

Form-to-expectation matching effects on first-pass eye movement measures during reading

Recent Electroencephalography/Magnetoencephalography (EEG/MEG) studies suggest that when contextual information is highly predictive of some property of a linguistic signal, expectations generated from context can be translated into surprisingly low-level estimates of the physical form-based properties likely to occur in subsequent portions of the unfolding signal. Whether form-based expectations are generated and assessed during natural reading, however, remains unclear. We monitored eye movements while participants read phonologically typical and atypical nouns in noun-predictive contexts (Experiment 1), demonstrating that when a noun is strongly expected, fixation durations on first-pass eye movement measures, including first fixation duration, gaze duration, and go-past times, are shorter for nouns with category typical form-based features. In Experiments 2 and 3, typical and atypical nouns were placed in sentential contexts normed to create expectations of variable strength for a noun. Context and typicality interacted significantly at gaze duration. These results suggest that during reading, form-based expectations that are translated from higher-level category-based expectancies can facilitate the processing of a word in context, and that their effect on lexical processing is graded based on the strength of category expectancy.

Using instruments to understand argument structure: Evidence for gradient representation

The arguments of a verb are commonly assumed to correspond to the event participants specified by the verb. That is, drink has two arguments because drink specifies two participants: someone who drinks and something that gets drunk. This correspondence does not appear to hold, however, in the case of instrumental participants, e.g. John drank the soda with a straw. Verbs such as slice and write have been argued to specify an instrumental participant, even though instruments do not pattern like arguments given other criteria. In this paper, we investigated how instrumental verbs are represented, testing the hypothesis that verbs such as slice encode three participants in the same way that dative verbs such as lend encode three participants. In two experiments English-speakers reported their judgments about the number of participants specified by a verb, e.g., that drink specifies two participants. These judgments indicate that slice does not encode three distinct arguments. Nonetheless, some verbs were systematically more likely to elicit the judgment that the instrument is specified by the verb, a pattern that held across individual subjects. To account for these findings, we propose that instruments are not independent verbal arguments but are represented in a gradient away: an instrument may be a more or less salient part of the force exerted by an agent. These results inform our understanding of the relationship between argument structure and event representation, raising questions concerning the role of arguments in language processing and learning.

Neural bases of event knowledge and syntax integration in comprehension of complex sentences

Comprehension of complex sentences is necessarily supported by both syntactic and semantic knowledge, but what linguistic factors trigger a readers' reliance on a specific system? This functional neuroimaging study orthogonally manipulated argument plausibility and verb event type to investigate cortical bases of the semantic effect on argument comprehension during reading. The data suggest that telic verbs facilitate online processing by means of consolidating the event schemas in episodic memory and by easing the computation of syntactico-thematic hierarchies in the left inferior frontal gyrus. The results demonstrate that syntax-semantics integration relies on trade-offs among a distributed network of regions for maximum comprehension efficiency.

The influence of event-related knowledge on verb-argument processing in aphasia

Event-related conceptual knowledge outside the language system rapidly affects verb-argument processing in unimpaired adults (McRae and Matsuki, 2009). Some have argued that verb-argument processing is in fact reducible to the activation of such event-related knowledge. However, data favoring this conclusion have come primarily from college-aged healthy adults, for whom both linguistic and conceptual semantic processing is fast and automatic. This study examined the influence of event-related knowledge on verb-argument processing among adults with aphasia (n = 8) and older unimpaired controls (n = 60), in two self-paced reading studies. Participants read sentences containing a plausible verb-argument combination (Mary used a knife to chop the large carrots before dinner), a combination that violated event-related world knowledge (Mary used some bleach to clean the large carrots before dinner), or a combination that violated the verb's selectional restrictions (Mary used a pump to inflate the large carrots before dinner). The participants with aphasia naturally split into two groups: Group 1 (n = 4) had conceptual-semantic impairments (evidenced by poor performance on tasks like Pyramids & Palm Trees) but reasonably intact language processing (higher Western Aphasia Battery Aphasia Quotients), while Group 2 (n = 4) had intact conceptual semantics but poorer language processing. Older unimpaired controls and aphasic Group 1 showed rapid on-line disruption for sentences with selectional-restriction violations (SRVs) and event-related knowledge violations, and also showed SRV-specific penalties in sentence-final acceptability judgments (Experiment 1) and comprehension questions (Experiment 2). In contrast, Group 2 showed very few reliable differences across conditions in either on-line or off-line measures. This difference between aphasic groups suggests that verb-related information and event-related knowledge may be dissociated in aphasia. Furthermore, it suggests that intact language processing is more critical for successful verb-argument integration than intact access to event-related world knowledge. This pattern is unexpected if verb-argument processing is reducible to activation of event-related conceptual knowledge.

When events change their nature: the neurocognitive mechanisms underlying aspectual coercion

The verb "pounce" describes a single, near-instantaneous event. Yet, we easily understand that, "For several minutes the cat pounced…" describes a situation in which multiple pounces occurred, although this interpretation is not overtly specified by the sentence's syntactic structure or by any of its individual words--a phenomenon known as "aspectual coercion." Previous psycholinguistic studies have reported processing costs in association with aspectual coercion, but the neurocognitive mechanisms giving rise to these costs remain contentious. Additionally, there is some controversy about whether readers commit to a full interpretation of the event when the aspectual information becomes available, or whether they leave it temporarily underspecified until later in the sentence. Using ERPs, we addressed these questions in a design that fully crossed context type (punctive, durative, frequentative) with verb type (punctive, durative). We found a late, sustained negativity to punctive verbs in durative contexts, but not in frequentative (e.g., explicitly iterative) contexts. This effect was distinct from the N400 in both its time course and scalp distribution, suggesting that it reflected a different underlying neurocognitive mechanism. We also found that ERPs to durative verbs were unaffected by context type. Together, our results provide strong evidence that neural activity associated with aspectual coercion is driven by the engagement of a morphosyntactically unrealized semantic operator rather than by violations of real-world knowledge, more general shifts in event representation, or event iterativity itself. More generally, our results add to a growing body of evidence that a set of late-onset sustained negativities reflect elaborative semantic processing that goes beyond simply combining the meaning of individual words with syntactic structure to arrive at a final representation of meaning.

Delayed Stimulus-Specific Improvements in Discourse Following Anomia Treatment Using an Intentional Gesture

Purpose: In this study, the authors assessed how the addition of intentional left-hand gestures to an intensive treatment for anomia affects 2 types of discourse: picture description and responses to open-ended questions.Method: Fourteen people with aphasia completed treatment for anomia comprising 30 treatment sessions over 3 weeks.Seven subjects also incorporated intentional left-hand gestures into each treatment trial.Results: Both groups demonstrated significant changes in trained items and improved naming of untrained items but no change in Western Aphasia Battery—Aphasia Quotient(WAB–AQ; Kertesz, 1982) scores. Changes in discourse were limited to the 3-month follow-up assessment. Several discourse measures showed significant improvements in the picture description task and declines during question responses. Additionally, the gesture group produced more words at each assessment, whereas the no gesture group produced fewer words at each assessment. These patterns led to improvements in picture descriptions and minimal declines in question responses in the gesture group. In contrast, the no gesture group showed minimal improvements in picture descriptions and production declines in question responses relative to pretreatment levels.Conclusion: The intensive treatment protocol is a successful method for improving picture naming even of untrained items.Further, the authors conclude that the intentional left-hand gesture contributed significantly to the generalization of treatment to discourse.

Action relevance in linguistic context drives word-induced motor activity

Many neurocognitive studies on the role of motor structures in action-language processing have implicitly adopted a “dictionary-like” framework within which lexical meaning is constructed on the basis of an invariant set of semantic features. The debate has thus been centered on the question of whether motor activation is an integral part of the lexical semantics (embodied theories) or the result of a post-lexical construction of a situation model (disembodied theories). However, research in psycholinguistics show that lexical semantic processing and context-dependent meaning construction are narrowly integrated. An understanding of the role of motor structures in action-language processing might thus be better achieved by focusing on the linguistic contexts under which such structures are recruited. Here, we therefore analyzed online modulations of grip force while subjects listened to target words embedded in different linguistic contexts. When the target word was a hand action verb and when the sentence focused on that action (John signs the contract) an early increase of grip force was observed. No comparable increase was detected when the same word occurred in a context that shifted the focus toward the agent's mental state (John wants to sign the contract). There mere presence of an action word is thus not sufficient to trigger motor activation. Moreover, when the linguistic context set up a strong expectation for a hand action, a grip force increase was observed even when the tested word was a pseudo-verb. The presence of a known action word is thus not required to trigger motor activation. Importantly, however, the same linguistic contexts that sufficed to trigger motor activation with pseudo-verbs failed to trigger motor activation when the target words were verbs with no motor action reference. Context is thus not by itself sufficient to supersede an “incompatible” word meaning. We argue that motor structure activation is part of a dynamic process that integrates the lexical meaning potential of a term and the context in the online construction of a situation model, which is a crucial process for fluent and efficient online language comprehension.

Multiple Influences of Semantic Memory on Sentence Processing: Distinct Effects of Semantic Relatedness on Violations of Real-World Event/State Knowledge and Animacy Selection Restrictions

We aimed to determine whether semantic relatedness between an incoming word and its preceding context can override expectations based on two types of stored knowledge: real-world knowledge about the specific events and states conveyed by a verb, and the verb's broader selection restrictions on the animacy of its argument. We recorded event-related potentials on post-verbal Agent arguments as participants read and made plausibility judgments about passive English sentences. The N400 evoked by incoming animate Agent arguments that violated expectations based on real-world event/state knowledge, was strongly attenuated when they were semantically related to the context. In contrast, semantic relatedness did not modulate the N400 evoked by inanimate Agent arguments that violated the preceding verb's animacy selection restrictions. These findings suggest that, under these task and experimental conditions, semantic relatedness can facilitate processing of post-verbal animate arguments that violate specific expectations based on real-world event/state knowledge, but only when the semantic features of these arguments match the coarser-grained animacy restrictions of the verb. Animacy selection restriction violations also evoked a P600 effect, which was not modulated by semantic relatedness, suggesting that it was triggered by propositional impossibility. Together, these data indicate that the brain distinguishes between real-world event/state knowledge and animacy-based selection restrictions during online processing.