Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle

Language models based on artificial neural networks increasingly capture key aspects of how humans process sentences. Most notably, model-based surprisals predict event-related potentials such as N400 amplitudes during parsing. Assuming that these models represent realistic estimates of human linguistic experience, their success in modeling language processing raises the possibility that the human processing system relies on no other principles than the general architecture of language models and on sufficient linguistic input. Here, we test this hypothesis on N400 effects observed during the processing of verb-final sentences in German, Basque, and Hindi. By stacking Bayesian generalised additive models, we show that, in each language, N400 amplitudes and topographies in the region of the verb are best predicted when model-based surprisals are complemented by an Agent Preference principle that transiently interprets initial role-ambiguous noun phrases as agents, leading to reanalysis when this interpretation fails. Our findings demonstrate the need for this principle independently of usage frequencies and structural differences between languages. The principle has an unequal force, however. Compared to surprisal, its effect is weakest in German, stronger in Hindi, and still stronger in Basque. This gradient is correlated with the extent to which grammars allow unmarked NPs to be patients, a structural feature that boosts reanalysis effects. We conclude that language models gain more neurobiological plausibility by incorporating an Agent Preference. Conversely, theories of human processing profit from incorporating surprisal estimates in addition to principles like the Agent Preference, which arguably have distinct evolutionary roots.

An Agent-First Preference in a Patient-First Language During Sentence Comprehension

The language comprehension system preferentially assumes that agents come first during incremental processing. While this might reflect a biologically fixed bias, shared with other domains and other species, the evidence is limited to languages that place agents first, and so the bias could also be learned from usage frequency. Here, we probe the bias with electroencephalography (EEG) in Äiwoo, a language that by default places patients first, but where sentence-initial nouns are still locally ambiguous between patient or agent roles. Comprehenders transiently interpreted nonhuman nouns as patients, eliciting a negativity when disambiguation was toward the less common agent-initial order. By contrast and against frequencies, human nouns were transiently interpreted as agents, eliciting an N400-like negativity when the disambiguation was toward patient-initial order. Consistent with the notion of a fixed property, the agent bias is robust against usage frequency for human referents. However, this bias can be reversed by frequency experience for nonhuman referents.

The Agent Preference in Visual Event Apprehension

A central aspect of human experience and communication is understanding events in terms of agent ("doer") and patient ("undergoer" of action) roles. These event roles are rooted in general cognition and prominently encoded in language, with agents appearing as more salient and preferred over patients. An unresolved question is whether this preference for agents already operates during apprehension, that is, the earliest stage of event processing, and if so, whether the effect persists across different animacy configurations and task demands. Here we contrast event apprehension in two tasks and two languages that encode agents differently; Basque, a language that explicitly case-marks agents ('ergative'), and Spanish, which does not mark agents. In two brief exposure experiments, native Basque and Spanish speakers saw pictures for only 300 ms, and subsequently described them or answered probe questions about them. We compared eye fixations and behavioral correlates of event role extraction with Bayesian regression. Agents received more attention and were recognized better across languages and tasks. At the same time, language and task demands affected the attention to agents. Our findings show that a general preference for agents exists in event apprehension, but it can be modulated by task and language demands.

Cross-linguistic differences in case marking shape neural power dynamics and gaze behavior during sentence planning

Languages differ in how they mark the dependencies between verbs and arguments, e.g., by case. An eye tracking and EEG picture description study examined the influence of case marking on the time course of sentence planning in Basque and Swiss German. While German assigns an unmarked (nominative) case to subjects, Basque specifically marks agent arguments through ergative case. Fixations to agents and event-related synchronization (ERS) in the theta and alpha frequency bands, as well as desynchronization (ERD) in the alpha and beta bands revealed multiple effects of case marking on the time course of early sentence planning. Speakers decided on case marking under planning early when preparing sentences with ergative-marked agents in Basque, whereas sentences with unmarked agents allowed delaying structural commitment across languages. These findings support hierarchically incremental accounts of sentence planning and highlight how cross-linguistic differences shape the neural dynamics underpinning language use.

Two methods for reducing moving metal artifacts in cone-beam CT

PURPOSE

In image-guided radiation therapy, fiducial markers or clips are often used to determine the position of the tumor. These markers lead to streak artifacts in cone-beam CT (CBCT) scans. Standard inpainting-based metal artifact reduction (MAR) methods fail to remove these artifacts in cases of large motion. We propose two methods to effectively reduce artifacts caused by moving metal inserts.

METHODS

The first method (MMAR) utilizes a coarse metal segmentation in the image domain and a refined segmentation in the rawdata domain. After an initial reconstruction, metal is segmented and forward projected giving a coarse metal mask in the rawdata domain. Inside the coarse mask, metal is segmented by utilizing a 2D Sobel filter. Metal is removed by linear interpolation in the refined metal mask. The second method (MoCoMAR) utilizes a motion compensation (MoCo) algorithm [Med Phys. 2013;40:101913] that provides us with a motion-free volume (3D) or with a time series of motion-free volumes (4D). We then apply the normalized metal artifact reduction (NMAR) [Med Phys. 2010;37:5482-5493] to these MoCo volumes. Both methods were applied to three CBCT data sets of patients with metal inserts in the thorax or abdomen region and a 4D thorax simulation. The results were compared to volumes corrected by a standard MAR1 [Radiology. 1987;164:576-577].

RESULTS

MMAR and MoCoMAR were able to remove all artifacts caused by moving metal inserts for the patients and the simulation. Both new methods outperformed the standard MAR1, which was only able to remove artifacts caused by metal inserts with little or no motion.

CONCLUSIONS

In this work, two new methods to remove artifacts caused by moving metal inserts are introduced. Both methods showed good results for a simulation and three patients. While the first method (MMAR) works without any prior knowledge, the second method (MoCoMAR) requires a respiratory signal for the MoCo step and is computationally more demanding and gives no benefit over MMAR, unless MoCo images are desired.

Motion vector field phase-to-amplitude resampling for 4D motion-compensated cone-beam CT

We propose a phase-to-amplitude resampling (PTAR) method to reduce motion blurring in motion-compensated (MoCo) 4D cone-beam CT (CBCT) image reconstruction, without increasing the computational complexity of the motion vector field (MVF) estimation approach. PTAR is able to improve the image quality in reconstructed 4D volumes, including both regular and irregular respiration patterns. The PTAR approach starts with a robust phase-gating procedure for the initial MVF estimation and then switches to a phase-adapted amplitude gating method. The switch implies an MVF-resampling, which makes them amplitude-specific. PTAR ensures that the MVFs, which have been estimated on phase-gated reconstructions, are still valid for all amplitude-gated reconstructions. To validate the method, we use an artificially deformed clinical CT scan with a realistic breathing pattern and several patient data sets acquired with a TrueBeam^TM integrated imaging system (Varian Medical Systems, Palo Alto, CA, USA). Motion blurring, which still occurs around the area of the diaphragm or at small vessels above the diaphragm in artifact-specific cyclic motion compensation (acMoCo) images based on phase-gating, is significantly reduced by PTAR. Also, small lung structures appear sharper in the images. This is demonstrated both for simulated and real patient data. A quantification of the sharpness of the diaphragm confirms these findings. PTAR improves the image quality of 4D MoCo reconstructions compared to conventional phase-gated MoCo images, in particular for irregular breathing patterns. Thus, PTAR increases the robustness of MoCo reconstructions for CBCT. Because PTAR does not require any additional steps for the MVF estimation, it is computationally efficient. Our method is not restricted to CBCT but could rather be applied to other image modalities.

Word Order and Voice Influence the Timing of Verb Planning in German Sentence Production

Theories of incremental sentence production make different assumptions about when speakers encode information about described events and when verbs are selected, accordingly. An eye tracking experiment on German testing the predictions from linear and hierarchical incrementality about the timing of event encoding and verb planning is reported. In the experiment, participants described depictions of two-participant events with sentences that differed in voice and word order. Verb-medial active sentences and actives and passives with sentence-final verbs were compared. Linear incrementality predicts that sentences with verbs placed early differ from verb-final sentences because verbs are assumed to only be planned shortly before they are articulated. By contrast, hierarchical incrementality assumes that speakers start planning with relational encoding of the event. A weak version of hierarchical incrementality assumes that only the action is encoded at the outset of formulation and selection of lexical verbs only occurs shortly before they are articulated, leading to the prediction of different fixation patterns for verb-medial and verb-final sentences. A strong version of hierarchical incrementality predicts no differences between verb-medial and verb-final sentences because it assumes that verbs are always lexically selected early in the formulation process. Based on growth curve analyses of fixations to agent and patient characters in the described pictures, and the influence of character humanness and the lack of an influence of the visual salience of characters on speakers' choice of active or passive voice, the current results suggest that while verb planning does not necessarily occur early during formulation, speakers of German always create an event representation early.

The Timing of Utterance Planning in Task-Oriented Dialogue: Evidence from a Novel List-Completion Paradigm

In conversation, interlocutors rarely leave long gaps between turns, suggesting that next speakers begin to plan their turns while listening to the previous speaker. The present experiment used analyses of speech onset latencies and eye-movements in a task-oriented dialogue paradigm to investigate when speakers start planning their responses. German speakers heard a confederate describe sets of objects in utterances that either ended in a noun [e.g., Ich habe eine Tür und ein Fahrrad ("I have a door and a bicycle")] or a verb form [e.g., Ich habe eine Tür und ein Fahrrad besorgt ("I have gotten a door and a bicycle")], while the presence or absence of the final verb either was or was not predictable from the preceding sentence structure. In response, participants had to name any unnamed objects they could see in their own displays with utterances such as Ich habe ein Ei ("I have an egg"). The results show that speakers begin to plan their turns as soon as sufficient information is available to do so, irrespective of further incoming words.

Verbal Semantics Drives Early Anticipatory Eye Movements during the Comprehension of Verb-Initial Sentences

Studies on anticipatory processes during sentence comprehension often focus on the prediction of postverbal direct objects. In subject-initial languages (the target of most studies so far), however, the position in the sentence, the syntactic function, and the semantic role of arguments are often conflated. For example, in the sentence "The frog will eat the fly" the syntactic object ("fly") is at the same time also the last word and the patient argument of the verb. It is therefore not apparent which kind of information listeners orient to for predictive processing during sentence comprehension. A visual world eye tracking study on the verb-initial language Tagalog (Austronesian) tested what kind of information listeners use to anticipate upcoming postverbal linguistic input. The grammatical structure of Tagalog allows to test whether listeners' anticipatory gaze behavior is guided by predictions of the linear order of words, by syntactic functions (e.g., subject/object), or by semantic roles (agent/patient). Participants heard sentences of the type "Eat frog fly" or "Eat fly frog" (both meaning "The frog will eat the fly") while looking at displays containing an agent referent ("frog"), a patient referent ("fly") and a distractor. The verb carried morphological marking that allowed the order and syntactic function of agent and patient to be inferred. After having heard the verb, listeners fixated on the agent irrespective of its syntactic function or position in the sentence. While hearing the first-mentioned argument, listeners fixated on the corresponding referent in the display accordingly and then initiated saccades to the last-mentioned referent before it was encountered. The results indicate that listeners used verbal semantics to identify referents and their semantic roles early; information about word order or syntactic functions did not influence anticipatory gaze behavior directly after the verb was heard. In this verb-initial language, event semantics takes early precedence during the comprehension of sentences, while arguments are anticipated temporally more local to when they are encountered. The current experiment thus helps to better understand anticipation during language processing by employing linguistic structures not available in previously studied subject-initial languages.

Cardiorespiratory motion-compensated micro-CT image reconstruction using an artifact model-based motion estimation

PURPOSE

Cardiac in vivo micro-CT imaging of small animals typically requires double gating due to long scan times and high respiratory rates. The simultaneous respiratory and cardiac gating can either be done prospectively or retrospectively. In any case, for true 5D imaging, i.e., three spatial dimensions plus one respiratory-temporal dimension plus one cardiac temporal dimension, the amount of information corresponding to a given respiratory and cardiac phase is orders of magnitude lower than the total amount of information acquired. Achieving similar image quality for 5D than for usual 3D investigations would require increasing the amount of data and thus the applied dose to the animal. Therefore, the goal is phase-correlated imaging with high image quality but without increasing the dose level.

METHODS

To achieve this, the authors propose a new image reconstruction algorithm that makes use of all available projection data, also of that corresponding to other motion windows. In particular, the authors apply a motion-compensated image reconstruction approach that sequentially compensates for respiratory and cardiac motion to decrease the impact of sparsification. In that process, all projection data are used no matter which motion phase they were acquired in. Respiratory and cardiac motion are compensated for by using motion vector fields. These motion vector fields are estimated from initial phase-correlated reconstructions based on a deformable registration approach. To decrease the sensitivity of the registration to sparse-view artifacts, an artifact model-based approach is used including a cyclic consistent nonrigid registration algorithm.

RESULTS

The preliminary results indicate that the authors' approach removes the sparse-view artifacts of conventional phase-correlated reconstructions while maintaining temporal resolution. In addition, it achieves noise levels and spatial resolution comparable to that of nongated reconstructions due to the improved dose usage. By using the proposed motion estimation, no sensitivity to streaking artifacts has been observed.

CONCLUSIONS

Using sequential double gating combined with artifact model-based motion estimation allows to accurately estimate respiratory and cardiac motion from highly undersampled data. No sensitivity to streaking artifacts introduced by sparse angular sampling has been observed for the investigated dose levels. The motion-compensated image reconstruction was able to correct for both, respiratory and cardiac motion, by applying the estimated motion vector fields. The administered dose per animal can thus be reduced for 5D imaging allowing for longitudinal studies at the highest image quality.

Automatic fusion of lateral cephalograms and digital volume tomography data-perspective for combining two modalities in the future

OBJECTIVES

This article investigates the combination of three-dimensional (3D) digital volume tomography data with two-dimensional (2D) cephalograms in dentomaxillofacial imaging.

METHODS

An automatic hierarchical method to adjust the geometrical relations of these two modalities is presented. The approach is tested on phantom and patient case data, where the feasibility, usability and potential possibilities of the presented innovative method are highlighted. Digitally reconstructed radiographs are computed by casting rays through the 3D volume to get a 2D projection of the volume to produce realistic simulated cephalograms. Different similarity measures are considered based on variations of statistical and deterministic optimization procedures. Stability, precision and accuracy of the method are investigated.

RESULTS

The presented algorithm demonstrates a reasonable solution of the corresponding 2D/3D registration problem. Exemplary results from phantom and patient case data are presented. Tooth movement could be determined, in contrast to the 2D lateral cephalogram, separated for each side in all three spatial directions.

CONCLUSIONS

Achieved results are highlighted from a clinical point of view and demonstrate the clinical benefit in daily praxis.