Not so secret agents: Event-related potentials to semantic roles in visual event comprehension

Research across domains has suggested that agents, the doers of actions, have a processing advantage over patients, the receivers of actions. We hypothesized that agents as "event builders" for discrete actions (e.g., throwing a ball, punching) build on cues embedded in their preparatory postures (e.g., reaching back an arm to throw or punch) that lead to (predictable) culminating actions, and that these cues afford frontloading of event structure processing. To test this hypothesis, we compared event-related brain potentials (ERPs) to averbal comic panels depicting preparatory agents (ex. reaching back an arm to punch) that cued specific actions with those to non-preparatory agents (ex. arm to the side) and patients that did not cue any specific actions. We also compared subsequent completed action panels (ex. agent punching patient) across conditions, where we expected an inverse pattern of ERPs indexing the differential costs of processing completed actions asa function of preparatory cues. Preparatory agents evoked a greater frontal positivity (600-900ms) relative to non-preparatory agents and patients, while subsequent completed actions panels following non-preparatory agents elicited a smaller frontal positivity (600-900ms). These results suggest that preparatory (vs. non-) postures may differentially impact the processing of agents and subsequent actions in real time.

The role of expectations and habitual emotion regulation in emotional processing: An ERP investigation

Available evidence from separate lines of event-related potential (ERP) research has highlighted the role of expectations and emotion regulation on emotional processing by revealing that (i) expectations can alter emotional responses, and (ii) the instructed use of emotion regulation strategies may modulate emotional responses. Yet, little is known about the interplay between expectations and habitual emotion regulation strategies prior to and at the onset of an emotional event. The present study aimed to investigate this potential relationship. Participants completed an affective-cueing task consisting of cues (red squares and blue circles) signaling the likely valence of upcoming target images (negative or neutral). This task allowed us to examine the impact of expectations at 2 temporal stages, Cue Interval and Target Interval, by measuring the late positive potential (LPP) as an index of emotional processing. Habitual use of emotion regulation strategies was assessed through the Emotion Regulation Questionnaire (ERQ), which measures the use of cognitive reappraisal and expressive suppression in everyday life. In the Cue Interval, LPP amplitude was greater for negative versus neutral cues (p < .001). In the Target Interval, LPP amplitude was greater for negatively cued versus neutrally cued targets, regardless of target valence (p = .003). ERQ reappraisal, but not suppression, negatively correlated with LPP modulation as a function of cue valence during both intervals (ps < .05). These findings provide novel insights regarding the interplay between expectations and habitual emotion regulation in emotional processing both prior to and at the onset of an emotional event. (PsycINFO Database Record

Placebo effects in cognitive training

Although a large body of research shows that general cognitive ability is heritable and stable in young adults, there is recent evidence that fluid intelligence can be heightened with cognitive training. Many researchers, however, have questioned the methodology of the cognitive-training studies reporting improvements in fluid intelligence: specifically, the role of placebo effects. We designed a procedure to intentionally induce a placebo effect via overt recruitment in an effort to evaluate the role of placebo effects in fluid intelligence gains from cognitive training. Individuals who self-selected into the placebo group by responding to a suggestive flyer showed improvements after a single, 1-h session of cognitive training that equates to a 5- to 10-point increase on a standard IQ test. Controls responding to a nonsuggestive flyer showed no improvement. These findings provide an alternative explanation for effects observed in the cognitive-training literature and the brain-training industry, revealing the need to account for confounds in future research.

Into the Wild: Neuroergonomic Differentiation of Hand-Held and Augmented Reality Wearable Displays during Outdoor Navigation with Functional Near Infrared Spectroscopy

Highly mobile computing devices promise to improve quality of life, productivity, and performance. Increased situation awareness and reduced mental workload are two potential means by which this can be accomplished. However, it is difficult to measure these concepts in the “wild”. We employed ultra-portable battery operated and wireless functional near infrared spectroscopy (fNIRS) to non-invasively measure hemodynamic changes in the brain’s Prefrontal cortex (PFC). Measurements were taken during navigation of a college campus with either a hand-held display, or an Augmented reality wearable display (ARWD). Hemodynamic measures were also paired with secondary tasks of visual perception and auditory working memory to provide behavioral assessment of situation awareness and mental workload. Navigating with an augmented reality wearable display produced the least workload during the auditory working memory task, and a trend for improved situation awareness in our measures of prefrontal hemodynamics. The hemodynamics associated with errors were also different between the two devices. Errors with an augmented reality wearable display were associated with increased prefrontal activity and the opposite was observed for the hand-held display. This suggests that the cognitive mechanisms underlying errors between the two devices differ. These findings show fNIRS is a valuable tool for assessing new technology in ecologically valid settings and that ARWDs offer benefits with regards to mental workload while navigating, and potentially superior situation awareness with improved display design.

Brief exposure to aversive stimuli impairs visual selective attention

Although it is well established that stress can disrupt complex cognitive functions, relatively little is known about how it influences visual processing, especially in terms of visual selective attention. In the current study, we used highly aversive images, taken from the International Affective Picture System, to induce acute, low-intensity stress while participants performed a visual discrimination task. Consistent with prior research, we found that anticipation of aversive stimuli increased overall amplitude of the N170, suggesting an increase in early sensory gain. More importantly, we found that stress disrupted visual selective attention. While in no-stress blocks, the amplitude of the face-sensitive N170 was higher when participants attended to faces rather than scenes in face-scene overlay images; this effect was absent under stress. This was because of an increase in N170 amplitude in the scene-attend condition under stress. We interpret these findings as suggesting that even low-intensity acute stress can impair participants' ability to filter out task-irrelevant information. We discuss our findings in relation to how even brief exposure to low-intensity stress may adversely impact both healthy and clinical populations.

The difference between "giving a rose" and "giving a kiss": Sustained neural activity to the light verb construction

We used event-related potentials (ERPs) to investigate the neurocognitive mechanisms associated with processing light verb constructions such as "give a kiss". These constructions consist of a semantically underspecified light verb ("give") and an event nominal that contributes most of the meaning and also activates an argument structure of its own ("kiss"). This creates a mismatch between the syntactic constituents and the semantic roles of a sentence. Native speakers read German verb-final sentences that contained light verb constructions (e.g., "Julius gave Anne a kiss"), non-light constructions (e.g., "Julius gave Anne a rose"), and semantically anomalous constructions (e.g., *"Julius gave Anne a conversation"). ERPs were measured at the critical verb, which appeared after all its arguments. Compared to non-light constructions, the light verb constructions evoked a widely distributed, frontally focused, sustained negative-going effect between 500 and 900 ms after verb onset. We interpret this effect as reflecting working memory costs associated with complex semantic processes that establish a shared argument structure in the light verb constructions.

When events change their nature: the neurocognitive mechanisms underlying aspectual coercion

The verb "pounce" describes a single, near-instantaneous event. Yet, we easily understand that, "For several minutes the cat pounced…" describes a situation in which multiple pounces occurred, although this interpretation is not overtly specified by the sentence's syntactic structure or by any of its individual words--a phenomenon known as "aspectual coercion." Previous psycholinguistic studies have reported processing costs in association with aspectual coercion, but the neurocognitive mechanisms giving rise to these costs remain contentious. Additionally, there is some controversy about whether readers commit to a full interpretation of the event when the aspectual information becomes available, or whether they leave it temporarily underspecified until later in the sentence. Using ERPs, we addressed these questions in a design that fully crossed context type (punctive, durative, frequentative) with verb type (punctive, durative). We found a late, sustained negativity to punctive verbs in durative contexts, but not in frequentative (e.g., explicitly iterative) contexts. This effect was distinct from the N400 in both its time course and scalp distribution, suggesting that it reflected a different underlying neurocognitive mechanism. We also found that ERPs to durative verbs were unaffected by context type. Together, our results provide strong evidence that neural activity associated with aspectual coercion is driven by the engagement of a morphosyntactically unrealized semantic operator rather than by violations of real-world knowledge, more general shifts in event representation, or event iterativity itself. More generally, our results add to a growing body of evidence that a set of late-onset sustained negativities reflect elaborative semantic processing that goes beyond simply combining the meaning of individual words with syntactic structure to arrive at a final representation of meaning.

Prediction, events, and the advantage of agents: the processing of semantic roles in visual narrative

Agents consistently appear prior to Patients in sentences, manual signs, and drawings, and Agents are responded to faster when presented in visual depictions of events. We hypothesized that this "Agent advantage" reflects Agents' role in event structure. We investigated this question by manipulating the depictions of Agents and Patients in preparatory actions in wordless visual narratives. We found that Agents elicited a greater degree of predictions regarding upcoming events than Patients, that Agents are viewed longer than Patients, independent of serial order, and that visual depictions of actions are processed more quickly following the presentation of an Agent vs. a Patient. Taken together these findings support the notion that Agents initiate the building of event representation. We suggest that Agent First orders facilitate the interpretation of events as they unfold and that the saliency of Agents within visual representations of events is driven by anticipation of upcoming events.

Multiple Influences of Semantic Memory on Sentence Processing: Distinct Effects of Semantic Relatedness on Violations of Real-World Event/State Knowledge and Animacy Selection Restrictions

We aimed to determine whether semantic relatedness between an incoming word and its preceding context can override expectations based on two types of stored knowledge: real-world knowledge about the specific events and states conveyed by a verb, and the verb's broader selection restrictions on the animacy of its argument. We recorded event-related potentials on post-verbal Agent arguments as participants read and made plausibility judgments about passive English sentences. The N400 evoked by incoming animate Agent arguments that violated expectations based on real-world event/state knowledge, was strongly attenuated when they were semantically related to the context. In contrast, semantic relatedness did not modulate the N400 evoked by inanimate Agent arguments that violated the preceding verb's animacy selection restrictions. These findings suggest that, under these task and experimental conditions, semantic relatedness can facilitate processing of post-verbal animate arguments that violate specific expectations based on real-world event/state knowledge, but only when the semantic features of these arguments match the coarser-grained animacy restrictions of the verb. Animacy selection restriction violations also evoked a P600 effect, which was not modulated by semantic relatedness, suggesting that it was triggered by propositional impossibility. Together, these data indicate that the brain distinguishes between real-world event/state knowledge and animacy-based selection restrictions during online processing.

(Pea)nuts and bolts of visual narrative: structure and meaning in sequential image comprehension

Just as syntax differentiates coherent sentences from scrambled word strings, the comprehension of sequential images must also use a cognitive system to distinguish coherent narrative sequences from random strings of images. We conducted experiments analogous to two classic studies of language processing to examine the contributions of narrative structure and semantic relatedness to processing sequential images. We compared four types of comic strips: (1) Normal sequences with both structure and meaning, (2) Semantic Only sequences (in which the panels were related to a common semantic theme, but had no narrative structure), (3) Structural Only sequences (narrative structure but no semantic relatedness), and (4) Scrambled sequences of randomly-ordered panels. In Experiment 1, participants monitored for target panels in sequences presented panel-by-panel. Reaction times were slowest to panels in Scrambled sequences, intermediate in both Structural Only and Semantic Only sequences, and fastest in Normal sequences. This suggests that both semantic relatedness and narrative structure offer advantages to processing. Experiment 2 measured ERPs to all panels across the whole sequence. The N300/N400 was largest to panels in both the Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in the Normal sequences. This implies that a combination of narrative structure and semantic relatedness can facilitate semantic processing of upcoming panels (as reflected by the N300/N400). Also, panels in the Scrambled sequences evoked a larger left-lateralized anterior negativity than panels in the Structural Only sequences. This localized effect was distinct from the N300/N400, and appeared despite the fact that these two sequence types were matched on local semantic relatedness between individual panels. These findings suggest that sequential image comprehension uses a narrative structure that may be independent of semantic relatedness. Altogether, we argue that the comprehension of visual narrative is guided by an interaction between structure and meaning.

Electrophysiological Evidence for Use of the Animacy Hierarchy, but not Thematic Role Assignment, During Verb Argument Processing

Animacy is known to play an important role in language processing and production, but debate remains as to how it exerts its effects: 1) through links to syntactic ordering, 2) through inherent differences between animate and inanimate entities in their salience/lexico-semantic accessibility, 3) through links to specific thematic roles. We contrasted these three accounts in two event related potential (ERP) experiments examining the processing of direct object arguments in simple English sentences. In Experiment 1, we found a larger N400 to animate than inanimate direct object arguments assigned the Patient role, ruling out the second account. In Experiment 2 we found no difference in the N400 evoked by animate direct object arguments assigned the Patient role (prototypically inanimate) and those assigned the Experiencer role (prototypically animate), ruling out the third account. We therefore suggest that animacy may impact processing through a direct link to syntactic linear ordering, at least on post-verbal arguments in English. We also examined processing on direct object arguments that violated the animacy-based selection restriction constraints of their preceding verbs. These violations evoked a robust P600, which was not modulated by thematic role assignment or reversibility, suggesting that the so-called semantic P600 is driven by overall propositional impossibility, rather than thematic role reanalysis.

Electrophysiological correlates of complement coercion

This study examined the electrophysiological correlates of complement coercion. ERPs were measured as participants read and made acceptability judgments about plausible coerced sentences, plausible noncoerced sentences, and highly implausible animacy-violated sentences ("The journalist began/wrote/astonished the article before his coffee break"). Relative to noncoerced complement nouns, the coerced nouns evoked an N400 effect. This effect was not modulated by the number of possible activities implied by the coerced nouns (e.g., began reading the article; began writing the article) and did not differ either in magnitude or scalp distribution from the N400 effect evoked by the animacy-violated complement nouns. We suggest that the N400 modulation to both coerced and animacy-violated complement nouns reflected different types of mismatches between the semantic restrictions of the verb and the semantic properties of the incoming complement noun. This is consistent with models holding that a verb's semantic argument structure is represented and stored at a distinct level from its syntactic argument structure. Unlike the coerced complement noun, the animacy-violated nouns also evoked a robust P600 effect, which may have been triggered by the judgments of the highly implausible (syntactically determined) meanings of the animacy-violated propositions. No additional ERP effects were seen in the coerced sentences until the sentence-final word that, relative to sentence-final words in the noncoerced sentences, evoked a sustained anteriorly distributed positivity. We suggest that this effect reflected delayed attempts to retrieve the specific event(s) implied by coerced complement nouns.

Establishing causal coherence across sentences: an ERP study

This study examined neural activity associated with establishing causal relationships across sentences during on-line comprehension. ERPs were measured while participants read and judged the relatedness of three-sentence scenarios in which the final sentence was highly causally related, intermediately related, and causally unrelated to its context. Lexico-semantic co-occurrence was matched across the three conditions using a Latent Semantic Analysis. Critical words in causally unrelated scenarios evoked a larger N400 than words in both highly causally related and intermediately related scenarios, regardless of whether they appeared before or at the sentence-final position. At midline sites, the N400 to intermediately related sentence-final words was attenuated to the same degree as to highly causally related words, but otherwise the N400 to intermediately related words fell in between that evoked by highly causally related and intermediately related words. No modulation of the late positivity/P600 component was observed across conditions. These results indicate that both simple and complex causal inferences can influence the earliest stages of semantically processing an incoming word. Further, they suggest that causal coherence, at the situation level, can influence incremental word-by-word discourse comprehension, even when semantic relationships between individual words are matched.

Acute stress persistently enhances estrogen levels in the female rat

Here we tested whether exposure to either tailshock or swim stress alters ovarian hormone levels, estrogen and progesterone, in females and whether the effects are persistent. Adrenal hormone levels were also measured in males and females. Estradiol levels were elevated in unstressed females during proestrus relative to females in other stages of estrous, and exposure to the stressors enhanced estradiol beyond basal levels. For females stressed during diestrus 2, estradiol levels were elevated immediately after stressor cessation and up to 24 hrs. Exposure to tailshock, but not swim-stress, transiently enhanced progesterone in females stressed during the stage of proestrus and estrus. Glucocorticoid levels were elevated in response to both stressors and were supraelevated in females under both basal and stress conditions relative to males, particularly in blood from females exposed to acute swim stress. These results indicate that exposure to a relatively acute stressful event immediately and persistently enhances serum estradiol and are discussed in the context of reports that exposure to the same stressors immediately and persistently impairs associative learning in the female rat.