Semantic incongruity attracts attention at a pre-conscious level: Evidence from a TMS study

Beyond visual integration: sensitivity of the temporal-parietal junction for objects, places, and faces

One important role of the TPJ is the contribution to perception of the global gist in hierarchically organized stimuli where individual elements create a global visual percept. However, the link between clinical findings in simultanagnosia and neuroimaging in healthy subjects is missing for real-world global stimuli, like visual scenes. It is well-known that hierarchical, global stimuli activate TPJ regions and that simultanagnosia patients show deficits during the recognition of hierarchical stimuli and real-world visual scenes. However, the role of the TPJ in real-world scene processing is entirely unexplored. In the present study, we first localized TPJ regions significantly responding to the global gist of hierarchical stimuli and then investigated the responses to visual scenes, as well as single objects and faces as control stimuli. All three stimulus classes evoked significantly positive univariate responses in the previously localized TPJ regions. In a multivariate analysis, we were able to demonstrate that voxel patterns of the TPJ were classified significantly above chance level for all three stimulus classes. These results demonstrate a significant involvement of the TPJ in processing of complex visual stimuli that is not restricted to visual scenes and that the TPJ is sensitive to different classes of visual stimuli with a specific signature of neuronal activations.

A transcranial magnetic stimulation study on the role of the left intraparietal sulcus in temporal orienting of attention

Adaptive behavior requires the ability to orient attention to the moment in time at which a relevant event is likely to occur. Temporal orienting of attention has been consistently associated with activation of the left intraparietal sulcus (IPS) in prior fMRI studies. However, a direct test of its causal involvement in temporal orienting is still lacking. The present study tackled this issue by transiently perturbing left IPS activity with either online (Experiment 1) or offline (Experiment 2) transcranial magnetic stimulation (TMS). In both experiments, participants performed a temporal orienting task, alternating between blocks in which a temporal cue predicted when a subsequent target would appear and blocks in which a neutral cue provided no information about target timing. In Experiment 1 we used an online TMS protocol, aiming to interfere specifically with cue-related temporal processes, whereas in Experiment 2 we employed an offline protocol whereby participants performed the temporal orienting task before and after receiving TMS. The right IPS and/or the vertex were stimulated as active control regions. While results replicated the canonical pattern of temporal orienting effects on reaction time, with faster responses for temporal than neutral trials, these effects were not modulated by TMS over the left IPS (as compared to the right IPS and/or vertex regions) regardless of the online or offline protocol used. Overall, these findings challenge the causal role of the left IPS in temporal orienting of attention inviting further research on its underlying neural substrates.

Perception of semantic relations in scenes: A registered report study of attention hold

To what extent the semantic relations present in scenes guide spatial attention automatically remains a matter of debate. Considering that spatial attention can be understood as a sequence of shifts, engagements, and disengagements, semantic relations might affect each stage of this process differently. Therefore, we investigated whether objects that violate semantic rules engage attention for longer than objects that are expected in a given context. The experiment involved a central presentation of a distractor scene that contained a semantically congruent or incongruent object, and a peripheral presentation of a small target letter. We found that incongruent scenes did not delay responses to the peripheral target, which indicates that they did not hold attention for longer than congruent scenes. Therefore, by showing that violations of semantic relations do not engage attention automatically, our study contributes to a better understanding of how attention operates in naturalistic settings.

To Grasp the World at a Glance: The Role of Attention in Visual and Semantic Associative Processing

Associative relations among words, concepts and percepts are the core building blocks of high-level cognition. When viewing the world ‘at a glance’, the associative relations between objects in a scene, or between an object and its visual background, are extracted rapidly. The extent to which such relational processing requires attentional capacity, however, has been heavily disputed over the years. In the present manuscript, I review studies investigating scene–object and object–object associative processing. I then present a series of studies in which I assessed the necessity of spatial attention to various types of visual–semantic relations within a scene. Importantly, in all studies, the spatial and temporal aspects of visual attention were tightly controlled in an attempt to minimize unintentional attention shifts from ‘attended’ to ‘unattended’ regions. Pairs of stimuli—either objects, scenes or a scene and an object—were briefly presented on each trial, while participants were asked to detect a pre-defined target category (e.g., an animal, a nonsense shape). Response times (RTs) to the target detection task were registered when visual attention spanned both stimuli in a pair vs. when attention was focused on only one of two stimuli. Among non-prioritized stimuli that were not defined as to-be-detected targets, findings consistently demonstrated rapid associative processing when stimuli were fully attended, i.e., shorter RTs to associated than unassociated pairs. Focusing attention on a single stimulus only, however, largely impaired this relational processing. Notably, prioritized targets continued to affect performance even when positioned at an unattended location, and their associative relations with the attended items were well processed and analyzed. Our findings portray an important dissociation between unattended task-irrelevant and task-relevant items: while the former require spatial attentional resources in order to be linked to stimuli positioned inside the attentional focus, the latter may influence high-level recognition and associative processes via feature-based attentional mechanisms that are largely independent of spatial attention.

Crossmodal Semantic Congruence Interacts with Object Contextual Consistency in Complex Visual Scenes to Enhance Short-Term Memory Performance

Object sounds can enhance the attentional selection and perceptual processing of semantically-related visual stimuli. However, it is currently unknown whether crossmodal semantic congruence also affects the post-perceptual stages of information processing, such as short-term memory (STM), and whether this effect is modulated by the object consistency with the background visual scene. In two experiments, participants viewed everyday visual scenes for 500 ms while listening to an object sound, which could either be semantically related to the object that served as the STM target at retrieval or not. This defined crossmodal semantically cued vs. uncued targets. The target was either in- or out-of-context with respect to the background visual scene. After a maintenance period of 2000 ms, the target was presented in isolation against a neutral background, in either the same or different spatial position as in the original scene. The participants judged the same vs. different position of the object and then provided a confidence judgment concerning the certainty of their response. The results revealed greater accuracy when judging the spatial position of targets paired with a semantically congruent object sound at encoding. This crossmodal facilitatory effect was modulated by whether the target object was in- or out-of-context with respect to the background scene, with out-of-context targets reducing the facilitatory effect of object sounds. Overall, these findings suggest that the presence of the object sound at encoding facilitated the selection and processing of the semantically related visual stimuli, but this effect depends on the semantic configuration of the visual scene.

Transcranial Magnetic Stimulation of the Right Superior Parietal Lobule Modulates the Retro-Cue Benefit in Visual Short-Term Memory

Several studies have shown enhanced performance in change detection tasks when spatial cues indicating the probe's location are presented after the memory array has disappeared (i.e., retro-cues) compared with spatial cues that are presented simultaneously with the test array (i.e., post-cues). This retro-cue benefit led some authors to propose the existence of two different stores of visual short-term memory: a weak but high-capacity store (fragile memory (FM)) linked to the effect of retro-cues and a robust but low-capacity store (working memory (WM)) linked to the effect of post-cues. The former is thought to be an attention-free system, whereas the latter would strictly depend on selective attention. Nonetheless, this dissociation is under debate, and several authors do not consider retro-cues as a proxy to measure the existence of an independent memory system (e.g., FM). We approached this controversial issue by altering the attention-related functions in the right superior parietal lobule (SPL) by transcranial magnetic stimulation (TMS), whose effects were mediated by the integrity of the right superior longitudinal fasciculus (SLF). Specifically, we asked whether TMS on the SPL affected the performance of retro cues vs. post-cues to a similar extent. The results showed that TMS on the SPL, mediated by right SLF-III integrity, produced a modulation of the retro-cue benefit, namely a memory capacity decrease in the post-cues but not in the retro-cues. These findings have strong implications for the debate on the existence of independent stages of visual short-term memory and for the growing literature showing a key role of the SLF for explaining the variability of TMS effects across participants.

Large-scale brain networks underlying non-spatial attention updating: Towards understanding the function of the temporoparietal junction

The temporoparietal junction (TPJ) and related areas are activated when a target stimulus appears at unexpected locations in Posner's spatial-cueing paradigm, and also when deviant stimuli are presented within a series of standard events in oddball paradigms. This type of activation corresponds to the ventral attention network (VAN), for regions defined on the basis of the spatial task. However, involvement of the VAN in object-based updating of attention has rarely been examined. In the present study, we used functional magnetic resonance imaging to investigate brain responses to (i) invalid targets after category-cueing and (ii) neutrally cued targets deviating in category from the background series of pictures. Bilateral TPJ activation was observed in response to invalidly cued targets, as compared to neutrally cued targets. Reference to the main large-scale brain networks showed that peaks of this activation located in the angular gyrus and inferior parietal lobule belonged to the default mode (DMN) and fronto-parietal networks (FPN), respectively. We found that VAN regions were involved only for simple detection activity. We conclude that spatial and non-spatial reorienting of attention rely on different network underpinnings. Our data suggest that DMN and FPN activity may support the ability to disengage from contextually irrelevant information.

Temporo-parietal contribution to the mental representations of self/other face

Face recognition requires comparing the current visual input with stored mental representations of faces. Based on its role in visual recognition of faces and mental representation of the body, we hypothesized that the right temporo-parietal junction (rTPJ) could be implicated also in processing mental representation of faces. To test this hypothesis, we asked 30 neurotypical participants to perform mental rotation (laterality judgment of rotated pictures) of self- and other-face images, before and after the inhibition of rTPJ through repetitive transcranial magnetic stimulation. After inhibition of rTPJ the mental rotation of self-face was slower than other-face. In the control condition the mental rotation of self/other faces was not significantly different. This supports that the role of rTPJ extends to mental representation of faces, specifically for the self. Since the experimental task did not require to explicitly recognize identity, we propose that unconscious identity attribution affects also the mental representation of faces. The present study offers insights on the involvement rTPJ in mental representation of faces and proposes that the neural substrate dedicated to mental representation of faces goes beyond the traditional visual and memory areas.

Automatic Attention Capture by Threatening, But Not by Semantically Incongruent Natural Scene Images

Visual objects are typically perceived as parts of an entire visual scene, and the scene’s context provides information crucial in the object recognition process. Fundamental insights into the mechanisms of context-object integration have come from research on semantically incongruent objects, which are defined as objects with a very low probability of occurring in a given context. However, the role of attention in processing of the context-object mismatch remains unclear, with some studies providing evidence in favor, but other against an automatic capture of attention by incongruent objects. Therefore, in the present study, 25 subjects completed a dot-probe task, in which pairs of scenes—congruent and incongruent or neutral and threatening—were presented as task-irrelevant distractors. Importantly, threatening scenes are known to robustly capture attention and thus were included in the present study to provide a context for interpretation of results regarding incongruent scenes. Using N2 posterior-contralateral ERP component as a primary measure, we revealed that threatening images indeed capture attention automatically and rapidly, but semantically incongruent scenes do not benefit from an automatic attentional selection. Thus, our results suggest that identification of the context-object mismatch is not preattentive.

The causal role of the left parietal lobe in facilitation and inhibition of return

Following non-informative peripheral cues, responses are facilitated at the cued compared to the uncued location at short cue-target intervals. This effect reverses at longer intervals, giving rise to Inhibition of Return (IOR). The integration-segregation hypothesis (Lupiáñez, 2010) suggests that peripheral cues always produce an onset-detection cost regardless the behavioral cueing effect that is measured - either facilitation or IOR. In the present study, we used transcranial magnetic stimulation (TMS) to investigate the causal contribution of this detection cost to performance. We used a cueing paradigm with a target discrimination task that was preceded by a non-informative peripheral cue. The presence-absence of a central intervening event was manipulated. Online TMS to the left superior parietal lobe (compared to an active vertex stimulation) lead to an overall more positive effect (faster responses for cued as compared to uncued trials), by putatively impairing the detection cost contribution to performance. The data revealed a strong association between overall RT and the TMS effect, and also between overall RT and the integrity of the first branch of the left superior longitudinal fascicule. These results have critical implications not only for the open debate about the mechanism/s underlying spatial orienting effects, but also for the growing literature demonstrating that white matter connectivity is crucial for explaining inter-individual behavioral variability.

Congruency Encoding Effects on Recognition Memory: A Stage-Specific Account of Desirable Difficulty

Recent research suggests that selectively attending to relevant stimuli while having to ignore or resist conflicting stimuli can lead to improvements in learning. While mostly discussed within a broader "desirable difficulty" framework in the memory and education literatures, some recent work has focused on more mechanistic questions of how processing conflict (e.g., from incongruent primes) might elicit increased attention and control, producing enhanced incidental encoding of high-conflict stimuli. This encoding benefit for high-control-demand or high-difficulty situations has been broadly conceptualized as a task-general property, with no strong prediction of what particular task elements should produce this effect. From stage processing models of single- and dual-task performance, we propose that memory-enhancing difficulty manipulations should strongly depend on inducing additional cognitive control at particular processing stages. Over six experiments, we show that a memory benefit is produced when increased cognitive control (via incongruency priming) focuses additional processing on the core meaning of to-be-tested stimuli at the semantic categorization stage. In contrast, incongruency priming targeted at response selection within the same task produces similar effects on initial task performance, but gives no memory benefit for high-conflict trials. We suggest that a simple model of limited-capacity and stage-specific cognitive control allocation can account for and predict where and when conflict/difficulty encoding benefits will occur, and may serve as a model for desirable difficulty effects more broadly.