Attentional modulation of repetition attenuation is anatomically dissociable for scenes and faces

Mapping the neural mechanism that distinguishes between holistic thinking and analytic thinking

Holistic and analytic thinking are two distinct modes of thinking used to interpret the world with relative preferences varying across cultures. While most research on these thinking styles has focused on behavioral and cognitive aspects, a few studies have utilized functional magnetic resonance imaging (fMRI) to explore the correlations between brain metrics and self-reported scale scores. Other fMRI studies used single holistic and analytic thinking tasks. As a single task may involve processing in spurious low-level regions, we used two different holistic and analytic thinking tasks, namely the frame-line task and the triad task, to seek convergent brain regions to distinguish holistic and analytic thinking using multivariate pattern analysis (MVPA). Results showed that brain regions fundamental to distinguish holistic and analytic thinking include the bilateral frontal lobes, bilateral parietal lobes, bilateral precentral and postcentral gyrus, bilateral supplementary motor areas, bilateral fusiform, bilateral insula, bilateral angular gyrus, left cuneus, and precuneus, left olfactory cortex, cingulate gyrus, right caudate and putamen. Our study maps brain regions that distinguish between holistic and analytic thinking and provides a new approach to explore the neural representation of cultural constructs. We provide initial evidence connecting culture-related brain regions with language function to explain the origins of cultural differences in cognitive styles.

Visual Distraction Disrupts Category-tuned Attentional Filters in Ventral Visual Cortex

Our behavioral goals shape how we process information via attentional filters that prioritize goal-relevant information, dictating both where we attend and what we attend to. When something unexpected or salient appears in the environment, it captures our spatial attention. Extensive research has focused on the spatiotemporal aspects of attentional capture, but what happens to concurrent nonspatial filters during visual distraction? Here, we demonstrate a novel, broader consequence of distraction: widespread disruption to filters that regulate category-specific object processing. We recorded fMRI while participants viewed arrays of face/house hybrid images. On distractor-absent trials, we found robust evidence for the standard signature of category-tuned attentional filtering: greater BOLD activation in fusiform face area during attend-faces blocks and in parahippocampal place area during attend-houses blocks. However, on trials where a salient distractor (white rectangle) flashed abruptly around a nontarget location, not only was spatial attention captured, but the concurrent category-tuned attentional filter was disrupted, revealing a boost in activation for the to-be-ignored category. This disruption was robust, resulting in errant processing-and early on, prioritization-of goal-inconsistent information. These findings provide a direct test of the filter disruption theory: that in addition to disrupting spatial attention, distraction also disrupts nonspatial attentional filters tuned to goal-relevant information. Moreover, these results reveal that, under certain circumstances, the filter disruption may be so profound as to induce a full reversal of the attentional control settings, which carries novel implications for both theory and real-world perception.

Infrequent faces bias social attention differently in manual and oculomotor measures

Although attention is thought to be spontaneously biased by social cues like faces and eyes, recent data have demonstrated that when extraneous content, context, and task factors are controlled, attentional biasing is abolished in manual responses while still occurring sparingly in oculomotor measures. Here, we investigated how social attentional biasing was affected by face novelty by measuring responses to frequently presented (i.e., those with lower novelty) and infrequently presented (i.e., those with higher novelty) face identities. Using a dot-probe task, participants viewed either the same face and house identity that was frequently presented on half of the trials or sixteen different face and house identities that were infrequently presented on the other half of the trials. A response target occurred with equal probability at the previous location of the eyes or mouth of the face or the top or bottom of the house. Experiment 1 measured manual responses to the target while participants maintained central fixation. Experiment 2 additionally measured participants' natural oculomotor behaviour when their eye movements were not restricted. Across both experiments, no evidence of social attentional biasing was found in manual data. However, in Experiment 2, there was a reliable oculomotor bias towards the eyes of infrequently presented upright faces. Together, these findings suggest that face novelty does not facilitate manual measures of social attention, but it appears to promote spontaneous oculomotor biasing towards the eyes of infrequently presented novel faces.

Attention modulates repetition effects in a context of low periodicity

Stimulus repetition can result in a reduction in neural responses (i.e., repetition suppression) in neuroimaging studies. Predictive coding models of perception postulate that this phenomenon largely reflects the top-down attenuation of prediction errors. Electroencephalography research further demonstrated that repetition effects consist of sequentially ordered attention-independent and attention-dependent components in a context of high periodicity. However, the statistical structure of our auditory environment is richer than that of a fixed pattern. It remains unclear if the attentional modulation of repetition effects can be generalised to a setting which better represents the nature of our auditory environment. Here we used electroencephalography to investigate whether the attention-independent and attention-dependent components of repetition effects previously described in the auditory modality remain in a context of low periodicity where temporary disruption might be absent/present. Participants were presented with repetition trains of various lengths, with/without temporary disruptions. We found attention-independent and attention-dependent repetition effects on, respectively, the P2 and P3a event-related potential components. This pattern of results is in line with previous research, confirming that the attenuation of prediction errors upon stimulus repetition is first registered regardless of attentional state before further attenuation of attended but not unattended prediction errors takes place. However, unlike previous reports, these effects manifested on later components. This divergence from previous studies is discussed in terms of the possible contribution of contextual factors.

Visual Gender Cues Guide Crossmodal Selective Attending to a Gender-Congruent Voice During Dichotic Listening

Visual input of a face appears to influence the ability to selectively attend to one voice over another simultaneous voice. We examined this crossmodal effect, specifically the role face gender may have on selective attention to male and female gendered simultaneous voices. Using a within-subjects design, participants were presented with a dynamic male face, female face, or fixation cross, with each condition being paired with a dichotomous audio stream of male and female voices reciting different lists of concrete nouns. In Experiment 1a, the female voice was played in the right ear and the male voice in the left ear. In Experiment 1b, both voices were played in both ears with differences in volume mimicking the interaural intensity difference between disparately localized voices in naturalistic situations. Free recall of words spoken by the two voices immediately following stimulus presentation served as a proxy measure of attention. In both sections of the experiment, crossmodal congruity of face gender enhanced same-gender word recall. This effect indicates that crossmodal interaction between voices and faces guides auditory attention. The results contribute to our understanding of how humans navigate the crossmodal relationship between voices and faces to direct attention in social interactions such as those in the cocktail party scenario.

The Intricate Interplay of Spatial Attention and Expectation: a Multisensory Perspective

Attention (i.e., task relevance) and expectation (i.e., signal probability) are two critical top-down mechanisms guiding perceptual inference. Attention prioritizes processing of information that is relevant for observers’ current goals. Prior expectations encode the statistical structure of the environment. Research to date has mostly conflated spatial attention and expectation. Most notably, the Posner cueing paradigm manipulates spatial attention using probabilistic cues that indicate where the subsequent stimulus is likely to be presented. Only recently have studies attempted to dissociate the mechanisms of attention and expectation and characterized their interactive (i.e., synergistic) or additive influences on perception. In this review, we will first discuss methodological challenges that are involved in dissociating the mechanisms of attention and expectation. Second, we will review research that was designed to dissociate attention and expectation in the unisensory domain. Third, we will review the broad field of crossmodal endogenous and exogenous spatial attention that investigates the impact of attention across the senses. This raises the critical question of whether attention relies on amodal or modality-specific mechanisms. Fourth, we will discuss recent studies investigating the role of both spatial attention and expectation in multisensory perception, where the brain constructs a representation of the environment based on multiple sensory inputs. We conclude that spatial attention and expectation are closely intertwined in almost all circumstances of everyday life. Yet, despite their intimate relationship, attention and expectation rely on partly distinct neural mechanisms: while attentional resources are mainly shared across the senses, expectations can be formed in a modality-specific fashion.

Reward supports flexible orienting of attention to category information and influences subsequent memory

Preparatory control of attention facilitates the efficient processing and encoding of an expected stimulus. However, this can occur at the expense of increasing the processing cost of unexpected stimuli. Preparatory control can be influenced by motivational factors, such as the expectation of a reward. Interestingly, expectation of a high reward can increase target processing, as well as reduce the cost associated with reorienting. Using a semantic cueing paradigm, we examined the interaction of reward expectation and cue-validity on semantic judgment performance and subsequent memory. Preparatory attention was assessed with pupillometry. Valid category cueing was associated with better semantic judgment performance and better subsequent memory compared to invalidly cued items. Higher reward also resulted in a larger pre-target pupil diameter, which could be indicative of increased preparatory task engagement or arousal. Critically, higher reward also reduced reorienting cost in both semantic judgment and subsequent memory performance. Our findings suggest that reward expectation can facilitate the effective control of preparatory attention for semantic information, and can support optimal goal-directed behavior based on changing task demands.

Additive and interactive effects of spatial attention and expectation on perceptual decisions

Spatial attention and expectation are two critical top-down mechanisms controlling perceptual inference. Based on previous research it remains unclear whether their influence on perceptual decisions is additive or interactive. We developed a novel multisensory approach that orthogonally manipulated spatial attention (i.e. task-relevance) and expectation (i.e. signal probability) selectively in audition and evaluated their effects on observers' responses in vision. Critically, while experiment 1 manipulated expectation directly via the probability of task-relevant auditory targets across hemifields, experiment 2 manipulated it indirectly via task-irrelevant auditory non-targets. Surprisingly, our results demonstrate that spatial attention and signal probability influence perceptual decisions either additively or interactively. These seemingly contradictory results can be explained parsimoniously by a model that combines spatial attention, general and spatially selective response probabilities as predictors with no direct influence of signal probability. Our model provides a novel perspective on how spatial attention and expectation facilitate effective interactions with the environment.

Real-time decoding of covert attention in higher-order visual areas

Brain-computer-interfaces (BCI) provide a means of using human brain activations to control devices for communication. Until now this has only been demonstrated in primary motor and sensory brain regions, using surgical implants or non-invasive neuroimaging techniques. Here, we provide proof-of-principle for the use of higher-order brain regions involved in complex cognitive processes such as attention. Using realtime fMRI, we implemented an online 'winner-takes-all approach' with quadrant-specific parameter estimates, to achieve single-block classification of brain activations. These were linked to the covert allocation of attention to real-world images presented at 4-quadrant locations. Accuracies in three target regions were significantly above chance, with individual decoding accuracies reaching upto 70%. By utilising higher order mental processes, 'cognitive BCIs' access varied and therefore more versatile information, potentially providing a platform for communication in patients who are unable to speak or move due to brain injury.

Auditory to Visual Cross-Modal Adaptation for Emotion: Psychophysical and Neural Correlates

Adaptation is fundamental in sensory processing and has been studied extensively within the same sensory modality. However, little is known about adaptation across sensory modalities, especially in the context of high-level processing, such as the perception of emotion. Previous studies have shown that prolonged exposure to a face exhibiting one emotion, such as happiness, leads to contrastive biases in the perception of subsequently presented faces toward the opposite emotion, such as sadness. Such work has shown the importance of adaptation in calibrating face perception based on prior visual exposure. In the present study, we showed for the first time that emotion-laden sounds, like laughter, adapt the visual perception of emotional faces, that is, subjects more frequently perceived faces as sad after listening to a happy sound. Furthermore, via electroencephalography recordings and event-related potential analysis, we showed that there was a neural correlate underlying the perceptual bias: There was an attenuated response occurring at ∼ 400 ms to happy test faces and a quickened response to sad test faces, after exposure to a happy sound. Our results provide the first direct evidence for a behavioral cross-modal adaptation effect on the perception of facial emotion, and its neural correlate.

Selective Attention to Faces in a Rapid Visual Stream: Hemispheric Differences in Enhancement and Suppression of Category-selective Neural Activity

In daily life, efficient perceptual categorization of faces occurs in dynamic and highly complex visual environments. Yet the role of selective attention in guiding face categorization has predominantly been studied under sparse and static viewing conditions, with little focus on disentangling the impact of attentional enhancement and suppression. Here we show that attentional enhancement and suppression exert a differential impact on face categorization supported by the left and right hemispheres. We recorded 128-channel EEG while participants viewed a 6-Hz stream of object images (buildings, animals, objects, etc.) with a face image embedded as every fifth image (i.e., OOOOFOOOOFOOOOF…). We isolated face-selective activity by measuring the response at the face presentation frequency (i.e., 6 Hz/5 = 1.2 Hz) under three conditions: Attend Faces, in which participants monitored the sequence for instances of female faces; Attend Objects, in which they responded to instances of guitars; and Baseline, in which they performed an orthogonal task on the central fixation cross. During the orthogonal task, face-specific activity was predominantly centered over the right occipitotemporal region. Actively attending to faces enhanced face-selective activity much more evidently in the left hemisphere than in the right, whereas attending to objects suppressed the face-selective response in both hemispheres to a comparable extent. In addition, the time courses of attentional enhancement and suppression did not overlap. These results suggest the left and right hemispheres support face-selective processing in distinct ways—where the right hemisphere is mandatorily engaged by faces and the left hemisphere is more flexibly recruited to serve current tasks demands.

Auditory Attention Causes Gain Enhancement and Frequency Sharpening at Successive Stages of Cortical Processing-Evidence from Human Electroencephalography

Previous findings have suggested that auditory attention causes not only enhancement in neural processing gain, but also sharpening in neural frequency tuning in human auditory cortex. The current study was aimed to reexamine these findings. Specifically, we aimed to investigate whether attentional gain enhancement and frequency sharpening emerge at the same or different processing levels and whether they represent independent or cooperative effects. For that, we examined the pattern of attentional modulation effects on early, sensory-driven cortical auditory-evoked potentials occurring at different latencies. Attention was manipulated using a dichotic listening task and was thus not selectively directed to specific frequency values. Possible attention-related changes in frequency tuning selectivity were measured with an adaptation paradigm. Our results show marked disparities in attention effects between the earlier N1 deflection and the subsequent P2 deflection, with the N1 showing a strong gain enhancement effect, but no sharpening, and the P2 showing clear evidence of sharpening, but no independent gain effect. They suggest that gain enhancement and frequency sharpening represent successive stages of a cooperative attentional modulation mechanism that increases the representational bandwidth of attended versus unattended sounds.

The Functional Neuroanatomy of Human Face Perception

Face perception is critical for normal social functioning and is mediated by a network of regions in the ventral visual stream. In this review, we describe recent neuroimaging findings regarding the macro- and microscopic anatomical features of the ventral face network, the characteristics of white matter connections, and basic computations performed by population receptive fields within face-selective regions composing this network. We emphasize the importance of the neural tissue properties and white matter connections of each region, as these anatomical properties may be tightly linked to the functional characteristics of the ventral face network. We end by considering how empirical investigations of the neural architecture of the face network may inform the development of computational models and shed light on how computations in the face network enable efficient face perception.

Degradation of neural representations in higher visual cortex by sleep deprivation

A night of total sleep deprivation (TSD) impairs selective attention and is accompanied by attenuated activation within ventral visual cortex (VVC). However, finer details of how TSD compromises selectivity of visual processing remain unclear. Drawing from prior work in cognitive aging, we predicted that TSD would result in dedifferentiation of neural responses for faces and houses within the VVC. Instead, we found preservation of category selectivity. This was observed both in voxels highly selective for each category, and also across multiple voxels evaluated using MVPA. Based on prior findings of impaired attentional modulation following TSD, we also predicted reduced biasing of neural representations towards the attended category when participants viewed ambiguous face/house images. When participants were well rested, attention to houses (or faces) caused activation patterns to more closely resemble those elicited by isolated house (face) images than face (house) images. During TSD, attention to faces enhanced neural similarity to both target (face) and distractor (house) representations, signifying reduced suppression of irrelevant information. Degraded sensory processing reflected in reduced VVC activation following TSD, thus appears to be a result of impaired top-down modulation of sensory representations instead of degraded selectivity of maximally category sensitive voxels, or the dedifferentiation of neural activation patterns.

Influence of Temporal Expectations on Response Priming by Subliminal Faces

Unconscious processes are often assumed immune from attention influence. Recent behavioral studies suggest however that the processing of subliminal information can be influenced by temporal attention. To examine the neural mechanisms underlying these effects, we used a stringent masking paradigm together with fMRI to investigate how temporal attention modulates the processing of unseen (masked) faces. Participants performed a gender decision task on a visible neutral target face, preceded by a masked prime face that could vary in gender (same or different than target) and emotion expression (neutral or fearful). We manipulated temporal attention by instructing participants to expect targets to appear either early or late during the stimulus sequence. Orienting temporal attention to subliminal primes influenced response priming by masked faces, even when gender was incongruent. In addition, gender-congruent primes facilitated responses regardless of attention while gender-incongruent primes reduced accuracy when attended. Emotion produced no differential effects. At the neural level, incongruent and temporally unexpected primes increased brain response in regions of the fronto-parietal attention network, reflecting greater recruitment of executive control and reorienting processes. Congruent and expected primes produced higher activations in fusiform cortex, presumably reflecting facilitation of perceptual processing. These results indicate that temporal attention can influence subliminal processing of face features, and thus facilitate information integration according to task-relevance regardless of conscious awareness. They also suggest that task-congruent information between prime and target may facilitate response priming even when temporal attention is not selectively oriented to the prime onset time.

Repetition suppression and its contextual determinants in predictive coding

This paper presents a review of theoretical and empirical work on repetition suppression in the context of predictive coding. Predictive coding is a neurobiologically plausible scheme explaining how biological systems might perform perceptual inference and learning. From this perspective, repetition suppression is a manifestation of minimising prediction error through adaptive changes in predictions about the content and precision of sensory inputs. Simulations of artificial neural hierarchies provide a principled way of understanding how repetition suppression – at different time scales – can be explained in terms of inference and learning implemented under predictive coding. This formulation of repetition suppression is supported by results of numerous empirical studies of repetition suppression and its contextual determinants.

Attentional Enhancement of Auditory Mismatch Responses: a DCM/MEG Study

Despite similar behavioral effects, attention and expectation influence evoked responses differently: Attention typically enhances event-related responses, whereas expectation reduces them. This dissociation has been reconciled under predictive coding, where prediction errors are weighted by precision associated with attentional modulation. Here, we tested the predictive coding account of attention and expectation using magnetoencephalography and modeling. Temporal attention and sensory expectation were orthogonally manipulated in an auditory mismatch paradigm, revealing opposing effects on evoked response amplitude. Mismatch negativity (MMN) was enhanced by attention, speaking against its supposedly pre-attentive nature. This interaction effect was modeled in a canonical microcircuit using dynamic causal modeling, comparing models with modulation of extrinsic and intrinsic connectivity at different levels of the auditory hierarchy. While MMN was explained by recursive interplay of sensory predictions and prediction errors, attention was linked to the gain of inhibitory interneurons, consistent with its modulation of sensory precision.

Distinguishing attentional gain and tuning in young and older adults

Here we examined with functional magnetic resonance imaging (fMRI) whether advanced age affects 2 mechanisms of attention that are widely thought to enhance signal processing in the sensory neocortex: gain and tuning. Healthy young and older adults discriminated faces under varying levels of object competition while fMRI was acquired. In young adults, cortical response magnitude to attended faces was maintained despite increasing competition, consistent with gain. Cortical response selectivity, indexed from repetition suppression, also increased only for attended faces despite increasing competition, consistent with tuning. Older adults exhibited intact gain, but altered tuning, with extrastriate cortical tuning determined by object salience rather than attention. Moreover, the magnitude of this susceptibility to stimulus-driven processing was associated with a redistribution of attention-driven competitive processes to the frontal cortices. These data indicate that although both gain and tuning are modulated by increased perceptual competition, they are functionally dissociable in the extrastriate cortices, exhibit differential susceptibility to advanced aging, and spare the frontal cortices a considerable processing burden through early selection.

Haptic perception of users with low vision and their needs in haptic-incorporated user interfaces

PURPOSE

This paper reports on research aimed at advancing understanding haptic capability and needs of users with low vision. The objective is to apply this understanding to the design of haptic-incorporated user interfaces.

METHOD

Study 1 investigated the haptic perception between sighted participants and those with low vision through the magnitude estimation technique, and Study 2 explored the degree to which similar user interface needs were observed among the two vision groups.

RESULTS

Overall, our findings indicate there was no significant difference between the two vision groups in terms of haptic perception and user interface needs. A few differences in user interface preference did exist, however, and designers should take these into account.

CONCLUSIONS

Participants with low vision were a group who relied on their vision in everyday life instead of touch. Thus, their haptic capability was less likely to be enhanced via brain plasticity, which probably contributed to no significant difference in haptic-incorporated user interface needs.

IMPLICATIONS FOR REHABILITATION

No significant different haptic capability and haptic user interface (UI) needs exists between cited participants and those with low vision. UI designers should take into consideration that a certain range of magnitude/type of haptic feedback is available to accommodate preferences of both vision groups, which would ultimately increase the likelihood of successfully developing universal designs.

Attention sharpens the distinction between expected and unexpected percepts in the visual brain

Attention, the prioritization of goal-relevant stimuli, and expectation, the modulation of stimulus processing by probabilistic context, represent the two main endogenous determinants of visual cognition. Neural selectivity in visual cortex is enhanced for both attended and expected stimuli, but the functional relationship between these mechanisms is poorly understood. Here, we adjudicated between two current hypotheses of how attention relates to predictive processing, namely, that attention either enhances or filters out perceptual prediction errors (PEs), the PE-promotion model versus the PE-suppression model. We acquired fMRI data from category-selective visual regions while human subjects viewed expected and unexpected stimuli that were either attended or unattended. Then, we trained multivariate neural pattern classifiers to discriminate expected from unexpected stimuli, depending on whether these stimuli had been attended or unattended. If attention promotes PEs, then this should increase the disparity of neural patterns associated with expected and unexpected stimuli, thus enhancing the classifier's ability to distinguish between the two. In contrast, if attention suppresses PEs, then this should reduce the disparity between neural signals for expected and unexpected percepts, thus impairing classifier performance. We demonstrate that attention greatly enhances a neural pattern classifier's ability to discriminate between expected and unexpected stimuli in a region- and stimulus category-specific fashion. These findings are incompatible with the PE-suppression model, but they strongly support the PE-promotion model, whereby attention increases the precision of prediction errors. Our results clarify the relationship between attention and expectation, casting attention as a mechanism for accelerating online error correction in predicting task-relevant visual inputs.

Online decoding of object-based attention using real-time fMRI

Visual attention is used to selectively filter relevant information depending on current task demands and goals. Visual attention is called object-based attention when it is directed to coherent forms or objects in the visual field. This study used real-time functional magnetic resonance imaging for moment-to-moment decoding of attention to spatially overlapped objects belonging to two different object categories. First, a whole-brain classifier was trained on pictures of faces and places. Subjects then saw transparently overlapped pictures of a face and a place, and attended to only one of them while ignoring the other. The category of the attended object, face or place, was decoded on a scan-by-scan basis using the previously trained decoder. The decoder performed at 77.6% accuracy indicating that despite competing bottom-up sensory input, object-based visual attention biased neural patterns towards that of the attended object. Furthermore, a comparison between different classification approaches indicated that the representation of faces and places is distributed rather than focal. This implies that real-time decoding of object-based attention requires a multivariate decoding approach that can detect these distributed patterns of cortical activity.

Adaptation of cerebral oxygen metabolism and blood flow and modulation of neurovascular coupling with prolonged stimulation in human visual cortex

Prolonged visual stimulation results in neurophysiologic and hemodynamic adaptation. However, the hemodynamic adaptation appears to be small compared to neural adaptation. It is not clear how the cerebral metabolic rate of oxygen (CMRO2) is affected by adaptation. We measured cerebral blood flow (CBF) and CMRO2 change in responses to peripheral stimulation either continuously, or intermittently (on/off cycles). A linear system's response to the continuous input should be equal to the sum of the original response to the intermittent input and a version of that response shifted by half a cycle. The CMRO2 response showed a large non-linearity consistent with adaptation, the CBF response adapted to a lesser degree, and the blood oxygenation level dependent (BOLD) response was nearly linear. The metabolic response was coupled with a larger flow in the continuous condition than in the intermittent condition. Our results suggest that contrast adaptation improves energy economy of visual processing. However BOLD modulations may not accurately represent the underlying metabolic nonlinearity due to modulation of the coupling of blood flow and oxygen metabolism changes.

Spatial and temporal attention modulate the early stages of face processing: behavioural evidence from a reaching paradigm

A presently unresolved question within the face perception literature is whether attending to the location of a face modulates face processing (i.e. spatial attention). Opinions on this matter diverge along methodological lines – where neuroimaging studies have observed that the allocation of spatial attention serves to enhance the neural response to a face, findings from behavioural paradigms suggest face processing is carried out independently of spatial attention. In the present study, we reconcile this divide by using a continuous behavioural response measure that indexes face processing at a temporal resolution not available in discrete behavioural measures (e.g. button press). Using reaching trajectories as our response measure, we observed that although participants were able to process faces both when attended and unattended (as others have found), face processing was not impervious to attentional modulation. Attending to the face conferred clear benefits on sex-classification processes at less than 350ms of stimulus processing time. These findings constitute the first reliable demonstration of the modulatory effects of both spatial and temporal attention on face processing within a behavioural paradigm.

Complementary attentional components of successful memory encoding

Attention during encoding improves later memory, but how this happens is poorly understood. To investigate the role of attention in memory formation, we combined a variant of a spatial attention cuing task with a subsequent memory fMRI design. Scene stimuli were presented in the periphery to either the left or right of fixation, preceded by a central face cue whose gaze oriented attention to the probable location of the scene. We contrasted activity for scenes appearing in cued versus uncued locations to identify: (1) regions where cuing facilitated processing, and (2) regions involved in reorienting. We then tested how activity in these facilitation and reorienting regions of interest predicted subsequent long-term memory for individual scenes. In facilitation regions such as parahippocampal cortex, greater activity during encoding predicted memory success. In reorienting regions such as right temporoparietal junction, greater activity during encoding predicted memory failure. We interpret these results as evidence that memory formation benefits from attentional facilitation of perceptual processing combined with suppression of the ventral attention network to prevent reorienting to distractors.

Bilateral theta-burst TMS to influence global gestalt perception

While early and higher visual areas along the ventral visual pathway in the inferotemporal cortex are critical for the recognition of individual objects, the neural representation of human perception of complex global visual scenes remains under debate. Stroke patients with a selective deficit in the perception of a complex global Gestalt with intact recognition of individual objects – a deficit termed simultanagnosia – greatly helped to study this question. Interestingly, simultanagnosia typically results from bilateral lesions of the temporo-parietal junction (TPJ). The present study aimed to verify the relevance of this area for human global Gestalt perception. We applied continuous theta-burst TMS either unilaterally (left or right) or bilateral simultaneously over TPJ. Healthy subjects were presented with hierarchically organized visual stimuli that allowed parametrical degrading of the object at the global level. Identification of the global Gestalt was significantly modulated only for the bilateral TPJ stimulation condition. Our results strengthen the view that global Gestalt perception in the human brain involves TPJ and is co-dependent on both hemispheres.

Hemispheric asymmetry of visual scene processing in the human brain: evidence from repetition priming and intrinsic activity

Asymmetrical specialization of cognitive processes across the cerebral hemispheres is a hallmark of healthy brain development and an important evolutionary trait underlying higher cognition in humans. While previous research, including studies of priming, divided visual field presentation, and split-brain patients, demonstrates a general pattern of right/left asymmetry of form-specific versus form-abstract visual processing, little is known about brain organization underlying this dissociation. Here, using repetition priming of complex visual scenes and high-resolution functional magnetic resonance imaging (MRI), we demonstrate asymmetrical form specificity of visual processing between the right and left hemispheres within a region known to be critical for processing of visual spatial scenes (parahippocampal place area [PPA]). Next, we use resting-state functional connectivity MRI analyses to demonstrate that this functional asymmetry is associated with differential intrinsic activity correlations of the right versus left PPA with regions critically involved in perceptual versus conceptual processing, respectively. Our results demonstrate that the PPA comprises lateralized subregions across the cerebral hemispheres that are engaged in functionally dissociable yet complementary components of visual scene analysis. Furthermore, this functional asymmetry is associated with differential intrinsic functional connectivity of the PPA with distinct brain areas known to mediate dissociable cognitive processes.

Fidelity of neural reactivation reveals competition between memories

Remembering an event from the past is often complicated by the fact that our memories are cluttered with similar events. Though competition is a fundamental part of remembering, there is little evidence of how mnemonic competition is neurally represented. Here, we assessed whether competition between visual memories is captured in the relative degree to which target vs. competing memories are reactivated within the ventral occipitotemporal cortex (VOTC). To assess reactivation, we used multivoxel pattern analysis of fMRI data, quantifying the degree to which retrieval events elicited patterns of neural activity that matched those elicited during encoding. Consistent with recent evidence, we found that retrieval of visual memories was associated with robust VOTC reactivation and that the degree of reactivation scaled with behavioral expressions of target memory retrieval. Critically, competitive remembering was associated with more ambiguous patterns of VOTC reactivation, putatively reflecting simultaneous reactivation of target and competing memories. Indeed, the more weakly that target memories were reactivated, the more likely that competing memories were later remembered. Moreover, when VOTC reactivation indicated that conflict between target and competing memories was high, frontoparietal mechanisms were markedly engaged, revealing specific neural mechanisms that tracked competing mnemonic evidence. Together, these findings provide unique evidence that neural reactivation captures competition between individual memories, providing insight into how well target memories are retrieved in the present and how likely competing memories will be remembered in the future.

The emergence of perceived position in the visual system

Representing object position is one of the most critical functions of the visual system, but this task is not as simple as reading off an object's retinal coordinates. A rich body of literature has demonstrated that the position in which we perceive an object depends not only on retinotopy but also on factors such as attention, eye movements, object and scene motion, and frames of reference, to name a few. Despite the distinction between perceived and retinal position, strikingly little is known about how or where perceived position is represented in the brain. In the present study, we dissociated retinal and perceived object position to test the relative precision of retina-centered versus percept-centered position coding in a number of independently defined visual areas. In an fMRI experiment, subjects performed a five-alternative forced-choice position discrimination task; our analysis focused on the trials in which subjects misperceived the positions of the stimuli. Using a multivariate pattern analysis to track the coupling of the BOLD response with incremental changes in physical and perceived position, we found that activity in higher level areas--middle temporal complex, fusiform face area, parahippocampal place area, lateral occipital cortex, and posterior fusiform gyrus--more precisely reflected the reported positions than the physical positions of the stimuli. In early visual areas, this preferential coding of perceived position was absent or reversed. Our results demonstrate a new kind of spatial topography present in higher level visual areas in which an object's position is encoded according to its perceived rather than retinal location. We term such percept-centered encoding “perceptotopy".

Failing to ignore: paradoxical neural effects of perceptual load on early attentional selection in normal aging

We examined visual selective attention under perceptual load--simultaneous presentation of task-relevant and -irrelevant information--in healthy young and older adult human participants to determine whether age differences are observable at early stages of selection in the visual cortices. Participants viewed 50/50 superimposed face/place images and judged whether the faces were male or female, rendering places perceptible but task-irrelevant. Each stimulus was repeated, allowing us to index dynamic stimulus-driven competition from places. Consistent with intact early selection in young adults, we observed no adaptation to unattended places in parahippocampal place area (PPA) and significant adaptation to attended faces in fusiform face area (FFA). Older adults, however, exhibited both PPA adaptation to places and weak FFA adaptation to faces. We also probed participants' associative recognition for face-place pairs post-task. Older adults with better place recognition memory scores were found to exhibit both the largest magnitudes of PPA adaptation and the smallest magnitudes of FFA adaptation on the attention task. In a control study, we removed the competing perceptual information to decrease perceptual load. These data revealed that the initial age-related impairments in selective attention were not due to a general decline in visual cortical selectivity; both young and older adults exhibited robust FFA adaptation and neither group exhibited PPA adaptation to repeated faces. Accordingly, distracting information does not merely interfere with attended input in older adults, but is co-encoded along with the contents of attended input, to the extent that this information can subsequently be recovered from recognition memory.

Refreshing and integrating visual scenes in scene-selective cortex

Constructing a rich and coherent visual experience involves maintaining visual information that is not perceptually available in the current view. Recent studies suggest that briefly thinking about a stimulus (refreshing) can modulate activity in category-specific visual areas. Here, we tested the nature of such perceptually refreshed representations in the parahippocampal place area (PPA) and retrosplenial cortex (RSC) using fMRI. We asked whether a refreshed representation is specific to a restricted view of a scene, or more view-invariant. Participants saw a panoramic scene and were asked to think back to (refresh) a part of the scene after it disappeared. In some trials, the refresh cue appeared twice on the same side (e.g., refresh left-refresh left), and other trials, the refresh cue appeared on different sides (e.g., refresh left-refresh right). A control condition presented halves of the scene twice on same sides (e.g., perceive left-perceive left) or different sides (e.g., perceive left-perceive right). When scenes were physically repeated, both the PPA and RSC showed greater activation for the different-side repetition than the same-side repetition, suggesting view-specific representations. When participants refreshed scenes, the PPA showed view-specific activity just as in the physical repeat conditions, whereas RSC showed an equal amount of activation for different- and same-side conditions. This finding suggests that in RSC, refreshed representations were not restricted to a specific view of a scene, but extended beyond the target half into the entire scene. Thus, RSC activity associated with refreshing may provide a mechanism for integrating multiple views in the mind.

Temporal and spatial integration of face, object, and scene features in occipito-temporal cortex

In three neuroimaging experiments, face, novel object, and building stimuli were compared under conditions of restricted (aperture) viewing and normal (whole) viewing. Aperture viewing restricted the view to a single face/object feature at a time, with the subjects able to move the aperture continuously though time to reveal different features. An analysis of the proportion of time spent viewing different features showed stereotypical exploration patterns for face, object, and building stimuli, and suggested that subjects constrained their viewing to the features most relevant for recognition. Aperture viewing showed much longer response times than whole viewing, due to sequential exploration of the relevant isolated features. An analysis of BOLD activation revealed face-selective activation with both whole viewing and aperture viewing in the left and right fusiform face areas (FFA). Aperture viewing showed strong and sustained activation throughout exploration, suggesting that aperture viewing recruited similar processes as whole viewing, but for a longer time period. Face-selective recruitment of the FFA with aperture viewing suggests that the FFA is involved in the integration of isolated features for the purpose of recognition.

Culture differences in neural processing of faces and houses in the ventral visual cortex

Behavioral and eye-tracking studies on cultural differences have found that while Westerners have a bias for analytic processing and attend more to face features, East Asians are more holistic and attend more to contextual scenes. In this neuroimaging study, we hypothesized that these culturally different visual processing styles would be associated with cultural differences in the selective activity of the fusiform regions for faces, and the parahippocampal and lingual regions for contextual stimuli. East Asians and Westerners passively viewed face and house stimuli during an functional magnetic resonance imaging experiment. As expected, we observed more selectivity for faces in Westerners in the left fusiform face area (FFA) reflecting a more analytic processing style. Additionally, Westerners showed bilateral activity to faces in the FFA whereas East Asians showed more right lateralization. In contrast, no cultural differences were detected in the parahippocampal place area (PPA), although there was a trend for East Asians to show greater house selectivity than Westerners in the lingual landmark area, consistent with more holistic processing in East Asians. These findings demonstrate group biases in Westerners and East Asians that operate on perceptual processing in the brain and are consistent with previous eye-tracking data that show cultural biases to faces.

Nonpreferred stimuli modify the representation of faces in the fusiform face area

The ventral visual cortex has a modular organization in which discrete and well-defined regions show a much stronger response to certain object categories (e.g., faces, bodies) than to other categories. The majority of previous studies have examined the response of these category-selective regions to isolated images of preferred or nonpreferred categories. Thus, little is known about the way these category-selective regions represent more complex visual stimuli, which include both preferred and nonpreferred stimuli. Here we examined whether glasses (nonpreferred) modify the representation of simultaneously presented faces (preferred) in the fusiform face area. We used an event-related fMR-adaptation paradigm in which faces were presented with glasses either on or above the face while subjects performed a face or a glasses discrimination task. Our findings show that the sensitivity of the fusiform face area to glasses was maximal when glasses were presented on the face than above the face during a face discrimination task rather than during a glasses discrimination task. These findings suggest that nonpreferred stimuli may significantly modify the representation of preferred stimuli, even when they are task irrelevant. Future studies will determine whether this interaction is specific to faces or may be found for other object categories in category-selective areas.

High-resolution fMRI reveals match enhancement and attentional modulation in the human medial temporal lobe

A primary function of the medial temporal lobe (MTL) is to signal prior encounter with behaviorally relevant stimuli. MTL match enhancement--increased activation when viewing previously encountered stimuli--has been observed for goal-relevant stimuli in nonhuman primates during delayed-match-to-sample tasks and in humans during more complex relational memory tasks. Match enhancement may alternatively reflect (a) an attentional response to familiar relative to novel stimuli or (b) the retrieval of contextual details surrounding the past encounter with familiar stimuli. To gain leverage on the functional significance of match enhancement in the hippocampus, high-resolution fMRI of human MTL was conducted while participants attended, ignored, or passively viewed face and scene stimuli in the context of a modified delayed-match-to-sample task. On each "attended" trial, two goal-relevant stimuli were encountered before a probe that either matched or mismatched one of the attended stimuli, enabling examination of the consequences of encountering one of the goal-relevant stimuli as a match probe on later memory for the other (nonprobed) goal-relevant stimulus. fMRI revealed that the hippocampus was insensitive to the attentional manipulation, whereas parahippocampal cortex was modulated by scene-directed attention, and perirhinal cortex showed more subtle and general effects of attention. By contrast, all hippocampal subfields demonstrated match enhancement to the probe, and a postscan test revealed more accurate recognition memory for the nonprobed goal-relevant stimulus on match relative to mismatch trials. These data suggest that match enhancement in human hippocampus reflects retrieval of other goal-relevant contextual details surrounding a stimulus's prior encounter.

Reduced neural selectivity increases fMRI adaptation with age during face discrimination

Ventral-visual activity in older adults has been characterized by dedifferentiation, or reduced distinctiveness, of responses to different categories of visual stimuli such as faces and houses, that typically elicit highly specialized responses in the fusiform and parahippocampal brain regions respectively in young adults (Park et al., 2004). In the present study, we demonstrate that age-related neural dedifferentiation applies to within-category stimuli (different types of faces) as well, such that older adults process less distinctive representations for individual faces than young adults. We performed a functional magnetic resonance imaging adaptation experiment while young and older participants made same-different judgments to serially presented face-pairs that were Identical, Moderate in similarity through morphing, or Different. As expected, older adults showed adaptation in the fusiform face area (FFA), during the Identical as well as the Moderate conditions relative to the Different condition. Young adults showed adaptation during the Identical condition, but minimal adaptation to the Moderate condition. These results indicate that older adults' FFA treated the morphed faces as Identical faces, reflecting decreased fidelity of neural representation of faces with age.

Cultural differences in the lateral occipital complex while viewing incongruent scenes

Converging behavioral and neuroimaging evidence indicates that culture influences the processing of complex visual scenes. Whereas Westerners focus on central objects and tend to ignore context, East Asians process scenes more holistically, attending to the context in which objects are embedded. We investigated cultural differences in contextual processing by manipulating the congruence of visual scenes presented in an fMR-adaptation paradigm. We hypothesized that East Asians would show greater adaptation to incongruent scenes, consistent with their tendency to process contextual relationships more extensively than Westerners. Sixteen Americans and 16 native Chinese were scanned while viewing sets of pictures consisting of a focal object superimposed upon a background scene. In half of the pictures objects were paired with congruent backgrounds, and in the other half objects were paired with incongruent backgrounds. We found that within both the right and left lateral occipital complexes, Chinese participants showed significantly greater adaptation to incongruent scenes than to congruent scenes relative to American participants. These results suggest that Chinese were more sensitive to contextual incongruity than were Americans and that they reacted to incongruent object/background pairings by focusing greater attention on the object.

Culture modulates eye-movements to visual novelty

BACKGROUND

When viewing complex scenes, East Asians attend more to contexts whereas Westerners attend more to objects, reflecting cultural differences in holistic and analytic visual processing styles respectively. This eye-tracking study investigated more specific mechanisms and the robustness of these cultural biases in visual processing when salient changes in the objects and backgrounds occur in complex pictures.

METHODOLOGY/PRINCIPAL FINDINGS

Chinese Singaporean (East Asian) and Caucasian US (Western) participants passively viewed pictures containing selectively changing objects and background scenes that strongly captured participants' attention in a data-driven manner. We found that although participants from both groups responded to object changes in the pictures, there was still evidence for cultural divergence in eye-movements. The number of object fixations in the US participants was more affected by object change than in the Singapore participants. Additionally, despite the picture manipulations, US participants consistently maintained longer durations for both object and background fixations, with eye-movements that generally remained within the focal objects. In contrast, Singapore participants had shorter fixation durations with eye-movements that alternated more between objects and backgrounds.

CONCLUSIONS/SIGNIFICANCE

The results demonstrate a robust cultural bias in visual processing even when external stimuli draw attention in an opposite manner to the cultural bias. These findings also extend previous studies by revealing more specific, but consistent, effects of culture on the different aspects of visual attention as measured by fixation duration, number of fixations, and saccades between objects and backgrounds.

Evaluating functional localizers: the case of the FFA

Functional localizers are routinely used in neuroimaging studies to test hypotheses about the function of specific brain areas. The specific tasks and stimuli used to localize particular regions vary widely from study to study even when the same cortical region is targeted. Thus, it is important to ask whether task and stimulus changes lead to differences in localization or whether localization procedures are largely immune to differences in tasks and contrasting stimuli. We present two experiments and a literature review that explore whether face localizer tasks yield differential localization in the fusiform gyrus as a function of task and contrasting stimuli. We tested standard localization tasks-passive viewing, 1-back, and 2-back memory tests--and did not find differences in localization based on task. We did, however, find differences in the extent, strength and patterns/reliabilities of the activation in the fusiform gyrus based on comparison stimuli (faces vs. houses compared to faces vs. scrambled stimuli).

Natural scene categories revealed in distributed patterns of activity in the human brain

Human subjects are extremely efficient at categorizing natural scenes, despite the fact that different classes of natural scenes often share similar image statistics. Thus far, however, it is unknown where and how complex natural scene categories are encoded and discriminated in the brain. We used functional magnetic resonance imaging (fMRI) and distributed pattern analysis to ask what regions of the brain can differentiate natural scene categories (such as forests vs mountains vs beaches). Using completely different exemplars of six natural scene categories for training and testing ensured that the classification algorithm was learning patterns associated with the category in general and not specific exemplars. We found that area V1, the parahippocampal place area (PPA), retrosplenial cortex (RSC), and lateral occipital complex (LOC) all contain information that distinguishes among natural scene categories. More importantly, correlations with human behavioral experiments suggest that the information present in the PPA, RSC, and LOC is likely to contribute to natural scene categorization by humans. Specifically, error patterns of predictions based on fMRI signals in these areas were significantly correlated with the behavioral errors of the subjects. Furthermore, both behavioral categorization performance and predictions from PPA exhibited a significant decrease in accuracy when scenes were presented up-down inverted. Together these results suggest that a network of regions, including the PPA, RSC, and LOC, contribute to the human ability to categorize natural scenes.

Category expectation modulates baseline and stimulus-evoked activity in human inferotemporal cortex

Expectation of locations and low-level features increases activity in extrastriate visual areas even in the absence of a stimulus, but it is unclear whether or how expectation of higher-level stimulus properties affects visual responses. Here, we used event-related functional magnetic resonance imaging (fMRI) to test whether category expectation affects baseline and stimulus-evoked activity in higher-level, category-selective inferotemporal (IT) visual areas. Word cues indicating an image category (FACE or HOUSE) were followed by a delay, then a briefly presented image of a face or a house. On most trials, the cue correctly predicted the upcoming stimulus. Baseline activity in regions within the fusiform face area (FFA) and parahippocampal place area (PPA) was modulated such that activity was higher during expectation of the preferred (e.g., FACE for FFA) vs. non-preferred category. Stimulus-evoked responses reflected an initial bias (higher overall activity) followed by increased selectivity (greater difference between activity to a preferred vs. non-preferred stimulus) after expectation of the preferred vs. non-preferred category. Consistent with the putative role of a frontoparietal network in top-down modulation of activity in sensory cortex, expectation-related activity in several frontal and parietal areas correlated with the magnitude of baseline shifts in the FFA and PPA across subjects. Furthermore, expectation-related activity in lateral prefrontal cortex also correlated with the magnitude of expectation-based increases in stimulus selectivity in IT areas. These findings demonstrate that category expectation influences both baseline and stimulus-evoked activity in category-selective inferotemporal visual areas, and that these modulations may be driven by a frontoparietal attentional control network.

Defining the face processing network: optimization of the functional localizer in fMRI

Functional localizers that contrast brain signal when viewing faces versus objects are commonly used in functional magnetic resonance imaging studies of face processing. However, current protocols do not reliably show all regions of the core system for face processing in all subjects when conservative statistical thresholds are used, which is problematic in the study of single subjects. Furthermore, arbitrary variations in the applied thresholds are associated with inconsistent estimates of the size of face-selective regions-of-interest (ROIs). We hypothesized that the use of more natural dynamic facial images in localizers might increase the likelihood of identifying face-selective ROIs in individual subjects, and we also investigated the use of a method to derive the statistically optimal ROI cluster size independent of thresholds. We found that dynamic facial stimuli were more effective than static stimuli, identifying 98% (versus 72% for static) of ROIs in the core face processing system and 69% (versus 39% for static) of ROIs in the extended face processing system. We then determined for each core face processing ROI, the cluster size associated with maximum statistical face-selectivity, which on average was approximately 50 mm(3) for the fusiform face area, the occipital face area, and the posterior superior temporal sulcus. We suggest that the combination of (a) more robust face-related activity induced by a dynamic face localizer and (b) a cluster-size determination based on maximum face-selectivity increases both the sensitivity and the specificity of the characterization of face-related ROIs in individual subjects.

Differential Influences of Emotion, Task, and Novelty on Brain Regions Underlying the Processing of Speech Melody

We investigated the functional characteristics of brain regions implicated in processing of speech melody by presenting words spoken in either neutral or angry prosody during a functional magnetic resonance imaging experiment using a factorial habituation design. Subjects judged either affective prosody or word class for these vocal stimuli, which could be heard for either the first, second, or third time. Voice-sensitive temporal cortices, as well as the amygdala, insula, and mediodorsal thalami, reacted stronger to angry than to neutral prosody. These stimulus-driven effects were not influenced by the task, suggesting that these brain structures are automatically engaged during processing of emotional information in the voice and operate relatively independent of cognitive demands. By contrast, the right middle temporal gyrus and the bilateral orbito-frontal cortices (OFC) responded stronger during emotion than word classification, but were also sensitive to anger expressed by the voices, suggesting that some perceptual aspects of prosody are also encoded within these regions subserving explicit processing of vocal emotion. The bilateral OFC showed a selective modulation by emotion and repetition, with particularly pronounced responses to angry prosody during the first presentation only, indicating a critical role of the OFC in detection of vocal information that is both novel and behaviorally relevant. These results converge with previous findings obtained for angry faces and suggest a general involvement of the OFC for recognition of anger irrespective of the sensory modality. Taken together, our study reveals that different aspects of voice stimuli and perceptual demands modulate distinct areas involved in the processing of emotional prosody.