Unconscious associative memory affects visual processing before 100 ms

When experience with scenes foils attentional orienting: ERP evidence against flexible target-context mapping in visual search

Visual search is speeded when a target is repeatedly presented in an invariant scene context of nontargets (contextual cueing), demonstrating observers' capability for using statistical long-term memory (LTM) to make predictions about upcoming sensory events, thus improving attentional orienting. In the current study, we investigated whether expectations arising from individual, learned environmental structures can encompass multiple target locations. We recorded event-related potentials (ERPs) while participants performed a contextual cueing search task with repeated and non-repeated spatial item configurations. Notably, a given search display could be associated with either a single target location (standard contextual cueing) or two possible target locations. Our result showed that LTM-guided attention was always limited to only one target position in single- but also in the dual-target displays, as evidenced by expedited reaction times (RTs) and enhanced N1pc and N2pc deflections contralateral to one ("dominant") target of up to two repeating target locations. This contrasts with the processing of non-learned ("minor") target positions (in dual-target displays), which revealed slowed RTs alongside an initial N1pc "misguidance" signal that then vanished in the subsequent N2pc. This RT slowing was accompanied by enhanced N200 and N400 waveforms over fronto-central electrodes, suggesting that control mechanisms regulate the competition between dominant and minor targets. Our study thus reveals a dissociation in processing dominant versus minor targets: While LTM templates guide attention to dominant targets, minor targets necessitate control processes to overcome the automatic bias towards previously learned, dominant target locations.

Multisensory Rather than Unisensory Representations Contribute to Statistical Context Learning in Tactile Search

Using a combination of behavioral and EEG measures in a tactile odd-one-out search task with collocated visual items, we investigated the mechanisms underlying facilitation of search by repeated (vs. nonrepeated) spatial distractor-target configurations ("contextual cueing") when either the tactile (same-modality) or the visual array (different-modality) context was predictive of the location of the tactile singleton target. Importantly, in both conditions, the stimulation was multisensory, consisting of tactile plus visual items, although the target was singled out in the tactile modality and so the visual items were task-irrelevant. We found that when the predictive context was tactile, facilitation of search RTs by repeated configurations was accompanied by, and correlated with, enhanced lateralized ERP markers of pre-attentive (N1, N2) and, respectively focal-attentional processing (contralateral delay activity) not only over central ("somatosensory"), but also posterior ("visual") electrode sites, although the ERP effects were less marked over visual cortex. A similar pattern-of facilitated RTs and enhanced lateralized (N2 and contralateral delay activity) ERP components-was found when the predictive context was visual, although the ERP effects were less marked over somatosensory cortex. These findings indicate that both somatosensory and visual cortical regions contribute to the more efficient processing of the tactile target in repeated stimulus arrays, although their involvement is differentially weighted depending on the sensory modality that contains the predictive information.

Socially induced negative affective knowledge modulates early face perception but not gaze cueing of attention

Prior affective and social knowledge about other individuals has been shown to modulate perception of their faces and gaze‐related attentional processes. However, it remains unclear whether emotionally charged knowledge acquired through interactive social learning also modulates face processing and attentional control. Thus, the aim of this study was to test whether affective knowledge induced through social interactions in a naturalistic exchange game can influence early stages of face processing and attentional shifts in a subsequent gaze‐cueing task. As indicated by self‐reported ratings, the game was successful in inducing valenced affective knowledge towards positive and negative players. In the subsequent task, in which the locations of future targets were cued by the gaze of the game players, we observed enhanced early neural activity (larger amplitude of the P1 component) in response to a photograph of the negative player. This indicates that negative affective knowledge about an individual indeed modulates very early stages of the processing of this individual's face. Our study contributes to the existing literature by providing further evidence for the saliency of interactive social exchange paradigms that are used to induce affective knowledge. Moreover, it extends the previous research by presenting a very early modulation of perception by socially learned affective knowledge. Importantly, it also offers increased ecological validity of the findings due to the use of naturalistic social exchange in the study design.

This research complements previous evidence that experimentally induced socio‐affective knowledge about other individuals, modulates processing of their faces, and shows that negative (but not positive) affect enhances very early face processing (the P1). Importantly, we provide an effective affect induction tool—an interactive social exchange game—which offers increased social ecological validity in experimental settings.

Multisensory visuo-tactile context learning enhances the guidance of unisensory visual search

Does multisensory distractor-target context learning enhance visual search over and above unisensory learning? To address this, we had participants perform a visual search task under both uni- and multisensory conditions. Search arrays consisted of one Gabor target that differed from three homogeneous distractors in orientation; participants had to discriminate the target's orientation. In the multisensory session, additional tactile (vibration-pattern) stimulation was delivered to two fingers of each hand, with the odd-one-out tactile target and the distractors co-located with the corresponding visual items in half the trials; the other half presented the visual array only. In both sessions, the visual target was embedded within identical (repeated) spatial arrangements of distractors in half of the trials. The results revealed faster response times to targets in repeated versus non-repeated arrays, evidencing 'contextual cueing'. This effect was enhanced in the multisensory session-importantly, even when the visual arrays presented without concurrent tactile stimulation. Drift-diffusion modeling confirmed that contextual cueing increased the rate at which task-relevant information was accumulated, as well as decreasing the amount of evidence required for a response decision. Importantly, multisensory learning selectively enhanced the evidence-accumulation rate, expediting target detection even when the context memories were triggered by visual stimuli alone.

Why Are Acquired Search-Guiding Context Memories Resistant to Updating?

Looking for goal-relevant objects in our various environments is one of the most ubiquitous tasks the human visual system has to accomplish (Wolfe, 1998). Visual search is guided by a number of separable selective-attention mechanisms that can be categorized as bottom-up driven – guidance by salient physical properties of the current stimuli – or top-down controlled – guidance by observers' “online” knowledge of search-critical object properties (e.g., Liesefeld and Müller, 2019). In addition, observers' expectations based on past experience also play also a significant role in goal-directed visual selection. Because sensory environments are typically stable, it is beneficial for the visual system to extract and learn the environmental regularities that are predictive of (the location of) the target stimulus. This perspective article is concerned with one of these predictive mechanisms: statistical context learning of consistent spatial patterns of target and distractor items in visual search. We review recent studies on context learning and its adaptability to incorporate consistent changes, with the aim to provide new directions to the study of processes involved in the acquisition of search-guiding context memories and their adaptation to consistent contextual changes – from a three-pronged, psychological, computational, and neurobiological perspective.

Contextual Cueing Effect Under Rapid Presentation

In contextual cueing, previously encountered context tends to facilitate the detection of the target embedded in it than when the target appears in a novel context. In this study, we investigated whether the contextual cueing could develop at early time when the search display was presented briefly. In four experiments, participants searched for a target T in an array of distractor Ls. The results showed that with a rather short presentation time of the search display, participants were able to learn the spatial context and speeded up their response time overall, with the learning effect lasting for a long period. Specifically, the contextual cueing effect was observed either with or without a mask after a duration of 300-ms presentation of the search display. Such a context learning under rapid presentation could not operate only with the local context information repeated, thus suggesting that a global context was required to guide spatial attention when the viewing time of the search display was limited. Overall, these findings indicate that contextual cueing might arise at an “early,” target selection stage and that the global context is necessary for the context learning under rapid presentation to function.

Automatic Guidance (and Misguidance) of Visuospatial Attention by Acquired Scene Memory: Evidence From an N1pc Polarity Reversal

Visual search is facilitated when the target is repeatedly encountered at a fixed position within an invariant (vs. randomly variable) distractor layout—that is, when the layout is learned and guides attention to the target, a phenomenon known as contextual cuing. Subsequently changing the target location within a learned layout abolishes contextual cuing, which is difficult to relearn. Here, we used lateralized event-related electroencephalogram (EEG) potentials to explore memory-based attentional guidance (N = 16). The results revealed reliable contextual cuing during initial learning and an associated EEG-amplitude increase for repeated layouts in attention-related components, starting with an early posterior negativity (N1pc, 80–180 ms). When the target was relocated to the opposite hemifield following learning, contextual cuing was effectively abolished, and the N1pc was reversed in polarity (indicative of persistent misguidance of attention to the original target location). Thus, once learned, repeated layouts trigger attentional-priority signals from memory that proactively interfere with contextual relearning after target relocation.

Prediction and memory: A predictive coding account

The hippocampus is crucial for episodic memory, but it is also involved in online prediction. Evidence suggests that a unitary hippocampal code underlies both episodic memory and predictive processing, yet within a predictive coding framework the hippocampal-neocortical interactions that accompany these two phenomena are distinct and opposing. Namely, during episodic recall, the hippocampus is thought to exert an excitatory influence on the neocortex, to reinstate activity patterns across cortical circuits. This contrasts with empirical and theoretical work on predictive processing, where descending predictions suppress prediction errors to 'explain away' ascending inputs via cortical inhibition. In this hypothesis piece, we attempt to dissolve this previously overlooked dialectic. We consider how the hippocampus may facilitate both prediction and memory, respectively, by inhibiting neocortical prediction errors or increasing their gain. We propose that these distinct processing modes depend upon the neuromodulatory gain (or precision) ascribed to prediction error units. Within this framework, memory recall is cast as arising from fictive prediction errors that furnish training signals to optimise generative models of the world, in the absence of sensory data.

Prior Expectations of Motion Direction Modulate Early Sensory Processing

Perception is a process of inference, integrating sensory inputs with prior expectations. However, little is known regarding the temporal dynamics of this integration. It has been proposed that expectation plays a role early in the perceptual process, biasing sensory processing. Alternatively, others suggest that expectations are integrated only at later, postperceptual decision-making stages. The current study aimed to dissociate between these hypotheses. We exposed human participants (male and female) to auditory cues predicting the likely direction of upcoming moving dot patterns, while recording neural activity using magnetoencephalography (MEG). Participants' reports of the moving dot directions were biased toward the direction predicted by the cues. To investigate when expectations affected sensory representations, we used inverted encoding models to decode the direction represented in early sensory signals. Strikingly, the cues modulated the direction represented in the MEG signal as early as 150 ms after visual stimulus onset. While this may not reflect a modulation of the initial feedforward sweep, it does reveal a modulation of early sensory representations. Exploratory analyses showed that the neural modulation was related to perceptual expectation effects: participants with a stronger perceptual bias toward the predicted direction also revealed a stronger reflection of the predicted direction in the MEG signal. For participants with this perceptual bias, a correlation between decoded and perceived direction already emerged before visual stimulus onset, suggesting that the prestimulus state of the visual cortex influences sensory processing. Together, these results suggest that expectations play an integral role in the neural computations underlying perception.

SIGNIFICANCE STATEMENT Perception can be thought of as an inferential process in which our brains integrate sensory inputs with prior expectations to make sense of the world. This study investigated whether this integration occurs early or late in the process of perception. We exposed human participants to auditory cues that predicted the likely direction of visual moving dots, while recording neural activity with millisecond resolution using magnetoencephalography. Participants' perceptual reports of the direction of the moving dots were biased toward the predicted direction. Additionally, the predicted direction modulated the neural representation of the moving dots just 150 ms after they appeared. This suggests that prior expectations affected sensory processing at early stages, playing an integral role in the perceptual process.

Distributed attention beats the down-side of statistical context learning in visual search

Spatial attention can be deployed with a narrower focus to process individual items or distributed relatively broadly to process larger parts of a scene. This study investigated how focused- versus distributed-attention modes contribute to the adaptation of context-based memories that guide visual search. In two experiments, participants were either required to fixate the screen center and use peripheral vision for search ("distributed attention"), or they could freely move their eyes, enabling serial scanning of the search array ("focused attention"). Both experiments consisted of an initial learning phase and a subsequent test phase. During learning, participants searched for targets presented either among repeated (invariant) or nonrepeated (randomly generated) spatial layouts of distractor items. Prior research showed that repeated encounters of invariant display arrangements lead to long-term context memory about these arrays, which can then come to guide search (contextual-cueing effect). The crucial manipulation in the test phase was a change of the target location within an otherwise constant distractor layout, which has previously been shown to abolish the cueing effect. The current results replicated these findings, although importantly only when attention was focused. By contrast, with distributed attention, the cueing effect recovered rapidly and attained a level comparable to the initial effect (before the target location change). This indicates that contextual cueing can adapt more easily when attention is distributed, likely because a broad attentional set facilitates the flexible updating of global (distractor-distractor), as compared to more local (distractor-target), context representations-allowing local changes to be incorporated more readily.

Contextual cueing facilitation arises early in the time course of visual search: An investigation with the `speed-accuracy tradeoff task

Visual search is expedited in a spatial context encountered repeatedly. A much-debated question is how early the facilitation by contextual memory arises. The current study examined the possibility of the facilitation of early attentional processing by constructing a descriptive model of the time course of visual search and its facilitation by contextual cueing. Participants in this experiment engaged in a speed-accuracy tradeoff (SAT) task after learning the spatial contexts in a standard visual search task in which they searched for a rotated T target among Ls. In the SAT task, they were required to search for the target and respond immediately when a sound probe was presented, even if they did not find or identify the target when the inter-stimulus interval between the search display and the probe varied from 40 ms to 2,000 ms. Participants completed two blocks of the SAT task, in which they searched in learned and new contexts. The results of the SAT procedure showed that responses were more accurate in repeated contexts than in new contexts, even when only a brief period of time elapsed after presenting the search display (> 90 ms). We conducted an analysis of the time course of contextual-cueing effects with Bayesian hierarchical modeling, which demonstrated that the rate of increase in accuracy was higher for the repeated than for the new contexts. These findings suggest that early attentional processing is enhanced by learning the contexts, and this enhancement arises very early in the time course of the visual search.

Crossmodal learning of target-context associations: When would tactile context predict visual search?

It is well established that statistical learning of visual target locations in relation to constantly positioned visual distractors facilitates visual search. In the present study, we investigated whether such a contextual-cueing effect would also work crossmodally, from touch onto vision. Participants responded to the orientation of a visual target singleton presented among seven homogenous visual distractors. Four tactile stimuli, two to different fingers of each hand, were presented either simultaneously with or prior to the visual stimuli. The identity of the stimulated fingers provided the crossmodal context cue: in half of the trials, a given visual target location was consistently paired with a given tactile configuration. The visual stimuli were presented above the unseen fingers, ensuring spatial correspondence between vision and touch. We found no evidence of crossmodal contextual cueing when the two sets of items (tactile, visual) were presented simultaneously (Experiment 1). However, a reliable crossmodal effect emerged when the tactile distractors preceded the onset of visual stimuli 700 ms (Experiment 2). But crossmodal cueing disappeared again when, after an initial learning phase, participants flipped their hands, making the tactile distractors appear at different positions in external space while their somatotopic positions remained unchanged (Experiment 3). In all experiments, participants were unable to explicitly discriminate learned from novel multisensory arrays. These findings indicate that search-facilitating context memory can be established across vision and touch. However, in order to guide visual search, the (predictive) tactile configurations must be remapped from their initial somatotopic into a common external representational format.

Predictive coding in a multisensory path integration task: An fMRI study

During self-motion through an environment, our sensory systems are confronted with a constant flow of information from different modalities. To successfully navigate, self-induced sensory signals have to be dissociated from externally induced sensory signals. Previous studies have suggested that the processing of self-induced sensory information is modulated by means of predictive coding mechanisms. However, the neural correlates of processing self-induced sensory information from different modalities during self-motion are largely unknown. Here, we asked if and how the processing of visually simulated self-motion and/or associated auditory stimuli is modulated by self-controlled action. Participants were asked to actively reproduce a previously observed simulated self-displacement (path integration). Blood oxygen level-dependent (BOLD) activation during this path integration was compared with BOLD activation during a condition in which we passively replayed the exact sensory stimulus that had been produced by the participants in previous trials. We found supramodal BOLD suppression in parietal and frontal regions. Remarkably, BOLD contrast in sensory areas was enhanced in a modality-specific manner. We conclude that the effect of action on sensory processing is strictly dependent on the respective behavioral task and its relevance.

Cognitive penetration of early vision in face perception

Cognitive and affective penetration of perception refers to the influence that higher mental states such as beliefs and emotions have on perceptual systems. Psychological and neuroscientific studies appear to show that these states modulate the visual system at the visuomotor, attentional, and late levels of processing. However, empirical evidence showing that similar consequences occur in early stages of visual processing seems to be scarce. In this paper, I argue that psychological evidence does not seem to be either sufficient or necessary to argue in favour of or against the cognitive penetration of perception in either late or early vision. In order to do that we need to have recourse to brain imaging techniques. Thus, I introduce a neuroscientific study and argue that it seems to provide well-grounded evidence for the cognitive penetration of early vision in face perception. I also examine and reject alternative explanations to my conclusion.

Attention is shaped by semantic level of event-structure during speech comprehension: an electroencephalogram study

The present electroencephalogram study used an attention probe paradigm to investigate how semantic and acoustic structures constrain temporal attention during speech comprehension. Spoken sentences were used as stimuli, with each one containing a four-character critical phrase, of which the third character was the target character. We manipulated not only the semantic relationship between the target character and the immediately preceding two characters, but also the presence/absence of a pitch accent on the first character. In addition, an attention probe was either presented concurrently with the target character or not. The results showed that the N1 effect evoked by the attention probe was of larger amplitude and started earlier (enhanced attention) when the target character and the preceding two characters belonged to the same semantic event than when they spanned a semantic-event boundary, and this effect occurred only in the un-accented conditions. The results suggest that, during speech comprehension, the semantic level of event-structure can constrain attention allocation along the temporal dimension, and reverse the attention attenuation effect of prediction; meanwhile, the semantic and acoustic levels of event-structure interact with each other immediately to modulate auditory-temporal attention. The results were discussed with regard to the predictive coding account of attention.

Pre-cueing, the Epistemic Role of Early Vision, and the Cognitive Impenetrability of Early Vision

I have argued (Raftopoulos, 2009, 2014) that early vision is not directly affected by cognition since its processes do not draw on cognition as an informational resource; early vision processes do not operate over cognitive contents, which is the essence of the claim that perception is cognitively penetrated; early vision is cognitively impenetrable. Recently it has been argued that there are cognitive effects that affect early vision, such as the various pre-cueing effects guided by cognitively driven attention, which suggests that early vision is cognitively penetrated. In addition, since the signatures of these effects are found in early vision it seems that early vision is directly affected by cognition since its processes seem to use cognitive information. I defend the cognitive impenetrability of early vision in three steps. First, I discuss the problems the cognitively penetrability of perception causes for the epistemic role of perception in grounding perceptual beliefs. Second, I argue that whether a set of perceptual processes is cognitively penetrated hinges on whether there are cognitive effects that undermine the justificatory role of these processes in grounding empirical beliefs, and I examine the epistemic role of early vision. I argue, third, that the cognitive effects that act through pre-cueing do not undermine this role and, thus, do not render early vision cognitively penetrable. In addition, they do not entail that early vision uses cognitive information.

A neuroimaging point of view on the diversity of social cognition: evidence for extended influence of experience- and emotion-related factors on face processing

Faces are key social stimuli that convey a wealth of information essential for person perception and adaptive interpersonal behaviour. Studies in the domain of cognitive, affective, and social neuroscience have put in light that the processing of faces recruits specific visual regions and activates a distributed set of brain regions related to attentional, emotional, social, and memory processes associated with the perception of faces and the extraction of the numerous information attached to them. Studies using neuroimaging techniques such as functional magnetic resonance imaging (fMRI) have allowed localizing these brain regions and characterizing their functional properties. Magnetoencephalography (MEG) and electroencephalography (EEG) techniques are complementary to fMRI in that they offer a unique insight into the temporal dynamics of mental processes. In this article, I review the contribution of neuroimaging techniques to the knowledge on face processing and person perception with the aim of putting in light the extended influence of experience-related factors, particularly in relation with emotions, on the face processing system. Although the face processing network has evolved under evolutionary selection pressure related to sociality-related needs and is therefore highly conserved throughout the human species, neuroimaging studies put in light both the extension and the flexibility of the brain network involved in face processing. MEG and EEG allow in particular to reveal that the human brain integrates emotion- and experience-related information from the earliest stage of face processing. Altogether, this emphasizes the diversity of social cognitive processes associated with face perception.

Underpowered samples, false negatives, and unconscious learning

The scientific community has witnessed growing concern about the high rate of false positives and unreliable results within the psychological literature, but the harmful impact of false negatives has been largely ignored. False negatives are particularly concerning in research areas where demonstrating the absence of an effect is crucial, such as studies of unconscious or implicit processing. Research on implicit processes seeks evidence of above-chance performance on some implicit behavioral measure at the same time as chance-level performance (that is, a null result) on an explicit measure of awareness. A systematic review of 73 studies of contextual cuing, a popular implicit learning paradigm, involving 181 statistical analyses of awareness tests, reveals how underpowered studies can lead to failure to reject a false null hypothesis. Among the studies that reported sufficient information, the meta-analytic effect size across awareness tests was d_z = 0.31 (95 % CI 0.24–0.37), showing that participants’ learning in these experiments was conscious. The unusually large number of positive results in this literature cannot be explained by selective publication. Instead, our analyses demonstrate that these tests are typically insensitive and underpowered to detect medium to small, but true, effects in awareness tests. These findings challenge a widespread and theoretically important claim about the extent of unconscious human cognition.

Banknote recognition: investigating processing and cognition framework using competitive neural network

Humans are apt at recognizing patterns and discovering even abstract features which are sometimes embedded therein. Our ability to use the banknotes in circulation for business transactions lies in the effortlessness with which we can recognize the different banknote denominations after seeing them over a period of time. More significant is that we can usually recognize these banknote denominations irrespective of what parts of the banknotes are exposed to us visually. Furthermore, our recognition ability is largely unaffected even when these banknotes are partially occluded. In a similar analogy, the robustness of intelligent systems to perform the task of banknote recognition should not collapse under some minimum level of partial occlusion. Artificial neural networks are intelligent systems which from inception have taken many important cues related to structure and learning rules from the human nervous/cognition processing system. Likewise, it has been shown that advances in artificial neural network simulations can help us understand the human nervous/cognition system even furthermore. In this paper, we investigate three cognition hypothetical frameworks to vision-based recognition of banknote denominations using competitive neural networks. In order to make the task more challenging and stress-test the investigated hypotheses, we also consider the recognition of occluded banknotes. The implemented hypothetical systems are tasked to perform fast recognition of banknotes with up to 75 % occlusion. The investigated hypothetical systems are trained on Nigeria's Naira banknotes and several experiments are performed to demonstrate the findings presented within this work.

Early effects of previous experience on conscious perception

Constructive theories of brain function such as predictive coding posit that prior knowledge affects our experience of the world quickly and directly. However, it is yet unknown how swiftly prior knowledge impacts the neural processes giving rise to conscious experience. Here we used an experimental paradigm where prior knowledge augmented perception and measured the timing of this effect with magnetoencephalography (MEG). By correlating the perceptual benefits of prior knowledge with the MEG activity, we found that prior knowledge took effect in the time-window 80–95 ms after stimulus onset, thus reflecting an early influence on conscious perception. The sources of this effect were localized to occipital and posterior parietal regions. These results are in line with the predictive coding framework.

Expectation Suppression in Early Visual Cortex Depends on Task Set

Stimulus expectation can modulate neural responses in early sensory cortical regions, with expected stimuli often leading to a reduced neural response. However, it is unclear whether this expectation suppression is an automatic phenomenon or is instead dependent on the type of task a subject is engaged in. To investigate this, human subjects were presented with visual grating stimuli in the periphery that were either predictable or non-predictable while they performed three tasks that differently engaged cognitive resources. In two of the tasks, the predictable stimulus was task-irrelevant and spatial attention was engaged at fixation, with a high load on either perceptual or working memory resources. In the third task, the predictable stimulus was task-relevant, and therefore spatially attended. We observed that expectation suppression is dependent on the cognitive resources engaged by a subjects’ current task. When the grating was task-irrelevant, expectation suppression for predictable items was visible in retinotopically specific areas of early visual cortex (V1-V3) during the perceptual task, but it was abolished when working memory was loaded. When the grating was task-relevant and spatially attended, there was no significant effect of expectation in early visual cortex. These results suggest that expectation suppression is not an automatic phenomenon, but dependent on attentional state and type of available cognitive resources.

Attention and prediction in human audition: a lesson from cognitive psychophysiology

Attention is a hypothetical mechanism in the service of perception that facilitates the processing of relevant information and inhibits the processing of irrelevant information. Prediction is a hypothetical mechanism in the service of perception that considers prior information when interpreting the sensorial input. Although both (attention and prediction) aid perception, they are rarely considered together. Auditory attention typically yields enhanced brain activity, whereas auditory prediction often results in attenuated brain responses. However, when strongly predicted sounds are omitted, brain responses to silence resemble those elicited by sounds. Studies jointly investigating attention and prediction revealed that these different mechanisms may interact, e.g. attention may magnify the processing differences between predicted and unpredicted sounds. Following the predictive coding theory, we suggest that prediction relates to predictions sent down from predictive models housed in higher levels of the processing hierarchy to lower levels and attention refers to gain modulation of the prediction error signal sent up to the higher level. As predictions encode contents and confidence in the sensory data, and as gain can be modulated by the intention of the listener and by the predictability of the input, various possibilities for interactions between attention and prediction can be unfolded. From this perspective, the traditional distinction between bottom-up/exogenous and top-down/endogenous driven attention can be revisited and the classic concepts of attentional gain and attentional trace can be integrated.

Rapid plasticity in the prefrontal cortex during affective associative learning

MultiCS conditioning is an affective associative learning paradigm, in which affective categories consist of many similar and complex stimuli. Comparing visual processing before and after learning, recent MultiCS conditioning studies using time-sensitive magnetoencephalography (MEG) revealed enhanced activation of prefrontal cortex (PFC) regions towards emotionally paired versus neutral stimuli already during short-latency processing stages (i.e., 50 to 80 ms after stimulus onset). The present study aimed at showing that this rapid differential activation develops as a function of the acquisition and not the extinction of the emotional meaning associated with affectively paired stimuli. MEG data of a MultiCS conditioning study were analyzed with respect to rapid changes in PFC activation towards aversively (electric shock) paired and unpaired faces that occurred during the learning of stimulus-reinforcer contingencies. Analyses revealed an increased PFC activation towards paired stimuli during 50 to 80 ms already during the acquisition of contingencies, which emerged after a single pairing with the electric shock. Corresponding changes in stimulus valence could be observed in ratings of hedonic valence, although participants did not seem to be aware of contingencies. These results suggest rapid formation and access of emotional stimulus meaning in the PFC as well as a great capacity for adaptive and highly resolving learning in the brain under challenging circumstances.

How and when predictability interacts with accentuation in temporally selective attention during speech comprehension

The present study used EEG to investigate how and when top-down prediction interacts with bottom-up acoustic signals in temporally selective attention during speech comprehension. Mandarin Chinese spoken sentences were used as stimuli. We systematically manipulated the predictability and de/accentuation of the critical words in the sentence context. Meanwhile, a linguistic attention probe 'ba' was presented concurrently with the critical words or not. The results showed that, first, words with a linguistic attention probe elicited a larger N1 than those without a probe. The latency of this N1 effect was shortened for accented or lowly predictable words, indicating more attentional resources allocated to these words. Importantly, prediction and accentuation showed a complementary interplay on the latency of this N1 effect, demonstrating that when the words had already attracted attention due to low predictability or due to the presence of pitch accent, the other factor did not modulate attention allocation anymore. Second, relative to the lowly predictable words, the highly predictable words elicited a reduced N400 and enhanced gamma-band power increases, especially under the accented conditions; moreover, under the accented conditions, shorter N1 peak-latency was found to correlate with larger gamma-band power enhancement, which indicates that a close relationship might exist between early selective attention and later semantic integration. Finally, the interaction between top-down selective attention (driven by prediction) and bottom-up selective attention (driven by accentuation) occurred before lexical-semantic processing, namely before the N400 effect evoked by predictability, which was discussed with regard to the language comprehension models.

Both attention and prediction are necessary for adaptive neuronal tuning in sensory processing

The brain as a proactive system processes sensory information under the top-down influence of attention and prediction. However, the relation between attention and prediction remains undetermined given the conflation of these two mechanisms in the literature. To evaluate whether attention and prediction are dependent of each other, and if so, how these two top-down mechanisms may interact in sensory processing, we orthogonally manipulated attention and prediction in a target detection task. Participants were instructed to pay attention to one of two interleaved stimulus streams of predictable/unpredictable tone frequency. We found that attention and prediction interacted on the amplitude of the N1 ERP component. The N1 amplitude in the attended/predictable condition was larger than that in any of the other conditions. Dipole source localization analysis showed that the effect came from the activation in bilateral auditory areas. No significant effect was found in the P2 time window. Our results suggest that attention and prediction are dependent of each other. While attention might determine the overall cortical responsiveness to stimuli when prediction is involved, prediction might provide an anchor for the modulation of the synaptic input strengths which needs to be operated on the basis of attention.

Is attention based on spatial contextual memory preferentially guided by low spatial frequency signals?

A popular model of visual perception states that coarse information (carried by low spatial frequencies) along the dorsal stream is rapidly transmitted to prefrontal and medial temporal areas, activating contextual information from memory, which can in turn constrain detailed input carried by high spatial frequencies arriving at a slower rate along the ventral visual stream, thus facilitating the processing of ambiguous visual stimuli. We were interested in testing whether this model contributes to memory-guided orienting of attention. In particular, we asked whether global, low-spatial frequency (LSF) inputs play a dominant role in triggering contextual memories in order to facilitate the processing of the upcoming target stimulus. We explored this question over four experiments. The first experiment replicated the LSF advantage reported in perceptual discrimination tasks by showing that participants were faster and more accurate at matching a low spatial frequency version of a scene, compared to a high spatial frequency version, to its original counterpart in a forced-choice task. The subsequent three experiments tested the relative contributions of low versus high spatial frequencies during memory-guided covert spatial attention orienting tasks. Replicating the effects of memory-guided attention, pre-exposure to scenes associated with specific spatial memories for target locations (memory cues) led to higher perceptual discrimination and faster response times to identify targets embedded in the scenes. However, either high or low spatial frequency cues were equally effective; LSF signals did not selectively or preferentially contribute to the memory-driven attention benefits to performance. Our results challenge a generalized model that LSFs activate contextual memories, which in turn bias attention and facilitate perception.

Preferential responses to extinguished face stimuli are preserved in frontal and occipito-temporal cortex at initial but not later stages of processing

Magnetoencephalographic correlates of rapid emotional responses (50-80 ms) in frontal and occipito-temporal regions have recently been reported using a novel MultiCS Conditioning paradigm with odor-conditioned faces. As those short-latency responses were supposed to partially reflect initial access to nonextinguished emotional memories, it could be predicted that they outlast the extinction phase. To test this hypothesis, appetitively and aversively odor-conditioned faces were frequently presented during extinction while event-related magnetic fields were recorded. Affect-specific responses in frontal and occipito-temporal areas were found in the early (50-80 ms) but not in the later (130-190 ms) time interval following extinction learning. These results suggest that previously acquired emotional memories can be accessed at initial processing stages but become ineffective in modulating processing at later stages as extinction proceeds.

Long-term memories bias sensitivity and target selection in complex scenes

In everyday situations, we often rely on our memories to find what we are looking for in our cluttered environment. Recently, we developed a new experimental paradigm to investigate how long-term memory (LTM) can guide attention and showed how the pre-exposure to a complex scene in which a target location had been learned facilitated the detection of the transient appearance of the target at the remembered location [Summerfield, J. J., Rao, A., Garside, N., & Nobre, A. C. Biasing perception by spatial long-term memory. The Journal of Neuroscience, 31, 14952-14960, 2011; Summerfield, J. J., Lepsien, J., Gitelman, D. R., Mesulam, M. M., & Nobre, A. C. Orienting attention based on long-term memory experience. Neuron, 49, 905-916, 2006]. This study extends these findings by investigating whether and how LTM can enhance perceptual sensitivity to identify targets occurring within their complex scene context. Behavioral measures showed superior perceptual sensitivity (d') for targets located in remembered spatial contexts. We used the N2pc ERP to test whether LTM modulated the process of selecting the target from its scene context. Surprisingly, in contrast to effects of visual spatial cues or implicit contextual cueing, LTM for target locations significantly attenuated the N2pc potential. We propose that the mechanism by which these explicitly available LTMs facilitate perceptual identification of targets may differ from mechanisms triggered by other types of top-down sources of information.

The Rapid Extraction of Gist—Early Neural Correlates of High-level Visual Processing

The human cognitive system is highly efficient in extracting information from our visual environment. This efficiency is based on acquired knowledge that guides our attention toward relevant events and promotes the recognition of individual objects as they appear in visual scenes. The experience-based representation of such knowledge contains not only information about the individual objects but also about relations between them, such as the typical context in which individual objects co-occur. The present EEG study aimed at exploring the availability of such relational knowledge in the time course of visual scene processing, using oscillatory evoked gamma-band responses as a neural correlate for a currently activated cortical stimulus representation. Participants decided whether two simultaneously presented objects were conceptually coherent (e.g., mouse–cheese) or not (e.g., crown–mushroom). We obtained increased evoked gamma-band responses for coherent scenes compared with incoherent scenes beginning as early as 70 msec after stimulus onset within a distributed cortical network, including the right temporal, the right frontal, and the bilateral occipital cortex. This finding provides empirical evidence for the functional importance of evoked oscillatory activity in high-level vision beyond the visual cortex and, thus, gives new insights into the functional relevance of neuronal interactions. It also indicates the very early availability of experience-based knowledge that might be regarded as a fundamental mechanism for the rapid extraction of the gist of a scene.

Biasing perception by spatial long-term memory

Human perception is highly flexible and adaptive. Selective processing is tuned dynamically according to current task goals and expectations to optimize behavior. Arguably, the major source of our expectations about events yet to unfold is our past experience; however, the ability of long-term memories to bias early perceptual analysis has remained untested. We used a noninvasive method with high temporal resolution to record neural activity while human participants detected visual targets that appeared at remembered versus novel locations within naturalistic visual scenes. Upon viewing a familiar scene, spatial memories changed oscillatory brain activity in anticipation of the target location. Memory also enhanced neural activity during early stages of visual analysis of the target and improved behavioral performance. Both measures correlated with subsequent target-detection performance. We therefore demonstrated that memory can directly enhance perceptual functions in the human brain.

Attention reverses the effect of prediction in silencing sensory signals

Predictive coding models suggest that predicted sensory signals are attenuated (silencing of prediction error). These models, though influential, are challenged by the fact that prediction sometimes seems to enhance rather than reduce sensory signals, as in the case of attentional cueing experiments. One possible explanation is that in these experiments, prediction (i.e., stimulus probability) is confounded with attention (i.e., task relevance), which is known to boost rather than reduce sensory signal. However, recent theoretical work on predictive coding inspires an alternative hypothesis and suggests that attention and prediction operate synergistically to improve the precision of perceptual inference. This model posits that attention leads to heightened weighting of sensory evidence, thereby reversing the sensory silencing by prediction. Here, we factorially manipulated attention and prediction in a functional magnetic resonance imaging study and distinguished between these 2 hypotheses. Our results support a predictive coding model wherein attention reverses the sensory attenuation of predicted signals.

Expectations change the signatures and timing of electrophysiological correlates of perceptual awareness

Previous experience allows the brain to predict what comes next. How these expectations affect conscious experience is poorly understood. In particular, it is unknown whether and when expectations interact with sensory evidence in granting access to conscious perception, and how this is reflected electrophysiologically. Here, we parametrically manipulate sensory evidence and expectations while measuring event-related potentials in human subjects to assess the time course of evoked responses that correlate with subjective visibility, the properties of the stimuli, and/or perceptual expectations. We found that expectations lower the threshold of conscious perception and reduce the latency of neuronal signatures differentiating seen and unseen stimuli. Without expectations, this differentiation occurs ∼300 ms and with expectations ∼200 ms after stimulus in occipitoparietal sensors. The amplitude of this differentiating response component (P2) decreases as visibility increases, regardless of whether this increase is attributable to enhanced sensory evidence and/or the gradual buildup of perceptual expectations. Importantly, at matched performance levels, responses to seen and unseen stimuli differed regardless of the physical stimulus properties. These findings indicate that the latency of the neuronal correlates of access to consciousness depend on whether access is driven by stimulus saliency or by a combination of expectations and sensory evidence.

Top-down effects on early visual processing in humans: a predictive coding framework

An increasing number of human electroencephalography (EEG) studies examining the earliest component of the visual evoked potential, the so-called C1, have cast doubts on the previously prevalent notion that this component is impermeable to top-down effects. This article reviews the original studies that (i) described the C1, (ii) linked it to primary visual cortex (V1) activity, and (iii) suggested that its electrophysiological characteristics are exclusively determined by low-level stimulus attributes, particularly the spatial position of the stimulus within the visual field. We then describe conflicting evidence from animal studies and human neuroimaging experiments and provide an overview of recent EEG and magnetoencephalography (MEG) work showing that initial V1 activity in humans may be strongly modulated by higher-level cognitive factors. Finally, we formulate a theoretical framework for understanding top-down effects on early visual processing in terms of predictive coding.

Framework of Consciousness from Semblance of Activity at Functionally LINKed Postsynaptic Membranes

Consciousness is seen as a difficult “binding” problem. Binding, a process where different sensations evoked by an item are associated in the nervous system, can be viewed as a process similar to associative learning. Several reports that consciousness is associated with some form of memory imply that different forms of memories have a common feature contributing to consciousness. Based on a proposed synaptic mechanism capable of explaining different forms of memory, we developed a framework for consciousness. It is based on the formation of semblance of sensory stimulus from (1) synaptic semblances when excitatory postsynaptic potentials arrive at functionally LINKed postsynaptic membranes, and (2) network semblances when these potentials summate to elicit action potential initiating activity in a network of neurons. It is then possible to derive a framework for consciousness as a multi-dimensional semblance. According to this framework, a continuum of semblances formed from background sensory stimuli and oscillating neuronal activities serve to maintain consciousness. Feasibility of this framework to explain various physiological and pathological states of consciousness, its subjective nature and qualia is examined.

Contextual cost: when a visual-search target is not where it should be

Visual search is often facilitated when the search display occasionally repeats, revealing a contextual-cueing effect. According to the associative-learning account, contextual cueing arises from associating the display configuration with the target location. However, recent findings emphasizing the importance of local context near the target have given rise to the possibility that low-level repetition priming may account for the contextual-cueing effect. This study distinguishes associative learning from local repetition priming by testing whether search is directed toward a target's expected location, even when the target is relocated. After participants searched for a T among Ls in displays that repeated 24 times, they completed a transfer session where the target was relocated locally to a previously blank location (Experiment 1) or to an adjacent distractor location (Experiment 2). Results revealed that contextual cueing decreased as the target appeared farther away from its expected location, ultimately resulting in a contextual cost when the target swapped locations with a local distractor. We conclude that target predictability is a key factor in contextual cueing.

Unconscious learning versus visual perception: dissociable roles for gamma oscillations revealed in MEG

Oscillatory synchrony in the gamma band (30-120 Hz) has been involved in various cognitive functions including conscious perception and learning. Explicit memory encoding, in particular, relies on enhanced gamma oscillations. Does this finding extend to unconscious memory encoding? Can we dissociate gamma oscillations related to unconscious learning and to conscious perception? We investigate these issues in a magnetoencephalographic experiment using a modified version of the contextual cueing paradigm. In this visual search task, repeated presentation of search arrays triggers an unconscious spatial learning process that speeds reaction times but leaves conscious perception unaffected. In addition to a high-frequency perceptual gamma activity present throughout the experiment, we reveal the existence of a fronto-occipital network synchronized in the low gamma range specifically engaged in unconscious learning. This network shows up as soon as a display is searched for the second time and disappears as behavior gets affected. We suggest that oscillations in this network shape neural processing to build an efficient neural route for learned displays. Accordingly, in the last part of the experiment, evoked responses dissociate learned images at early latencies, suggesting that a sharpened representation is activated without resort on learning gamma oscillations, whereas perceptual gamma oscillations remain unaffected.