Here today, gone tomorrow--adaptation to change in memory-guided visual search

When experience with scenes foils attentional orienting: ERP evidence against flexible target-context mapping in visual search

Visual search is speeded when a target is repeatedly presented in an invariant scene context of nontargets (contextual cueing), demonstrating observers' capability for using statistical long-term memory (LTM) to make predictions about upcoming sensory events, thus improving attentional orienting. In the current study, we investigated whether expectations arising from individual, learned environmental structures can encompass multiple target locations. We recorded event-related potentials (ERPs) while participants performed a contextual cueing search task with repeated and non-repeated spatial item configurations. Notably, a given search display could be associated with either a single target location (standard contextual cueing) or two possible target locations. Our result showed that LTM-guided attention was always limited to only one target position in single- but also in the dual-target displays, as evidenced by expedited reaction times (RTs) and enhanced N1pc and N2pc deflections contralateral to one ("dominant") target of up to two repeating target locations. This contrasts with the processing of non-learned ("minor") target positions (in dual-target displays), which revealed slowed RTs alongside an initial N1pc "misguidance" signal that then vanished in the subsequent N2pc. This RT slowing was accompanied by enhanced N200 and N400 waveforms over fronto-central electrodes, suggesting that control mechanisms regulate the competition between dominant and minor targets. Our study thus reveals a dissociation in processing dominant versus minor targets: While LTM templates guide attention to dominant targets, minor targets necessitate control processes to overcome the automatic bias towards previously learned, dominant target locations.

Mission impossible? Spatial context relearning following a target relocation event depends on cue predictiveness

Visual search for a target is faster when the spatial layout of distractors is repeatedly encountered, illustrating that statistical learning of contextual invariances facilitates attentional guidance (contextual cueing; Chun & Jiang, 1998, Cognitive Psychology, 36, 28-71). While contextual learning is usually relatively efficient, relocating the target to an unexpected location (within an otherwise unchanged search layout) typically abolishes contextual cueing and the benefits deriving from invariant contexts recover only slowly with extensive training (Zellin et al., 2014, Psychonomic Bulletin & Review, 21(4), 1073-1079). However, a recent study by Peterson et al. (2022, Attention, Perception, & Psychophysics, 84(2), 474-489) in fact reported rather strong adaptation of spatial contextual memories following target position changes, thus contrasting with prior work. Peterson et al. argued that previous studies may have been underpowered to detect a reliable recovery of contextual cueing after the change. However, their experiments also used a specific display design that frequently presented the targets at the same locations, which might reduce the predictability of the contextual cues thereby facilitating its flexible relearning (irrespective of statistical power). The current study was a (high-powered) replication of Peterson et al., taking into account both statistical power and target overlap in context-memory adaptation. We found reliable contextual cueing for the initial target location irrespective of whether the targets shared their location across multiple displays, or not. However, contextual adaptation following a target relocation event occurred only when target locations were shared. This suggests that cue predictability modulates contextual adaptation, over and above a possible (yet negligible) influence of statistical power.

No evidence for contextual cueing beyond explicit recognition

Many studies claim that visual regularities can be learned unconsciously and without explicit awareness. For example in the contextual cueing paradigm, studies often make claims using a standard reasoning based on two results: (1) a reliable response time (RT) difference between repeated vs. new stimulus displays and (2) a close-to-chance sensitivity when participants are asked to explicitly recognize repeated stimulus displays. From this pattern of results, studies routinely conclude that the sensitivity of RT responses is higher than that of explicit responses-an empirical situation we call Indirect Task Advantage (ITA). Many studies further infer from an ITA that RT effects were driven by a form of recognition that exceeds explicit memory: implicit recognition. However, this reasoning is flawed because the sensitivity underlying RT effects is never computed. To properly establish a difference, a sensitivity comparison is required. We apply this sensitivity comparison in a reanalysis of 20 contextual cueing studies showing that not a single study provides consistent evidence for ITAs. Responding to recent correlation-based arguments, we also demonstrate the absence of evidence for ITAs at the level of individual participants. This lack of ITAs has serious consequences for the field: If RT effects can be fully explained by weak but above-chance explicit recognition sensitivity, what is the empirical content of the label "implicit"? Thus, theoretical discussions in this paradigm-and likely in other paradigms using this standard reasoning-require serious reassessment because the current data from contextual cueing studies is insufficient to consider recognition as implicit.

Rethinking statistical learning as a continuous dynamic stochastic process, from the motor systems perspective

The brain integrates streams of sensory input and builds accurate predictions, while arriving at stable percepts under disparate time scales. This stochastic process bears different unfolding dynamics for different people, yet statistical learning (SL) currently averages out, as noise, individual fluctuations in data streams registered from the brain as the person learns. We here adopt a new analytical approach that instead of averaging out fluctuations in continuous electroencephalographic (EEG)-based data streams, takes these gross data as the important signals. Our new approach reassesses how individuals dynamically learn predictive information in stable and unstable environments. We find neural correlates for two types of learners in a visuomotor task: narrow-variance learners, who retain explicit knowledge of the regularity embedded in the stimuli. They seem to use an error-correction strategy steadily present in both stable and unstable environments. This strategy can be captured by current optimization-based computational frameworks. In contrast, broad-variance learners emerge only in the unstable environment. Local analyses of the moment-by-moment fluctuations, naïve to the overall outcome, reveal an initial period of memoryless learning, well characterized by a continuous gamma process starting out exponentially distributed whereby all future events are equally probable, with high signal (mean) to noise (variance) ratio. The empirically derived continuous Gamma process smoothly converges to predictive Gaussian signatures comparable to those observed for the error-corrective mode that is captured by current optimization-driven computational models. We coin this initially seemingly purposeless stage exploratory. Globally, we examine a posteriori the fluctuations in distributions' shapes over the empirically estimated stochastic signatures. We then confirm that the exploratory mode of those learners, free of expectation, random and memoryless, but with high signal, precedes the acquisition of the error-correction mode boasting smooth transition from exponential to symmetric distributions' shapes. This early naïve phase of the learning process has been overlooked by current models driven by expected, predictive information and error-based learning. Our work demonstrates that (statistical) learning is a highly dynamic and stochastic process, unfolding at different time scales, and evolving distinct learning strategies on demand.

Spatial context target relearning following a target relocation event: Not mission impossible

Our visual system relies on memory to store and retrieve goal-relevant structures and information from the environment for the purpose of optimizing the allocation of attention. This concept, referred to as contextual cueing, has been demonstrated using visual search tasks, wherein repeated visual contexts lead to reduced search times compared with random displays. Subsequently, when an unexpected change occurs in the environment, or memory fails, a cognitive expense is incurred as the mind tries to resolve the conflict with the memory of the previous environmental context. How memory resolves these conflicts and is updated is of great interest. Previous studies showed that, without extensive practice, individuals were unable to associate a secondary target location with a previously learned spatial context following the relocation of the initially learned target. Here, we explored variables that could potentially affect contextual learning and relearning, such as display size, crowding, context color, and whether the target switched to a previously occupied or unoccupied location. In a series of four experiments, we find relearning occurring in all instances. Previous research may have suffered from underpowered designs.

Contextual cueing is not flexible

Target detection is faster when search displays repeat, but properties of the memory representations that give rise to this contextual cueing effect remain uncertain. We adapted the contextual cueing task using an ABA design and recorded the eye movements of healthy young adults to determine whether the memory representations are flexible. Targets moved to a new location during the B phase and then returned to their original locations (second A phase). Contextual cueing effects in the first A phase were reinstated immediately in the second A phase, and response time costs eventually gave way to a repeated search advantage in the B phase, suggesting that two target-context associations were learned. However, this apparent flexibility disappeared when eye tracking data were used to subdivide repeated displays based on B-phase viewing of the original target quadrant. Therefore, memory representations acquired in the contextual cueing task resist change and are not flexible.

Raising awareness about measurement error in research on unconscious mental processes

Experimental psychologists often neglect the poor psychometric properties of the dependent measures collected in their studies. In particular, a low reliability of measures can have dramatic consequences for the interpretation of key findings in some of the most popular experimental paradigms, especially when strong inferences are drawn from the absence of statistically significant correlations. In research on unconscious cognition, for instance, it is commonly argued that the lack of a correlation between task performance and measures of awareness or explicit recollection of the target stimuli provides strong support for the conclusion that the cognitive processes underlying performance must be unconscious. Using contextual cuing of visual search as a case study, we show that given the low reliability of the dependent measures collected in these studies, it is usually impossible to draw any firm conclusion about the unconscious character of this effect from correlational analyses. Furthermore, both a psychometric meta-analysis of the available evidence and a cognitive-modeling approach suggest that, in fact, we should expect to see very low correlations between performance and awareness at the empirical level, even if both constructs are perfectly related at the latent level. Convincing evidence for the unconscious character of contextual cuing and other effects will most likely demand richer and larger data sets, coupled with more powerful analytic approaches.

Why Are Acquired Search-Guiding Context Memories Resistant to Updating?

Looking for goal-relevant objects in our various environments is one of the most ubiquitous tasks the human visual system has to accomplish (Wolfe, 1998). Visual search is guided by a number of separable selective-attention mechanisms that can be categorized as bottom-up driven – guidance by salient physical properties of the current stimuli – or top-down controlled – guidance by observers' “online” knowledge of search-critical object properties (e.g., Liesefeld and Müller, 2019). In addition, observers' expectations based on past experience also play also a significant role in goal-directed visual selection. Because sensory environments are typically stable, it is beneficial for the visual system to extract and learn the environmental regularities that are predictive of (the location of) the target stimulus. This perspective article is concerned with one of these predictive mechanisms: statistical context learning of consistent spatial patterns of target and distractor items in visual search. We review recent studies on context learning and its adaptability to incorporate consistent changes, with the aim to provide new directions to the study of processes involved in the acquisition of search-guiding context memories and their adaptation to consistent contextual changes – from a three-pronged, psychological, computational, and neurobiological perspective.

Stimulus-driven updating of long-term context memories in visual search

Visual search for a target is faster when the spatial layout of nontarget items is repeatedly encountered, illustrating that learned contextual invariances can improve attentional selection (contextual cueing). This type of contextual learning is usually relatively efficient, but relocating the target to an unexpected location (within otherwise unchanged layouts) typically abolishes contextual cueing. Here, we explored whether bottom-up attentional guidance can mediate the efficient contextual adaptation after the change. Two experiments presented an initial learning phase, followed by a subsequent relocation phase that introduced target location changes. This location change was accompanied by transient attention-guiding signals that either up-modulated the changed target location (Experiment 1), or which provided an inhibitory tag to down-modulate the initial target location (Experiment 2). The results from these two experiments showed reliable contextual cueing both before and after the target location change. By contrast, an additional control experiment (Experiment 3) that did not present any attention-guiding signals together with the changed target showed no reliable cueing in the relocation phase, thus replicating previous findings. This pattern of results suggests that attentional guidance (by transient stimulus-driven facilitatory and inhibitory signals) enhances the flexibility of long-term contextual learning.

Learning of association between a context and multiple possible target locations in a contextual cueing paradigm

Searching for a target is faster in a repeated context compared to a new context, possibly because the learned contextual information guides visual attention to the target location (attentional guidance). Previous studies showed that switching the target location following learning, or having the target appear in one of multiple possible locations during learning, fails to produce search facilitation in repeated contexts. In this study, we re-examined whether the learning of an association between a distractor configuration context and a target is limited to one-to-one context-target associations. Visual search response times were facilitated even when a repeated context was associated with one of four possible target locations, provided the target locations were also shared by other repeated distractor contexts. These results suggest that contextual cueing may involve mechanisms other than attentional guidance by one-to-one context-target associations.

Influences of luminance contrast and ambient lighting on visual context learning and retrieval

Invariant spatial context can guide attention and facilitate visual search, an effect referred to as "contextual cueing." Most previous studies on contextual cueing were conducted under conditions of photopic vision and high search item to background luminance contrast, leaving open the question whether the learning and/or retrieval of context cues depends on luminance contrast and ambient lighting. Given this, we conducted three experiments (each contains two subexperiments) to compare contextual cueing under different combinations of luminance contrast (high/low) and ambient lighting (photopic/mesopic). With high-contrast displays, we found robust contextual cueing in both photopic and mesopic environments, but the acquired contextual cueing could not be transferred when the display contrast changed from high to low in the photopic environment. By contrast, with low-contrast displays, contextual facilitation manifested only in mesopic vision, and the acquired cues remained effective following a switch to high-contrast displays. This pattern suggests that, with low display contrast, contextual cueing benefited from a more global search mode, aided by the activation of the peripheral rod system in mesopic vision, but was impeded by a more local, fovea-centered search mode in photopic vision.

Automatic Guidance (and Misguidance) of Visuospatial Attention by Acquired Scene Memory: Evidence From an N1pc Polarity Reversal

Visual search is facilitated when the target is repeatedly encountered at a fixed position within an invariant (vs. randomly variable) distractor layout—that is, when the layout is learned and guides attention to the target, a phenomenon known as contextual cuing. Subsequently changing the target location within a learned layout abolishes contextual cuing, which is difficult to relearn. Here, we used lateralized event-related electroencephalogram (EEG) potentials to explore memory-based attentional guidance (N = 16). The results revealed reliable contextual cuing during initial learning and an associated EEG-amplitude increase for repeated layouts in attention-related components, starting with an early posterior negativity (N1pc, 80–180 ms). When the target was relocated to the opposite hemifield following learning, contextual cuing was effectively abolished, and the N1pc was reversed in polarity (indicative of persistent misguidance of attention to the original target location). Thus, once learned, repeated layouts trigger attentional-priority signals from memory that proactively interfere with contextual relearning after target relocation.

Distributed attention beats the down-side of statistical context learning in visual search

Spatial attention can be deployed with a narrower focus to process individual items or distributed relatively broadly to process larger parts of a scene. This study investigated how focused- versus distributed-attention modes contribute to the adaptation of context-based memories that guide visual search. In two experiments, participants were either required to fixate the screen center and use peripheral vision for search ("distributed attention"), or they could freely move their eyes, enabling serial scanning of the search array ("focused attention"). Both experiments consisted of an initial learning phase and a subsequent test phase. During learning, participants searched for targets presented either among repeated (invariant) or nonrepeated (randomly generated) spatial layouts of distractor items. Prior research showed that repeated encounters of invariant display arrangements lead to long-term context memory about these arrays, which can then come to guide search (contextual-cueing effect). The crucial manipulation in the test phase was a change of the target location within an otherwise constant distractor layout, which has previously been shown to abolish the cueing effect. The current results replicated these findings, although importantly only when attention was focused. By contrast, with distributed attention, the cueing effect recovered rapidly and attained a level comparable to the initial effect (before the target location change). This indicates that contextual cueing can adapt more easily when attention is distributed, likely because a broad attentional set facilitates the flexible updating of global (distractor-distractor), as compared to more local (distractor-target), context representations-allowing local changes to be incorporated more readily.

Predictive visual search: Role of environmental regularities in the learning of context cues

Repeatedly searching through invariant spatial arrangements in visual search displays leads to the buildup of memory about these displays (contextual-cueing effect). In the present study, we investigate (1) whether contextual cueing is influenced by global statistical properties of the task and, if so, (2) whether these properties increase the overall strength (asymptotic level) or the temporal development (speed) of learning. Experiment 1a served as baseline against which we tested the effects of increased or decreased proportions of repeated relative to nonrepeated displays (Experiments 1b and 1c, respectively), thus manipulating the global statistical properties of search environments. Importantly, probability variations were achieved by manipulating the number of nonrepeated (baseline) displays so as to equate the total number of repeated displays across experiments. In Experiment 1d, repeated and nonrepeated displays were presented in longer streaks of trials, thus establishing a stable environment of sequences of repeated displays. Our results showed that the buildup of contextual cueing was expedited in the statistically rich Experiments 1b and 1d, relative to the baseline Experiment 1a. Further, contextual cueing was entirely absent when repeated displays occurred in the minority of trials (Experiment 1c). Together, these findings suggest that contextual cueing is modulated by observers' assumptions about the reliability of search environments.

Taking Attention Out of Context: Frontopolar Transcranial Magnetic Stimulation Abolishes the Formation of New Context Memories in Visual Search

This study investigates the causal contribution of the left frontopolar cortex (FPC) to the processing of violated expectations from learned target-distractor spatial contingencies during visual search. The experiment consisted of two phases: learning and test. Participants searched for targets presented either among repeated or nonrepeated target-distractor configurations. Prior research showed that repeated encounters of identically arranged displays lead to memory about these arrays, which then can come to guide search (contextual cueing effect). The crucial manipulation was a change of the target location, in a nevertheless constant distractor layout, at the transition from learning to test. In addition to this change, we applied repetitive transcranial magnetic stimulation (rTMS) over the left lateral FPC, over a posterior control site, or no rTMS at all (baseline; between-group manipulation) to see how FPC rTMS influences the ability of observers to adapt context-based memories acquired in the training phase. The learning phase showed expedited search in repeated relative to nonrepeated displays, with this context-based facilitation being comparable across all experimental groups. For the test phase, the recovery of cueing was critically dependent on the stimulation site: Although there was evidence of context adaptation toward the end of the experiment in the occipital and no-rTMS conditions, observers with FPC rTMS showed no evidence of relearning at all after target location changes. This finding shows that FPC plays an important role in the regulation of prediction errors in statistical context learning, thus contributing to an update of the spatial target-distractor contingencies after target position changes in learned spatial arrays.

Testing the controllability of contextual cuing of visual search

Locating a target among distractors improves when the configuration of distractors consistently cues the target's location across search trials, an effect called contextual cuing of visual search (CC). The important issue of whether CC is automatic has previously been studied by asking whether it can occur implicitly (outside awareness). Here we ask the novel question: is CC of visual search controllable? In 3 experiments participants were exposed to a standard CC procedure during Phase 1. In Phase 2, they localized a new target, embedded in configurations (including the previous target) repeated from Phase 1. Despite robust contextual cuing, congruency effects - which would imply the orientation of attention towards the old target in repeated configurations - were found in none of the experiments. The results suggest that top-down control can be exerted over contextually-guided visual search.

Underpowered samples, false negatives, and unconscious learning

The scientific community has witnessed growing concern about the high rate of false positives and unreliable results within the psychological literature, but the harmful impact of false negatives has been largely ignored. False negatives are particularly concerning in research areas where demonstrating the absence of an effect is crucial, such as studies of unconscious or implicit processing. Research on implicit processes seeks evidence of above-chance performance on some implicit behavioral measure at the same time as chance-level performance (that is, a null result) on an explicit measure of awareness. A systematic review of 73 studies of contextual cuing, a popular implicit learning paradigm, involving 181 statistical analyses of awareness tests, reveals how underpowered studies can lead to failure to reject a false null hypothesis. Among the studies that reported sufficient information, the meta-analytic effect size across awareness tests was d_z = 0.31 (95 % CI 0.24–0.37), showing that participants’ learning in these experiments was conscious. The unusually large number of positive results in this literature cannot be explained by selective publication. Instead, our analyses demonstrate that these tests are typically insensitive and underpowered to detect medium to small, but true, effects in awareness tests. These findings challenge a widespread and theoretically important claim about the extent of unconscious human cognition.

From Foreground to Background: How Task-Neutral Context Influences Contextual Cueing of Visual Search

Selective attention determines the effectiveness of implicit contextual learning (e.g., Jiang and Leung, 2005). Visual foreground-background segmentation, on the other hand, is a key process in the guidance of attention (Wolfe, 2003). In the present study, we examined the impact of foreground-background segmentation on contextual cueing of visual search in three experiments. A visual search display, consisting of distractor 'L's and a target 'T', was overlaid on a task-neutral cuboid on the same depth plane (Experiment 1), on stereoscopically separated depth planes (Experiment 2), or spread over the entire display on the same depth plane (Experiment 3). Half of the search displays contained repeated target-distractor arrangements, whereas the other half was always newly generated. The task-neutral cuboid was constant during an initial training session, but was either rotated by 90° or entirely removed in the subsequent test sessions. We found that the gains resulting from repeated presentation of display arrangements during training (i.e., contextual-cueing effects) were diminished when the cuboid was changed or removed in Experiment 1, but remained intact in Experiments 2 and 3 when the cuboid was placed in a different depth plane, or when the items were randomly spread over the whole display but not on the edges of the cuboid. These findings suggest that foreground-background segmentation occurs prior to contextual learning, and only objects/arrangements that are grouped as foreground are learned over the course of repeated visual search.

Investigating implicit statistical learning mechanisms through contextual cueing

Since its inception, the contextual cueing (CC) paradigm has generated considerable interest in various fields of cognitive sciences because it constitutes an elegant approach to understanding how statistical learning (SL) mechanisms can detect contextual regularities during a visual search. In this article we review and discuss five aspects of CC: (i) the implicit nature of learning, (ii) the mechanisms involved in CC, (iii) the mediating factors affecting CC, (iv) the generalization of CC phenomena, and (v) the dissociation between implicit and explicit CC phenomena. The findings suggest that implicit SL is an inherent component of ongoing processing which operates through clustering, associative, and reinforcement processes at various levels of sensory-motor processing, and might result from simple spike-timing-dependent plasticity.

Long-term adaptation to change in implicit contextual learning

The visual world consists of spatial regularities that are acquired through experience in order to guide attentional orienting. For instance, in visual search, detection of a target is faster when a layout of nontarget items is encountered repeatedly, suggesting that learned contextual associations can guide attention (contextual cuing). However, scene layouts sometimes change, requiring observers to adapt previous memory representations. Here, we investigated the long-term dynamics of contextual adaptation after a permanent change of the target location. We observed fast and reliable learning of initial context-target associations after just three repetitions. However, adaptation of acquired contextual representations to relocated targets was slow and effortful, requiring 3 days of training with overall 80 repetitions. A final test 1 week later revealed equivalent effects of contextual cuing for both target locations, and these were comparable to the effects observed on day 1. That is, observers learned both initial target locations and relocated targets, given extensive training combined with extended periods of consolidation. Thus, while implicit contextual learning efficiently extracts statistical regularities of our environment at first, it is rather insensitive to change in the longer term, especially when subtle changes in context-target associations need to be acquired.

Response cost to repeated displays--when previous distractors become targets

A varied-target search task was used to evaluate the response cost of previous distractors becoming current targets in repeated visual search. We compared the relative contributions of distractor identity and location to producing response cost. During an exposure phase, half of the items were possible targets in each repeated display, and the other half were always distractors. Participants searched for a different target from the set of potential targets when the search displays were repeated. In the test phase of Experiments 1a and 1b, the roles of targets and distractors were reversed while the overall configuration was unchanged. Results indicated significant contextual costs after the switch of identities/locations between distractors and targets. In the test phase of Experiments 2a and 2b, target identities were changed again but the target locations remained the same. Less response cost was observed in this condition relative to when both identities and locations were changed. Proximity between target and distractors in the repeated displays also influenced response cost. The mechanisms responsible for the various response cost effects and the interplay between identity, location, and proximity in the production of response cost were discussed.