Reward learning biases the direction of saccades

Reward history modulates visual attention in an avian model

Attention can be biased towards previously reward-associated stimuli even when they are task-irrelevant and physically non-salient, although studies of reward-modulated attention have been largely limited to primate (including human and nonhuman) models. Birds have been shown to have the capacity to discriminate reward and spatial cues in a manner similar to primates, but whether reward history involuntarily affects their attention in the same way remains unclear. We adapted a spatial cueing paradigm with differential rewards to investigate how reward modulates the allocation of attention in peafowl (Pavo cristatus). The birds were required to locate and peck a target on a computer screen that was preceded by a high-value or low-value color cue that was uninformative with respect to the location of the upcoming target. All birds exhibited a validity effect (performance enhanced on valid compared to invalid cue), and an interaction effect between value and validity was evident at the group level, being particularly pronounced in the birds with the greatest amount of reward training. The time course of reward learning was conspicuously incremental, phenomenologically slower compared to primates. Our findings suggest a similar influence of reward history on attention across phylogeny despite a significant difference in neuroanatomy.

Neural correlates of value-driven spatial orienting

Reward learning has been shown to habitually guide overt spatial attention to specific regions of a scene. However, the neural mechanisms that support this bias are unknown. In the present study, participants learned to orient themselves to a particular quadrant of a scene (a high-value quadrant) to maximize monetary gains. This learning was scene-specific, with the high-value quadrant varying across different scenes. During a subsequent test phase, participants were faster at identifying a target if it appeared in the high-value quadrant (valid), and initial saccades were more likely to be made to the high-value quadrant. fMRI analyses during the test phase revealed learning-dependent priority signals in the caudate tail, superior colliculus, frontal eye field, anterior cingulate cortex, and insula, paralleling findings concerning feature-based, value-driven attention. In addition, ventral regions typically associated with scene selection and spatial information processing, including the hippocampus, parahippocampal gyrus, and temporo-occipital cortex, were also implicated. Taken together, our findings offer new insights into the neural architecture subserving value-driven attention, both extending our understanding of nodes in the attention network previously implicated in feature-based, value-driven attention and identifying a ventral network of brain regions implicated in reward's influence on scene-dependent spatial orienting.

Rewards weaken cross-modal inhibition of return with visual targets

Previous studies have shown that rewards weaken visual inhibition of return (IOR). However, the specific mechanisms underlying the influence of rewards on cross-modal IOR remain unclear. Based on the Posner exogenous cue-target paradigm, the present study was conducted to investigate the effect of rewards on exogenous spatial cross-modal IOR in both visual cue with auditory target (VA) and auditory cue with visual target (AV) conditions. The results showed the following: in the AV condition, the IOR effect size in the high-reward condition was significantly lower than that in the low-reward condition. However, in the VA condition, there was no significant IOR in either the high- or low-reward condition and there was no significant difference between the two conditions. In other words, the use of rewards modulated exogenous spatial cross-modal IOR with visual targets; specifically, high rewards may have weakened IOR in the AV condition. Taken together, our study extended the effect of rewards on IOR to cross-modal attention conditions and demonstrated for the first time that higher motivation among individuals under high-reward conditions weakened the cross-modal IOR with visual targets. Moreover, the present study provided evidence for future research on the relationship between reward and attention.

Reward Weakened Inhibition of Return (IOR) in the Near Depth Plane

In attentional orienting, researchers have proposed that reward history is a component of attentional control, as the reward value might enhance the spatial attention process to achieve more efficient goal-directed behavior and to improve target-detection performance. Although the effect of reward-induced motivation on attentional orienting has been studied in two-dimensional (2-D) space, the specific mechanisms underlying the influence of reward on inhibition of return (IOR) of attentional orienting in three-dimensional space (3-D) remain unclear. In the present study, by incorporating the Posner spatial-cueing paradigm into a virtual 3-D environment, we aimed to investigate the influence of reward on IOR in 3-D space. The results showed the following: (1) IOR size in the rewarded conditions was smaller than IOR size in the unrewarded condition in the near depth plane, resulting in an IOR difference with or without reward. (2) Reward weakened IOR in the near depth plane because the response to the uncued location was delayed, not because the response to the cued location was accelerated. The present study indicated that the different depth planes of the target location in 3-D space could influence the interaction between reward and IOR, and reward weakened IOR in the near depth plane.

The past, present, and future of selection history

The last ten years of attention research have witnessed a revolution, replacing a theoretical dichotomy (top-down vs. bottom-up control) with a trichotomy (biased by current goals, physical salience, and selection history). This third new mechanism of attentional control, selection history, is multifaceted. Some aspects of selection history must be learned over time whereas others reflect much more transient influences. A variety of different learning experiences can shape the attention system, including reward, aversive outcomes, past experience searching for a target, target‒non-target relations, and more. In this review, we provide an overview of the historical forces that led to the proposal of selection history as a distinct mechanism of attentional control. We then propose a formal definition of selection history, with concrete criteria, and identify different components of experience-driven attention that fit within this definition. The bulk of the review is devoted to exploring how these different components relate to one another. We conclude by proposing an integrative account of selection history centered on underlying themes that emerge from our review.

Throwing open the doors of perception: The role of dopamine in visual processing

Animals form associations between visual cues and behaviours. Although dopamine is known to be critical in many areas of the brain to bind sensory information with appropriate responses, dopamine's role in the visual system is less well understood. Visual signals, which indicate the likely occurrence of a rewarding or aversive stimulus or indicate the context within which such stimuli may arrive, modulate activity in the superior colliculus and alter behaviour. However, such signals primarily originate in cortical and basal ganglia circuits, and evidence of direct signalling from midbrain dopamine neurons to superior colliculus is lacking. Instead, hypothalamic A13 dopamine neurons innervate the superior colliculus, and dopamine receptors are differentially expressed in the superior colliculus, with D1 receptors in superficial layers and D2 receptors in deep layers. However, it remains unknown if A13 dopamine neurons control behaviours through their effect on afferents within the superior colliculus. We propose that A13 dopamine neurons may play a critical role in processing information in the superior colliculus, modifying behavioural responses to visual cues, and propose some testable hypotheses regarding dopamine's effect on visual perception.

Gotcha: Working memory prioritization from automatic attentional biases

Attention is an important resource for prioritizing information in working memory (WM), and it can be deployed both strategically and automatically. Most research investigating the relationship between WM and attention has focused on strategic efforts to deploy attentional resources toward remembering relevant information. However, such voluntary attentional control represents a mere subset of the attentional processes that select information to be encoded and maintained in WM (Theeuwes, Journal of Cognition, 1[1]: 29, 1-15, 2018). Here, we discuss three ways in which information becomes prioritized automatically in WM-physical salience, statistical learning, and reward learning. This review integrates findings from perception and working memory studies to propose a more sophisticated understanding of the relationship between attention and working memory.

Using aversive conditioning with near-real-time feedback to shape eye movements during naturalistic viewing

Strategically shaping patterns of eye movements through training has manifold promising applications, with the potential to improve the speed and efficiency of visual search, improve the ability of humans to extract information from complex displays, and help correct disordered eye movement patterns. However, training how a person moves their eyes when viewing an image or scene is notoriously difficult, with typical approaches relying on explicit instruction and strategy, which have notable limitations. The present study introduces a novel approach to eye movement training using aversive conditioning with near-real-time feedback. Participants viewed indoor scenes (eight scenes presented over 48 trials) with the goal of remembering those scenes for a later memory test. During viewing, saccades meeting specific amplitude and direction criteria probabilistically triggered an aversive electric shock, which was felt within 50 ms after the eliciting eye movement, allowing for a close temporal coupling between an oculomotor behavior and the feedback intended to shape it. Results demonstrate a bias against performing an initial saccade in the direction paired with shock (Experiment 1) or generally of the amplitude paired with shock (Experiment 2), an effect that operates without apparent awareness of the relationship between shocks and saccades, persists into extinction, and generalizes to the viewing of novel images. The present study serves as a proof of concept concerning the implementation of near-real-time feedback in eye movement training.

Inertia in value-driven attention

Previously reward-associated stimuli persistently capture attention. We attempted to extinguish this attentional bias through a reversal learning procedure where the high-value color changed unexpectedly. Attentional priority shifted during training in favor of the currently high-value color, although a residual bias toward the original high-value color was still evident. Importantly, during a subsequent test phase, attention was initially more strongly biased toward the original high-value color, counter to the attentional priorities evident at the end of training. Our results show that value-based attentional biases do not quickly update with new learning and lag behind the reshaping of strategic attentional priorities by reward.

Selection history is relative

Visual attention can be tuned to specific features to aid in visual search. The way in which these search strategies are established and maintained is flexible, reflecting goal-directed attentional control, but can exert a persistent effect on selection that remains even when these strategies are no longer advantageous, reflecting an attentional bias driven by selection history. Apart from feature-specific search, recent studies have shown that attention can be tuned to target-nontarget relationships. Here we tested whether a relational search strategy continues to bias attention in a subsequent task, where the relationally better color and former target color both serve as distractors (Experiment 1) or as potential targets (Experiment 2). We demonstrate that a relational bias can persist in a subsequent task in which color serves as a task-irrelevant feature, both impairing and facilitating visual search performance. Our findings extend our understanding of the relational account of attentional control and the nature of selection history effects on attention.