Shifting visual attention in stereographic displays: a time course analysis

Orienting attention across binocular disparity

The spatial distribution of covert visual attention following an exogenous cue is often described as a spotlight, which disregards depth. Here, we study the orienting of attention across binocular disparity, a key depth cue in primates. A small Gabor patch target was displayed at ±12-arcmin horizontal offset in each eye independently, resulting in four possible 3D locations. With some latency relative to target onset (0-300 ms), an attentional cue was displayed at one of five binocular locations, resulting in various combinations of relative azimuth (horizontal position) and disparity (depth). Observers' task was to discriminate the orientation of the target. Observers' performance decreased as the relative azimuth between the cue and the target increased. Performance also decreased with the difference in disparity, even when the azimuth remained constant. Performance varied with the delay between the cue and the target and was maximal between 100 and 150 ms. The orienting of attention in azimuth and depth followed the same time course. We mapped the 3D shape of attentional focus over time and found that the spatial envelope was approximately a Gaussian modulated in time. These results could not be explained by monocular confounds nor by eye movements. We conclude that exogenous cues direct attention not only to their visual direction but also to their depth and that binocular disparity is sufficient to define that depth. The identical time course and interaction between azimuth and depth suggest a shared mechanism, and therefore that visual attention to spatial location is an intrinsically 3D process.

The early attentional pancake: Minimal selection in depth for rapid attentional cueing

There have been conflicting findings on the degree to which rapidly deployed visual attention is selective for depth, and this issue has important implications for attention models. Previous findings have attempted to find depth-based cueing effects on such attention using reaction time (RT) measures for stimuli presented in stereo goggles with a display screen. Results stemming from such approaches have been mixed, depending on whether target/distractor discrimination was required. To help clarify the existence of such depth effects, we have developed a paradigm that measures accuracy rather than RT in an immersive virtual-reality environment, providing a more appropriate context of depth. Three modified Posner Cueing paradigms were run to test for depth-specific rapid attentional selectivity. Participants fixated a cross while attempting to identify a rapidly masked black letter preceded by a red cue that could be valid in depth, side, or both. In Experiment 1a, a potent cueing effect was found for lateral cueing validity, but a weak effect was found for depth despite an extreme difference in virtual depth (1 vs. 300 m). In Experiment 1b, a near-replication of 1a, the lateral effect replicated while the depth effect did not. Finally, in Experiment 2, to increase the depth cue's effectiveness, the letter matched the cue's color, and the presentation duration was increased; however, again only a minimal depth-based cueing effect - no greater than that of Experiment 1a - was observed. Thus, we conclude that rapidly deployed attention is driven largely by spatiotopic rather than depth-based information.

Shifts of spatial attention in perceived 3-D space

Previous studies have shown that spatial attention can shift in three-dimensional (3-D) space determined by binocular disparity. Using Posner′s precueing paradigm, the current work examined whether attentional selection occurs in perceived 3-D space defined by occlusion. Experiment 1 showed that shifts of spatial attention induced by central cues between two surfaces in the left and right visual fields did not differ between the conditions when the two surfaces were located at the same or different perceptual depth. In contrast, Experiment 2 found that peripheral cues generated a stronger cue validity effect when the two surfaces were perceived at a different rather than at the same perceptual depth. The results suggest that exogenous but not endogenous attention operates in perceived 3-D space.

Evidence for the effect of depth on visual working memory

Visual working memory (VWM) is a cognitive memory buffer for temporarily holding, processing, and manipulating visual information. Previous studies have demonstrated mixed results of the effect of depth perception on VWM, with some showing a beneficial effect while others not. In this study, we employed an adapted change detection paradigm to investigate the effects of two depth cues, binocular disparity and relative size. The memory array consisted of a set of pseudo-randomly positioned colored items, and the task was to judge whether the test item was changed compared to the memory item after a retention interval. We found that presenting the items in stereoscopic depth alone hardly affected VWM performance. When combining the two coherent depth cues, a significant larger VWM capacity of the perceptually closer-in-depth items was observed than that of the farther items, but the capacity for the two-depth-planes condition was not significantly different from that for the one-plane condition. Conflicting the two depth cues resulted in cancelling the beneficial effect of presenting items at a closer depth plane. The results indicate that depth perception could affect VWM, and the visual system may have an advantage in maintaining closer-in-depth objects in working memory.

Attentional Control in 3-D Space

Two experiments investigated the nature of attention in 3-D space. In Experiment 1, the hypothesis that attention can be localized to a depth plane was tested. Observers searched for a red line in two arrays of green lines. The arrays of lines were near in 2-D space but separated in depth. Search for the target was faster when the depth plane where the target would appear was cued, indicating attention can be localized in depth. A second experiment tested the hypothesis that attending to a location in depth would reduce the effect of a distracter at other depth locations. In this experiment, search for a tilted red line was faster when a distracting vertical line was present at another depth than when it was present at the same depth as the target. Implications for display design using depth information is discussed.

Is visual short-term memory depthful?

Does visual short-term memory (VSTM) depend on depth, as it might be if information was stored in more than one depth layer? Depth is critical in natural viewing and might be expected to affect retention, but whether this is so is currently unknown. Cued partial reports of letter arrays (Sperling, 1960) were measured up to 700 ms after display termination. Adding stereoscopic depth hardly affected VSTM capacity or decay inferred from total errors. The pattern of transposition errors (letters reported from an uncued row) was almost independent of depth and cue delay. We conclude that VSTM is effectively two-dimensional.

Depth modulation of the attentional repulsion effect

Shifts of attention can cause mislocalisation of visual objects. Abriefcue that attracts attention can cause a shift in the perceived location of a subsequently presented visual object--specifically, moving it away from the focus of attention (attentional repulsion). In the present study we investigated whether depth would influence the magnitude of attentional repulsion by presenting peripheral visual cues in different depth planes from the target or fixation. In experiment 1 the results showed that the magnitude of the attentional repulsion was larger when the cue was presented at the depth plane farther away from the target and fixation. In experiment 2 we presented the fixation point in the same depth plane as the nearest cues and found larger repulsion effects when the cues were presented in depth planes farther away from the fixation. In experiment 3 no depth modulation was observed when the fixation was presented in the same depth plane with the farthest cues. Taken together, when the cues were presented in the depth plane farther away from the fixation, the magnitude of the attentional repulsion effect increased. It is speculated that the residual coarser spatial representation in the space farther from the fixation plane or the enhanced attentional process for the space closer than the fixation may be responsible for the larger attentional repulsion effect.

Spatial orienting of attention in stereo depth

The aim of this study was to investigate the spatial orienting of visual attention in depth under purely stereoscopic viewing conditions. Random-dot stereograms were used to present disparity-defined target stimuli that were either validly or invalidly cued in depth. In separate tasks, participants responded either to the relative depth of the target (protruding vs. receding) or to its shape (square vs. diamond). Stimulus onset asynchronies (SOAs) between an uninformative exogenous cue and target were varied from 250 to 600 ms. For both tasks, mean response times (RTs) were shorter for validly than invalidly cued target depths and this RT advantage was essentially restricted to the shortest SOA of 250 ms. These results indicate that attention can be reflexively allocated to locations in stereo depth under conditions of low perceptual load, and independent of whether depth is relevant to the task or not.

Change detection for objects on surfaces slanted in depth

Change detection for objects associated with a surface extended in depth might be more difficult than for a frontal surface if it is easier to shift attention within a frontal surface. On the other hand, previous research has shown that ground surfaces have a special role in organizing the 3-D layout of objects shown against scene backgrounds. In the current study, we examined whether a frontal background or a ground surface background would result in superior change detection performance using a change detection flicker paradigm. In the first experiment, we considered whether background slant affects change detection performance. In Experiment 2, we examined the effect of height in the image on change detection performance. In Experiment 3, we examined change detection performance on slanted ceiling surfaces. The results of these experiments indicate that change detection is more efficient on near-ground planes than on surfaces at intermediate slants or ceiling surfaces. This suggests that any superiority of frontal plane backgrounds in a change detection task may be equivalent to the superiority of a near-ground plane in organizing a scene, with the lowest level of performance occurring for surfaces that are not frontal but further from a ground surface orientation.

Effect of fixation point distances on allocation of attention in real three-dimensional space

Although variation in spatial relations between observer and objects is important when attention functions in three-dimensional space, the effects of changes in spatial relations on attention in real three-dimensional space are still unclear. In Exp. 1, the effects of varying distances of the fixation point on attention in a spatial cuing paradigm were investigated. In Exp. 2, the effects of increase in task demand were examined by making observers detect a blinking fixation point, while performing the same task as in Exp. 1. In both experiments, attention could be manipulated by a precue when the fixation-point distance changed from trial to trial, and in Exp. 2 distribution of attention was manipulated by perceptual load. These findings together indicate that allocation of attentional resources in three-dimensional space varies by task demand.

Effects of self-motion on attention in real 3D space

Human behavioral space is three-dimensional (3D), and when moving through 3D space individuals selectively allocate their attention to acquire necessary information. This study aimed to examine the characteristics of attention in real 3D space when observers were moving forward. In static and self-motion situations, relative and absolute cues were used. Results indicated that internal representation for allocation of attention in 3D space is in depth-aware mode whether in static or moving condition. Moreover, near superior asymmetrical switching of attention is more strongly manifested under moving than under static conditions. Our findings indicate that focusing of attention by relative and absolute cues is maintained in self-motion, and that allocation of attention during movement is more viewer-centered than when observers are static.

Dynamics of attention in depth: evidence from multi-element tracking

We examined the allocation of attention in depth using a multi-element tracking paradigm. Observers were required to track a predefined subset of from two to eight elements in displays containing up to sixteen identical moving elements. We first show that depth cues, such as binocular disparity and occlusion through T-junctions, improve performance in a multi-element tracking task in the case where element boundaries are allowed to intersect in the depiction of motion in a single frontoparallel plane. We also show that the allocation of attention across two perceptually distinguishable planar surfaces, either frontoparallel or receding at a slanting angle and defined by coplanar elements, is easier than allocation of attention within a single surface. The same result was not found when attention was required to be deployed across items of two-color populations rather than across items of a single color. Our results suggest that, when surface information does not suffice to distinguish between targets and distractors that are embedded in these surfaces, division of attention across two surfaces aids in tracking moving targets. A final experiment with populations of elements moving within distinct volumes produced similar results, suggesting that spatial separation in three dimensions, rather than confinement to surfaces as such, may explain the improved performance for the two-surface case.

Functional representation of 3D space in endogenous attention shifts

The aim of this study was to explore whether the attentional system, as far as an endogenous orienting is concerned, allocates resources along the sagittal plane and whether such a process is affected by, and is likely to be based on, different functional representations of 3D space in the brain. Several models make a main action-based distinction between representations of peripersonal space and of those extrapersonal space. Accordingly, if attention has to move from one representation to another, it should be possible to observe a decrease in performance during such a transition. To test this hypothesis three experiments were run in which participants performed a cued detection task. Cue stimuli were informative and were centrally located around the fixation point. Target stimuli were displayed at four different depth planes. In the first experiment, assuming that the border between the peripersonal space and the extrapersonal space was at 1 m from the observer, half the target stimuli were located in the peripersonal space and half in the extrapersonal space. The fixation point was located at 1 m from the observer. In the second experiment, the fixation point was moved at 2 m from the observer in order to rule out the possible effects of ocular motor programming. In the third experiment, in order to rule out effects related to the spatial layout of target stimuli (i.e., centre of mass effect) two target stimuli were located in the peripersonal space and six in the extrapersonal space. In all the experiments, besides a validity effect, we observed greater reaction times when attention shift was across spatial representations than when it was within the same representation. The implications for action-oriented models of attention are discussed.

Attention switching in depth using random-dot autostereograms: attention gradient asymmetries

Random-dot autostereograms (RDASs) were used to investigate attention shifts along the sagittal plane in distractor-free tasks of high perceptual load. In three experiments using a same/different comparison task, the shape of the gradient over five different depths was examined and the conditions under which the gradient is and is not observed were compared. When the target set consisted of five similar objects, a robust asymmetric depth gradient was observed. When the target set consisted of two dissimilar objects, no gradient was observed. The results support a hypothesis of a viewer-centered asymmetric attention gradient in the depth plane that is dependent on perceptual or attentional load defined by target-set discriminability.