Visual search in depth: The neural correlates of segmenting a display into relevant and irrelevant three-dimensional regions

Orienting attention across binocular disparity

The spatial distribution of covert visual attention following an exogenous cue is often described as a spotlight, which disregards depth. Here, we study the orienting of attention across binocular disparity, a key depth cue in primates. A small Gabor patch target was displayed at ±12-arcmin horizontal offset in each eye independently, resulting in four possible 3D locations. With some latency relative to target onset (0-300 ms), an attentional cue was displayed at one of five binocular locations, resulting in various combinations of relative azimuth (horizontal position) and disparity (depth). Observers' task was to discriminate the orientation of the target. Observers' performance decreased as the relative azimuth between the cue and the target increased. Performance also decreased with the difference in disparity, even when the azimuth remained constant. Performance varied with the delay between the cue and the target and was maximal between 100 and 150 ms. The orienting of attention in azimuth and depth followed the same time course. We mapped the 3D shape of attentional focus over time and found that the spatial envelope was approximately a Gaussian modulated in time. These results could not be explained by monocular confounds nor by eye movements. We conclude that exogenous cues direct attention not only to their visual direction but also to their depth and that binocular disparity is sufficient to define that depth. The identical time course and interaction between azimuth and depth suggest a shared mechanism, and therefore that visual attention to spatial location is an intrinsically 3D process.

Brain activity underlying visual search in depth when viewing volumetric multiplanar images

The study investigated the cortical activity associated with 3D and 2D image perception on a volumetric multiplanar display by analyzing event-related potentials (ERPs) and power spectral density (PSD). In this study, we used a volumetric multiplanar display to present visual targets, and the brain signals were recorded via an EEG amplifier and analyzed using the EEGLAB toolbox on MATLAB. The study found no significant differences in amplitude between the 3D and 2D conditions across five occipital and parietal electrodes. However, there was a significant difference in latency of the P3 component on the Pz electrode. The analysis of PSD showed no significant differences between the two conditions, although there was a slightly higher alpha and beta activity observed in the 2D visualization. The study concluded that 3D image representation on a volumetric multiplanar display has no more sensory or cognitive load on the human brain than 2D representation, and that depth perception on a multiplanar display requires less brain activity.

Top-down control of attention by stereoscopic depth

Stereoscopic depth has a mixed record as a guiding attribute in visual attention. Visual search can be efficient if the target lies at a unique depth; whereas automatic segmentation of search arrays into different depth planes does not appear to be pre-attentive. These prior findings describe bottom-up, stimulus-driven depth guidance. Here, we ask about the top-down selection of depth information. To assess the ability to direct attention to specific depth planes, Experiment 1 used the centroid judgment paradigm which permits quantitative measures of selective processing of items of different depths or colors. Experiment 1 showed that a subset of observers could deploy specific attention filters for each of eight depth planes, suggesting that at least some observers can direct attention to a specific depth plane quite precisely. Experiment 2 used eight depth planes in a visual search experiment. Observers were encouraged to guide their attention to far or near depth planes with an informative but imperfect cue. The benefits of this probabilistic cue were small. However, this may not be a specific problem with guidance by stereoscopic depth. Equivalently poor results were obtained with color. To check and prove that depth guidance in search is possible, Experiment 3 presented items in only two depth planes. In this case, information about the target depth plane allows observers to search more efficiently, replicating earlier work. We conclude that top-down guidance by stereoscopic depth is possible but that it is hard to apply the full range of our stereoscopic ability in search.

Allocation of attention in 3D space is adaptively modulated by relative position of target and distractor stimuli

Allocation of attention across different depth planes is a prerequisite for visual selection in a three-dimensional environment. Previous research showed that participants successfully used stereoscopic depth information to focus their attention. This, however, does not mean that salient information from other depth planes is completely neglected. The present study investigated the question of whether competing visual information is differentially processed when displayed in a single depth plane or across two different depth planes. Moreover, it was of interest whether potential effects were further modulated by the items' relative spatial position (near or far). In three experiments participants performed a variant of the additional singleton paradigm. Targets were defined by stereoscopic depth information and as such appeared either in a near or far depth plane. Distractor stimuli were displayed in the same or in the opposed depth plane. The results consistently showed that visual selection was slower when target and distractor coincided within the same depth plane. There was no general advantage for targets presented in near or far depth planes. However, differential effects of target depth plane and the target-distractor relation were observed. Selection of near targets was more affected by distractors within the same depth plane while far targets were identified more slowly when the amount of information in closer depth planes increased. While attentional resources could not be exclusively centered to a distinct depth plane, the allocation of attention might be organized along an egocentric gradient through space and varies with the organization of the visual surrounding.

The influence of relevant and irrelevant stereoscopic depth cues: Depth information does not always capture attention

Previous research reported ambiguous findings regarding the relationship of visuospatial attention and (stereoscopic) depth information. Some studies indicate that attention can be focused on a distinct depth plane, while other investigations revealed attentional capture from irrelevant items located in other, unattended depth planes. To evaluate whether task relevance of depth information modulates the deployment of attentional resources across depth planes, the additional singleton paradigm was adapted: Singletons defined by depth (i.e., displayed behind or in front of a central depth plane) or color (green against gray) were presented among neutral items and served as targets or (irrelevant) distractors. When participants were instructed to search for a color target, no attentional capture from irrelevant depth distractors was observed. In contrast, it took substantially longer to search for depth targets when an irrelevant distractor was presented simultaneously. Color distractors as well as depth distractors caused attentional capture, independent of the distractors' relative depth position (i.e., in front of or behind the target). However, slight differences in task performance were obtained depending on whether or not participants fixated within the target depth plane. Thus, the current findings indicate that attentional resources in general are uniformly distributed across different depth planes. Although task relevant depth singletons clearly affect the attentional system, this information might be processed subsequent to other stimulus features.

Differential patterns of 2D location versus depth decoding along the visual hierarchy

Visual information is initially represented as 2D images on the retina, but our brains are able to transform this input to perceive our rich 3D environment. While many studies have explored 2D spatial representations or depth perception in isolation, it remains unknown if or how these processes interact in human visual cortex. Here we used functional MRI and multi-voxel pattern analysis to investigate the relationship between 2D location and position-in-depth information. We stimulated different 3D locations in a blocked design: each location was defined by horizontal, vertical, and depth position. Participants remained fixated at the center of the screen while passively viewing the peripheral stimuli with red/green anaglyph glasses. Our results revealed a widespread, systematic transition throughout visual cortex. As expected, 2D location information (horizontal and vertical) could be strongly decoded in early visual areas, with reduced decoding higher along the visual hierarchy, consistent with known changes in receptive field sizes. Critically, we found that the decoding of position-in-depth information tracked inversely with the 2D location pattern, with the magnitude of depth decoding gradually increasing from intermediate to higher visual and category regions. Representations of 2D location information became increasingly location-tolerant in later areas, where depth information was also tolerant to changes in 2D location. We propose that spatial representations gradually transition from 2D-dominant to balanced 3D (2D and depth) along the visual hierarchy.