Impoverished second-order input to global linking in human vision

Integration of contours defined by second-order contrast-modulation of texture

Boundaries in the visual world can be defined by changes in luminance and texture in the input image. A "contour integration" process joins together local changes into percepts of lines or edges. A previous study tested the integration of contours defined by second-order contrast-modulation. Their contours were placed in a background of random wavelets. Participants performed near chance. We re-visited second-order contour integration with a different task. Participants distinguished contours with "good continuation" from distractors. We measured thresholds in different amounts of external orientation or position noise. This gave two noise-masking functions. We also measured thresholds for contours with a baseline curvature to assess performance with more curvy targets. Our participants were able to discriminate the good continuation of second-order contours. Thresholds were higher than for first-order contours. In our modelling, we found this was due to multiple factors. There was a doubling of equivalent internal noise between first- and second-order contour integration. There was also a reduction in efficiency. The efficiency difference was only significant in our orientation noise condition. For both first- and second-order stimuli, participants were also able to perform our task with more curved contours. We conclude that humans can integrate second-order contours, even when they are curved. There is however reduced performance compared to first-order contours. We find both an impaired input to the integrating mechanism, and reduced efficiency seem responsible. Second-order contour integration may be more affected by the noise background used in the previous study. Difficulty segregating that background may explain their result.

The efficiency of second order orientation coherence detection

Neurons in early visual cortex respond to both luminance- (1st order) and contrast-modulated (2nd order) local features in the visual field. In later extra-striate areas neurons with larger receptive fields integrate information across the visual field. For example, local luminance-defined features can be integrated into contours and shapes. Evidence for the global integration of features defined by contrast-modulation is less well established. While good performance in some shape tasks has been demonstrated with 2nd order stimuli, the integration of contours fails with 2nd order elements. Recently we developed a global orientation coherence task that is more basic than contour integration, bearing similarity to the well-established global motion coherence task. Similar to our previous 1st order result for this task, we find 2nd order coherence detection to be scale-invariant. There was a small but significant threshold elevation for 2nd order relative to 1st order. We used a noise masking approach to compare the efficiency of orientation integration for the 1st and 2nd order. We find a significant deficit for 2nd order detection at both the local and global level, however the small size of this effect stands in stark contrast against previous results from contour-integration experiments, which are almost impossible with 2nd order stimuli.

Component processes in contour integration: a direct comparison between snakes and ladders in a detection and a shape discrimination task

In contour integration, a relevant question is whether snakes and ladders are processed similarly. Higher presentation time thresholds for ladders in detection tasks indicate this is not the case. However, in a detection task only processing differences at the level of element linking and possibly contour localization might be picked up, while differences at the shape encoding level cannot be noticed. In this study, we make a direct comparison of detection and shape discrimination tasks to investigate if processing differences in the visual system between snakes and ladders are limited to contour detection or extend to higher level contour processing, like shape encoding. Stimuli consisted of elements that were oriented collinearly (snakes) or orthogonally (ladders) to the contour path and were surrounded by randomly oriented background elements. In two tasks, six experienced subjects either detected the contour when presented with a contour and a completely random stimulus or performed a shape discrimination task when presented with two contours with different curvature. Presentation time was varied in 9 steps between 8 and 492 ms. By applying a generalized linear mixed model we found that differences in snake and ladder processing are not limited to a detection stage but are also apparent at a shape encoding stage.

Flexible learning of natural statistics in the human brain

The ability to detect and identify targets in cluttered scenes is a critical skill for survival and interactions. To solve this challenge the brain has optimized mechanisms for capitalizing on frequently occurring regularities in the environment. Although evolution and development have been suggested to shape the brain's architecture in a manner that resembles these natural statistics, we provide novel evidence that short-term experience in adulthood may modify the brain's functional organization to support integration of signals atypical of shape contours in natural scenes. Although collinearity is a prevalent principle for perceptual integration in natural scenes, we show that observers learn to exploit other image regularities (i.e., orthogonal alignments of segments at an angle to the contour path) that typically signify discontinuities. Combining behavioral and functional MRI measurements, we demonstrate that this flexible learning is mediated by changes in the neural representations of behaviorally relevant image regularities primarily in dorsal visual areas. These changes in neural sensitivity are in line with changes in perceptual sensitivity for the detection of orthogonal contours and are evident only in observers that show significant performance improvement. In contrast, changes in the activation extent in frontoparietal regions are evident independent of performance changes, may support the detection of salient regions, and modulate perceptual integration in occipitotemporal areas in a top-down manner. Thus experience at shorter timescales in adulthood supports the adaptive functional optimization of visual circuits for flexible interpretation of natural scenes.

Experience shapes the utility of natural statistics for perceptual contour integration

Segmenting meaningful targets from cluttered scenes is a fundamental function of the visual system. Evolution and development have been suggested to optimize the brain's solution to this computationally challenging task by tuning the visual system to features that co-occur frequently in natural scenes (e.g., collinear edges) [1, 2, 3]. However, the role of shorter-term experience in shaping the utility of scene statistics remains largely unknown. Here, we ask whether collinearity is a specialized case, or whether the brain can learn to recruit any image regularity for the purpose of target identification. Consistent with long-term optimization for typical scene statistics, observers were better at detecting collinear contours than configurations of elements oriented at orthogonal or acute angles to the contour path. However, training resulted in improved detection of orthogonal contours that lasted for several months, suggesting retuning rather than transient changes of visual sensitivity. Improvement was also observed for acute contours but only after longer training. These results demonstrate that the brain flexibly exploits image regularities and learns to use discontinuities typically associated with surface boundaries (orthogonal, acute alignments) for contour linking and target identification. Thus, short-term experience in adulthood shapes the interpretation of scenes by assigning new statistical utility to image regularities.

Collinear facilitation: Effect of additive and multiplicative external noise

The detectability of a Gabor patch is improved by the presence of collinear flanking Gabors, this phenomenon is termed collinear facilitation. In experiment 1, we investigate the effects of adding 2D spatial luminance noise as a means of investigating different transects through the suprathreshold contrast space to see whether facilitation is ubiquitous throughout the contrast domain or whether it is confined to absolute contrast threshold. The results show that adding luminance noise abolishes the facilitation, showing it is confined to absolute threshold. In experiment 2, we assess whether 2nd order stimuli exhibit collinear facilitation and whether 1st order flanks can induce facilitation in 2nd order stimuli and vice versa. Our results suggest that collinear facilitation, albeit weaker, does occur for some 2nd order stimuli but we did not find any 1st/2nd order interactions, suggesting separate 1st/2nd order cortical processing streams, at least at the level at which this phenomenon occurs. Our two main findings, namely the lack of facilitation at suprathreshold contrasts and its presence for 2nd order processing argue against it playing a pivotal role in contour integration which does occur at all contrasts but not for these 2nd order stimuli.

Spatial versus temporal grouping in a modified Ternus display

The Ternus display can induce a percept of 'element motion' or 'group motion'. Conventionally, this has been attributed to two different motion processes, with different spatial and temporal ranges. In contrast, recent studies have emphasised spatial and temporal grouping principles as underlying the apparent motion percepts in the Ternus display. The present study explored effects of spatial and temporal grouping on the apparent motion percept in a novel Ternus display of oriented Gabor elements with no inter-frame interval. Each frame of this stimulus could be further divided into 'sub-frames', and the orientation of the carriers was changed across these sub-frames. In four experiments transitions were found between the motion percepts with changes in orientation across time (Experiment 1) and space (Experiment 2), and with a temporal offset in the orientation change of the outer element (Experiment 3) to the extent that group motion was not perceived even with large orientation changes over time that previously led to group motion (Experiment 4). Collectively, these results indicate that while spatial properties have an influence in determining the percept of the Ternus display, temporal properties also have a strong influence, and can override spatial grouping. However, these temporal effects cannot be attributed to spatio-temporal limits of low-level motion processes. Some aspects of the observed spatial grouping effects can be accounted for in terms of a modified association field, which may occur through connectivity of orientation selective units in V1. The temporal effects observed are considered in terms of temporal integration, the transitional value at a temporal offset of 40ms being remarkably similar to psychophysical and neurophysiological estimates of the peak temporal impulse response. These temporal responses could be detected at a higher level in the system, providing a basis for apparent motion perception.

Grouping local orientation and direction signals to extract spatial contours: empirical tests of "association field" models of contour integration

Over the last decade or so a great deal of psychophysical research has attempted to delineate the principles by which local orientations and motions are combined across space to facilitate the detection of simple spatial contours. This has led to the development of "association field" models of contour detection which suggest that the strength of linking between neighbouring elements in an image, is determined by the degree to which they aligned along smooth (first-order) curves. To test this assumption we used a path detection paradigm to compare the ability of observers to identify the presence of contours defined by either spatial orientation, motion direction or by specific combinations of both types of visual attribute. The relative alignment of the local orientations and/or directions with respect to the axis of the depicted contour was systematically varied. For orientation-defined contours detection was best when the elements were aligned along (parallel with) the contour axis, approached chance levels for obliquely oriented elements and then improved for elements that were orthogonal to the contour axis (i.e., performance was a U-shaped function of degree of orientation misalignment). This pattern of results was found for both straight and curved contours and is not readily explicable in terms of current association field theories. For motion-defined contours, however, performance simply deteriorated as the relative directions of the constituent path elements were progressively misaligned with respect to the contour. Thus the rules by which local orientations are linked to define spatial contours are qualitatively different from those used for linking local directions and each may be mediated by distinct visual mechanisms. When both orientation and motion cues were simultaneously available, contour detection performance was generally enhanced, in a manner that is consistent with probability summation. We suggest that association field models of orientation linking may need to be extended in light of the present findings.

Integration of orientation information in amblyopia

A recent report suggests that amblyopes are deficient in processing local orientation at supra-threshold contrasts. To determine whether amblyopes are also poor at integrating local orientation signals, we assessed performance for an orientation integration task in which the orientations of static signals are integrated across space. Our results show that amblyopic visual systems can integrate local static oriented signals with the same level of efficiency as normal visual systems. Although internal noise was slightly elevated, there was no indication that fewer samples were used to achieve optimal performance. This finding suggests normal integration of local orientation signals in amblyopia.

Glass pattern studies of local and global processing of contrast variations

Using Glass patterns [Nature 223 (1969) 578; Nature 246 (1973) 360; Perception 5 (1976) 67], we have studied the role of contrast differences in local and global processes of form perception. The virtue of these patterns (composed of a set of randomly distributed elements combined with a geometrically transformed copy) for studying object formation is that they allow ready isolation of local processes, the combination of dots to form a perceptual pair, from global processes, the combination of dipoles into the percept of an overall rotational or translational pattern. We find that a contrast difference within dot-pairs reduces the ability to resolve local features; large differences totally abolish the perception of the pattern. Contrast differences between dot-pairs lessen, but do not abolish, the global integration among local features. In both cases the effect is proportional to the ratio of the two contrast levels employed. Effects which differ for rotations and translations, are consistent with the greater areal integration required to resolve rotational patterns.

Good continuation with kinetic edges

We examined the perceptual formation of extended contours from second-order kinetic-edges created by motion discontinuities. Paths were formed by spatially separate kinetic-edge elements, aligned along smooth paths, and embedded in randomly oriented noise elements. Path detection was severely degraded when the sign of motion contrast alternated along the path compared to when the same sign was used, or if random motion direction was assigned to each edge element, or if alternating opposite motion directions was used along the paths. Performance increased monotonically with the length of the path. Irrespectively of path curvature a fast temporal summation occurs within the first 200-400 ms and then levels off. Hence, the kinetic-edge grouping is relatively fast and a pure second-order process that senses whether the motion is globally in the same phase and direction along extended contours.

Investigating local network interactions underlying first- and second-order processing

We compared the spatial lateral interactions for first-order cues to those for second-order cues, and investigated spatial interactions between these two types of cues. We measured the apparent modulation depth of a target Gabor at fixation, in the presence and the absence of horizontally flanking Gabors. The Gabors' gratings were either added to (first-order) or multiplied with (second-order) binary 2-D noise. Apparent "contrast" or modulation depth (i.e., the perceived difference between the high and low luminance regions for the first-order stimulus, or between the high and low contrast regions for the second-order stimulus) was measured with a modulation depth-matching paradigm. For each observer, the first- and second-order Gabors were equated for apparent modulation depth without the flankers. Our results indicate that at the smallest inter-element spacing, the perceived reduction in modulation depth is significantly smaller for the second-order than for the first-order stimuli. Further, lateral interactions operate over shorter distances and the spatial frequency and orientation tuning of the suppression effect are broader for second- than first-order stimuli. Finally, first- and second-order information interact in an asymmetrical fashion; second-order flankers do not reduce the apparent modulation depth of the first-order target, whilst first-order flankers reduce the apparent modulation depth of the second-order target.

Contour integration and cortical processing

Our understanding of visual processing in general, and contour integration in particular, has undergone great change over the last 10 years. There is now an accumulation of psychophysical and neurophysiological evidence that the outputs of cells with conjoint orientation preference and spatial position are integrated in the process of explication of rudimentary contours. Recent neuroanatomical and neurophysiological results suggest that this process takes place at the cortical level V1. The code for contour integration may be a temporal one in that it may only manifest itself in the latter part of the spike train as a result of feedback and lateral interactions. Here we review some of the properties of contour integration from a psychophysical perspective and we speculate on their underlying neurophysiological substrate.

Is there a pop-out of exclusively binocular (cyclopean) contours and regions?

In this work, contours and texture regions were designed so that they could not be observed by either eye alone, but only after images from both eyes were fused into a single stereoscopic picture. Two planes of random dots were positioned one in front of the other so that their random-dot patterns were transparently overlaid. In this way the dot pattern in the front plane was completely masked by the dots in the back plane for either eye on its own. Such exclusively binocular or 'cyclopean' stimuli are therefore defined by a conjunction of depth information with other basic visual features. It is shown that, unlike their monocular counterparts, cyclopean contours do not pop-out. It is also shown that cyclopean regions pop-out only when they have either much higher or much lower luminance contrast than their surroundings, or (for some observers) when the cyclopean region is defined by motion contrast. Colour, luminance-contrast sign, and orientation-defined regions are not easily detected even when viewed attentively.

Integration of first- and second-order orientation

The problem of how visual information such as orientation is combined across space bears on key visual abiities, such as texture perception. Orientation signals can be derived from both luminance and contrast, but it is not well understood how such information is pooled or how these different orientation signals interact in the integration process. We measured orientation discrimination thresholds for arrays of equivisible first-order and second-order Gabors. Thresholds were measured as the orientation variability in the arrays increased, and we estimated the number of samples (or efficiency) and internal noise of the mechanism being used. Observers were able to judge the mean orientation of arrays of either first- or second-order Gabors. For arrays of first-order and arrays of second-order Gabors, estimates of the number of samples used increased as the number of Gabors increased. When judging the orientation of arrays of either order, observers were able to ignore randomly oriented Gabors of the opposite order. If observers did not know which Gabor type carried the more useful orientation information, they tended to use the information from first-order Gabors (even when this was poorer information). Observers were unable to combine information from first- and second-order Gabors, though this would have improved their performance. The visual system appears to have separate integrators for combining local orientation across space for luminance- and contrast-defined features.

Longer VEP latencies and slower reaction times to the onset of second-order motion than to the onset of first-order motion

We compared visual evoked potentials and psychophysical reaction times to the onset of first- and second-order motion. The stimuli consisted of luminance-modulated (first-order) and contrast-modulated (second-order) 1 cpd vertical sine-wave gratings drifting rightward for 140 ms at a velocity of 6 degrees /s. For each condition, we analysed the latencies and peak-to-baseline amplitudes of the P1 and N2 peaks recorded at Oz. For first-order motion, both P1 and N2 peaks were present at low (3%) contrast (i.e., depth modulations) whereas for second-order motion they appeared only at higher (25%) contrasts. When the two types of motion were equated for visibility, responses were slower for second-order motion than for first-order motion: about 44 ms slower for P1 latencies, 53 ms slower for N2 latencies, and 76 ms slower for reaction times. The longer VEP latencies for second-order motion support models that postulate additional processing steps for the extraction of second-order motion. The slower reaction time to the onset of second-order motion suggests that the longer neurophysiological analysis translates into slower detection.