Intrinsic two-dimensional features as textons

Features and the 'primal sketch'

This review is concerned primarily with psychophysical and physiological evidence relevant to the question of the existence of spatial features or spatial primitives in human vision. The review will be almost exclusively confined to features defined in the luminance domain. The emphasis will be on the experimental and computational methods that have been used for revealing features, rather than on a detailed comparison between different models of feature extraction. Color and texture fall largely outside the scope of the review, though the principles may be similar. Stereo matching and motion matching are also largely excluded because they are covered in other contributions to this volume, although both have addressed the question of the spatial primitives involved in matching. Similarities between different psychophysically-based model will be emphasized rather than minor differences. All the models considered in the review are based on the extraction of directional spatial derivatives of the luminance profile, typically the first and second, but in one case the third order, and all have some form of non-linearity, be it rectification or thresholding.

BOLD activation varies parametrically with corner angle throughout human retinotopic cortex

The Alternating Brightness Star (ABS) is an illusion that provides insight into the relationship between brightness perception and corner angle. Recent psychophysical studies of this illusion have shown that corner salience varies parametrically with corner angle, with sharp angles leading to strong illusory percepts and shallow angles leading to weak percepts. It is hypothesized that the illusory effects arise because of an interaction between surface corners and the shape of visual receptive fields: sharp surface corners may create hotspots of high local contrast due to processing by center-surround and other early receptive fields. If this hypothesis is correct, early visual neurons should respond powerfully to sharp corners and curved portions of surface edges. Indeed, the primary role of early visual neurons may be to localize the discontinuities along the edges of surfaces. If so, all early visual areas should show greater BOLD responses to sharp corners than to shallow corners. On the other hand, if corner processing is exclusively constrained to certain areas of the brain, only those specific areas will show greater responses to sharp vs shallow corners. To address this we explored the BOLD correlates of the ABS illusion in the human visual cortex using fMRI. We found that BOLD signal varies parametrically with corner angle throughout the visual cortex, offering the first neurophysiological correlates of the ABS illusion. This finding provides a neurophysiological basis for the previously reported psychophysical data that showed that corner salience varied parametrically with corner angle. We propose that all early visual areas localize discontinuities along the edges of surfaces, and that specific cortical corner-processing circuits further establish the specific nature of those discontinuities, such as their orientation.

Beyond fourth-order texture discrimination: generation of extreme-order and statistically-balanced textures

Julesz introduced the concept of statistically defined textures and their perceptual discrimination. Julesz showed that discrimination was possible with statistics equated to third-order, specifying fourth-order textures. Klein and Tyler offered a variety of paradigms suggesting that fourth order might be the limit on human texture processing. To go beyond this limit, new texture paradigms are now introduced to avoid contamination by luminance extrema, to control local and long-range texture properties, and to provide textures without global statistical structure. Local luminance contamination is avoided by novel orientation plaids, in which higher-order rules govern the orientation of local elements rather than their coloring. These textures allow evaluation of texture discrimination up to thirty-second order by cortical pattern elements. Long-range processing is studied by random strip rotation and by interlacing of independent textures. Each substantially degrades the visibility of the fourth-order textures, revealing that the fourth-order information is conveyed largely by local rather than long-range perturbations from random statistics. Finally, textures equated at all orders can be defined in terms of their global statistics, but may nevertheless readily be discriminated in human vision. The discrimination on the basis of local perturbations implies that human vision assesses textures through a local sampling window, and is largely insensitive to longer-range statistical properties.

A spectral histogram model for texton modeling and texture discrimination

We suggest a spectral histogram, defined as the marginal distribution of filter responses, as a quantitative definition for a texton pattern. By matching spectral histograms, an arbitrary image can be transformed to an image with similar textons to the observed. We use the chi(2)-statistic to measure the difference between two spectral histograms, which leads to a texture discrimination model. The performance of the model well matches psychophysical results on a systematic set of texture discrimination data and it exhibits the nonlinearity and asymmetry phenomena in human texture discrimination. A quantitative comparison with the Malik-Perona model is given, and a number of issues regarding the model are discussed.

Invariance of the psychometric function for character recognition across the visual field

The psychometric function for recognition of singly presented digits as a function of digit contrast was measured at 2 degrees steps across the horizontal meridian of the visual field, under monocular and binocular viewing conditions. A maximum-likelihood staircase procedure was used in a 10-alternative forced-choice recognition paradigm to gather the data Both the Weibull and the logistic psychometric functions provide excellent fits to the observed data. The slopes of these functions at their point of inflection ranged from 4.0 to 5.0 proportion-correct/log10-unit contrast, for both monocular and binocular viewing and for all loci in the visual field. These slope values correspond to short-term measurements (around 30 trials, or 1 min) and do not include performance variations of longer duration; the latter are estimated to increase slope by a factor of about 1.5. A single psychometric function shape, centered around a threshold value, therefore describes recognition performance at all retinal loci and binocularity. An empirical comparison of slope results across the literature shows that the function's slope is about twice that reported for a number of detection tasks. The comparison of recognition contrast thresholds, percentage correct values, and other performance measures across studies requires the knowledge of the psychometric function's slope, and our results thus provide a firm basis for the study of low-contrast character recognition.

Inferior temporal neurons show greater sensitivity to nonaccidental than to metric shape differences

It has long been known that macaque inferior temporal (IT) neurons tend to fire more strongly to some shapes than to others, and that different IT neurons can show markedly different shape preferences. Beyond the discovery that these preferences can be elicited by features of moderate complexity, no general principle of (nonface) object recognition had emerged by which this enormous variation in selectivity could be understood. Psychophysical, as well as computational work, suggests that one such principle is the difference between viewpoint-invariant, nonaccidental (NAP) and view-dependent, metric shape properties (MPs). We measured the responses of single IT neurons to objects differing in either a NAP (namely, a change in a geon) or an MP of a single part, shown at two orientations in depth. The cells were more sensitive to changes in NAPs than in MPs, even though the image variation (as assessed by wavelet-like measures) produced by the former were smaller than the latter. The magnitude of the response modulation from the rotation itself was, on average, similar to that produced by the NAP differences, although the image changes from the rotation were much greater than that produced by NAP differences. Multidimensional scaling of the neural responses indicated a NAP/MP dimension, independent of an orientation dimension. The present results thus demonstrate that a significant portion of the neural code of IT cells represents differences in NAPs rather than MPs. This code may enable immediate recognition of novel objects at new views.

From image segmentation to anti-textons

We apply the 'patchwork engine' (PE; van Tonder and Ejima, 2000 Neural Networks forthcoming) to encode spaces between textons in an attempt to find a suitable feature representation of anti-textons [Williams and Julesz, 1991, in Neural Networks for Perception volume 1: Human and Machine Perception Ed. H Wechsler (San Diego, CA: Academic Press); 1992, Proceedings of the National Academy of Sciences of the USA 89 6531-6534]. With computed anti-textons it is possible to show that tessellation and distribution of anti-textons can differ from that of textons depending on the ratio of texton size to anti-texton size. From this we hypothesise that variability of anti-textons can enhance texture segregation, and test our hypothesis in two psychophysical experiments. Texture segregation asymmetry is the topic of the first test. We found that targets on backgrounds with regular anti-textons segregate more strongly than on backgrounds with highly variable anti-textons. This neatly complements other explanations for texture segregation asymmetry (e.g. Rubenstein and Sagi, 1990 Journal of the Optical Society of America A 7 1632-1643). Second the relative significance of textons and anti-textons in human texture segregation is investigated for a limited set of texture patterns. Subjects consistently judged a combination of texton and anti-texton gradients as more conspicuous than texton-only gradients, and judged texton-only gradients as being more conspicuous than anti-texton-only gradients. In the absence of strong texton gradients the regularity versus irregularity of anti-textons agrees with perceived texture segregation. Using PE outputs as anti-texton features thus enabled the conception of various useful tests on texture segregation. The PE is originally intended as a general image segmentation method based on symmetry axes. With this paper we therefore hope to relate anti-textons with visual processing in a wider sense.

Computational modelling of visual attention

Five important trends have emerged from recent work on computational models of focal visual attention that emphasize the bottom-up, image-based control of attentional deployment. First, the perceptual saliency of stimuli critically depends on the surrounding context. Second, a unique 'saliency map' that topographically encodes for stimulus conspicuity over the visual scene has proved to be an efficient and plausible bottom-up control strategy. Third, inhibition of return, the process by which the currently attended location is prevented from being attended again, is a crucial element of attentional deployment. Fourth, attention and eye movements tightly interplay, posing computational challenges with respect to the coordinate system used to control attention. And last, scene understanding and object recognition strongly constrain the selection of attended locations. Insights from these five key areas provide a framework for a computational and neurobiological understanding of visual attention.

A geometric framework for nonlinear visual coding

It is argued that important aspects of early and middle level visual coding may be understood as resulting from basic geometric processing of the visual input. The input is treated as a hypersurface defined by image intensity as a function of two spatial coordinates and time. Analytical results show how the Riemann curvature tensor R of this hypersurface represents speed and direction of motion. Moreover, the results can predict the selectivity of MT neurons for multiple motions and for motion in a direction along the optimal spatial orientation. Finally, a model based on integrated R components predicts global-motion percepts related to the barber-pole illusion.