Some tests of the Marr-Ullman model of movement detection

Predicting individual neuron responses with anatomically constrained task optimization

Artificial neural networks trained to solve sensory tasks can develop statistical representations that match those in biological circuits. However, it remains unclear whether they can reproduce properties of individual neurons. Here, we investigated how artificial networks predict individual neuron properties in the visual motion circuits of the fruit fly Drosophila. We trained anatomically constrained networks to predict movement in natural scenes, solving the same inference problem as fly motion detectors. Units in the artificial networks adopted many properties of analogous individual neurons, even though they were not explicitly trained to match these properties. Among these properties was the split into ON and OFF motion detectors, which is not predicted by classical motion detection models. The match between model and neurons was closest when models were trained to be robust to noise. These results demonstrate how anatomical, task, and noise constraints can explain properties of individual neurons in a small neural network.

Flies and humans share a motion estimation strategy that exploits natural scene statistics

Sighted animals extract motion information from visual scenes by processing spatiotemporal patterns of light falling on the retina. The dominant models for motion estimation exploit intensity correlations only between pairs of points in space and time. Moving natural scenes, however, contain more complex correlations. We found that fly and human visual systems encode the combined direction and contrast polarity of moving edges using triple correlations that enhance motion estimation in natural environments. Both species extracted triple correlations with neural substrates tuned for light or dark edges, and sensitivity to specific triple correlations was retained even as light and dark edge motion signals were combined. Thus, both species separately process light and dark image contrasts to capture motion signatures that can improve estimation accuracy. This convergence argues that statistical structures in natural scenes have greatly affected visual processing, driving a common computational strategy over 500 million years of evolution.

Motion detection in interleaved random dot patterns: evidence for a rectifying nonlinearity preceding motion analysis

Three experiments examined direction discrimination in temporally interleaved random dot patterns. The stimulus consisted of two or more uncorrelated random patterns presented in a repeating temporal sequence, so that each pattern appeared only once every n frames, separated by uncorrelated patterns. Each pattern shifted either leftward or rightward at each re-appearance (all patterns shifted in the same direction in any one presentation). Subjects could specify shift direction correctly even when eight different patterns were interleaved, provided that the duration of each frame was brief. An explanation based on responses in first-order motion energy detectors tuned to low spatiotemporal frequencies (effectively summating the interleaved patterns over time) was tested using a stimulus in which each pattern inverted in contrast mid-way through each frame. Contrary to predictions based on temporal summation, performance with contrast-inverting patterns was only slightly lower than with non-inverting patterns. An alternative explanation was examined, based on responses in motion detectors that full-wave rectify image contrast before extracting motion energy. Computed responses from such detectors successfully predicted psychophysical performance with interleaved random patterns. Implications for models of motion analysis are discussed.

Experiments on the afterimages of stimulus change (Dvorák 1870): a translation with commentary

In 1870 Dvorák rejected Helmholtz's eye-movement account of motion aftereffects (MAEs) on the grounds that it was inconsistent with previous reports of nonuniform rotation in MAEs induced with Plateau spirals. Subsequent observations with spirals that were modified to induce both expanding and contracting MAEs simultaneously, together with the use of stationary negative afterimages during induction and test, were offered as further counter-examples to the eye-movement hypothesis. Dvorák's conjectures that perception (and misperception) of movement involves a unitary perceptual dimension of stimulus change also led him to investigate whether aftereffects comparable to MAEs could be induced along other stimulus dimensions in vision (luminance gradients), and in audition (gradients of pitch and intensity). It is suggested that Dvorák's observations, taken as a whole, may be interpreted as an attempt to provide evidence challenging the Helmholtzian traditions underpinning eye-movement accounts of MAEs. The nature and outcomes of these observations are provided in a translation of the original work, and are subsequently discussed in relation to some contemporary empirical counterparts.

The dimensionality of texture-defined motion: a single channel theory

We examine apparent motion carried by textural properties. The texture stimuli consist of a sequence of grating patches of various spatial frequencies and amplitudes. Phases are randomized between frames to insure that first-order motion mechanisms directly applied to stimulus luminance are not systematically engaged. We use ambiguous apparent motion displays in which a heterogeneous motion path defined by alternating patches of texture s (standard) and texture v (variable) competes with a homogeneous motion path defined solely by patches of texture s. Our results support a one-dimensional (single-channel) model of motion-from-texture in which motion strength is computed from a single spatial transformation of the stimulus--an activity transformation. The value assigned to a point in space-time by this activity transformation is directly proportional to the modulation amplitude of the local texture and inversely proportional to local spatial frequency (within the range of spatial frequencies examined). The activity transformation is modeled as the rectified output of a low-pass spatial filter applied to stimulus contrast. Our data further suggest that the strength of texture-defined motion between a patch of texture s and a patch of texture v is proportional to the product of the activities of s and v. A strongly counterintuitive prediction of this model borne out in our data is that motion between patches of different texture can be stronger than motion between patches of similar texture (e.g. motion between patches of a low contrast, low frequency texture 1 and patches of high contrast, high frequency texture h can be stronger than motion between patches of similar texture h).

Properties of the visual channels that underlie adaptation to gradual change of luminance

Following adaptation to a spatially uniform patch of light that is gradually brightening (or dimming), a steady test patch appears to be gradually dimming (or brightening). We measured this ramp aftereffect with a nulling method, as a function of the amplitude and temporal repetition rate of the adapting sawtooth waveform and at various retinal eccentricities and levels of dark adaptation. We conclude that the underlying visual channels respond best to large-amplitude sweeps in luminance of at least 20 dB (1 log unit); but they are fairly insensitive to the temporal rate of this sweep. The channels are present out to an eccentricity of at least 40 degrees but they almost disappear during dark adaptation. The ramp aftereffects were asymmetrical: the subjectively darkening aftereffect produced by a brightening adapting ramp was slightly stronger than vice versa.

Polarity specific adaptation to motion in the human visual system

Three experiments investigated polarity specific adaptation to movement. Experiment 1 tested for temporal polarity specific adaptation, using counterphase sawtooth gratings as adapting and test stimuli. Each counterphase grating contained oppositely moving sawtooth components, and was thus balanced for direction, but both components of the adapting grating created only one polarity of luminance change over time, whereas the components of the test grating presented different signs. After adaptation, only the test component containing the unadapted temporal change was visible. A second experiment, using an analogous procedure, found evidence for spatial polarity specific adaptation. Experimental results can be explained by motion detectors which preserve information about spatial and temporal polarity. A third experiment found that spatial and temporal polarity specific adaptation differ in their dependence on temporal frequency.

Visual processing of rotary motion

Local descriptions of velocity fields (e.g., rotation, divergence, and deformation) contain a wealth of information for form perception and ego motion. In spite of this, human psychophysical performance in estimating these entities has not yet been thoroughly examined. In this paper, we report on the visual discrimination of rotary motion. A sequence of image frames is used to elicit an apparent rotation of an annulus, composed of dots in the frontoparallel plane, around a fixation spot at the center of the annulus. Differential angular velocity thresholds are measured as a function of the angular velocity, the diameter of the annulus, the number of dots, the display time per frame, and the number of frames. The results show a U-shaped dependence of angular velocity discrimination on spatial scale, with minimal Weber fractions of 7%. Experiments with a scatter in the distance of the individual dots to the center of rotation demonstrate that angular velocity cannot be assessed directly; perceived angular velocity depends strongly on the distance of the dots relative to the center of rotation. We suggest that the estimation of rotary motion is mediated by local estimations of linear velocity.

Neural gradient models for the measurement of image velocity

Although gradient schemes for detecting the motion of images and measuring their velocities are commonly used in computer vision, and although there is increasing evidence to support the existence of such schemes in biological vision, little attention has been directed to suggesting how such computations might be realized by neural hardware. This paper proposes two simple models, consisting of physiologically realistic networks of neurons, that approximate the gradient scheme. Computer simulations demonstrate that the models measure the speed of an object or pattern independently of its structural properties.

Motion aftereffects from a motionless stimulus

Dimming or brightening regions superimposed, slightly out of register, on static light or dark blobs, give rise to apparent motion. When these regions are replaced by apparent brightening or dimming produced by ramp aftereffects, a directional motion aftereffect is perceived. It is concluded that filters sensitive to temporal derivative signals of net brightening or dimming provide an input into the motion pathways.

Some additional predictions and further tests of the Marr-Ullman model of motion perception

The Marr-Ullman model for motion detection in the human visual system functions by means of the dual input of polarity-specific edge detectors and luminance change detectors. Moulden and Begg (1986) found a polarity-specific motion aftereffect which they claimed provided support for this dual input model. The logic of their experiment is examined, and it is shown that several additional predictions arise from the Marr-Ullman model, which were not supported by Moulden and Begg's study. A more powerful experiment was carried out and these additional predictions were disconfirmed, although the polarity-specific effect did emerge. A consideration of alternative explanations of this effect led to a second experiment in which an attempt was made to discover the actual determinants of the effect. This revealed that polarity-specific units are unlikely to play any part in the phenomenon. It was concluded, in the light of this and other evidence, that one of a class of alternative models is more likely to be the actual mechanism for motion perception. However, careful consideration of the Marr-Ullman model indicated that it may be untestable in principle if various differentially weighted levels of neural integration are envisaged.

Principles of visual motion detection

Motion information is required for the solution of many complex tasks of the visual system such as depth perception by motion parallax and figure/ground discrimination by relative motion. However, motion information is not explicitly encoded at the level of the retinal input. Instead, it has to be computed from the time-dependent brightness patterns of the retinal image as sensed by the two-dimensional array of photoreceptors. Different models have been proposed which describe the neural computations underlying motion detection in various ways. To what extent do biological motion detectors approximate any of these models? As will be argued here, there is increasing evidence from the different disciplines studying biological motion vision, that, throughout the animal kingdom ranging from invertebrates to vertebrates including man, the mechanisms underlying motion detection can be attributed to only a few, essentially equivalent computational principles. Motion detection may, therefore, be one of the first examples in computational neurosciences where common principles can be found not only at the cellular level (e.g., dendritic integration, spike propagation, synaptic transmission) but also at the level of computations performed by small neural networks.

Motion-detecting mechanisms: dual input devices?

Displacement thresholds for luminance step edges were measured for a wide range of contrasts and mean luminances. Thresholds for extended edges (longer than about 0.5°) are determined not by contrast but rather by the amplitude (Lmax-Lmin) of the luminance change produced by the displacement. Arguing from the standpoint of the Marr-Ullman model of movement detection, we had expected that thresholds might be jointly determined by both contrast and amplitude. Using a range of edges of different lengths, we found that differential effects of luminance and contrast can be revealed: for short edges (less than about 0.5°) thresholds are influenced by both amplitude and contrast, while for more extensive edges only amplitude has an influence. The results are consistent with the properties of a mechanism that has two separate inputs, one from a spatial operator that is contrast-dependent and one from a temporal operator that is amplitude-dependent. The spatial operator is markedly sensitive to changes in edge extent, the temporal operator much less so. The output of the spatial operator saturates early as a function of contrast.

Illusory continuous motion from oscillating positive-negative patterns: implications for motion perception

A black and white (positive) grating pattern was superimposed in exact register on its own photographic negative. Four operations were repetitively applied to this positive pattern so that it moved fractionally to the right, grew dimmer, moved back to the left, and grew brighter again. This sequence produced a strong illusion of continuous apparent motion to the right for as long as the cycle was repeated. The small relative motion between the two patterns generated two new illusory effects: enhanced real movement (ERM) and reversed real movement (RRM). The dimming and brightening phases gave rise to reversed apparent movement (RAM). All three effects are attributed to spatial filtering by neural mechanisms, which shifts the effective position of the positive-negative contours.