Restricted ability to recover three-dimensional global motion from one-dimensional local signals: theoretical observations

Nonuniform surround suppression of visual responses in mouse V1

Complex receptive field characteristics, distributed across a population of neurons, are thought to be critical for solving perceptual inference problems that arise during motion and image segmentation. For example, in a class of neurons referred to as "end-stopped," increasing the length of stimuli outside of the bar-responsive region into the surround suppresses responsiveness. It is unknown whether these properties exist for receptive field surrounds in the mouse. We examined surround modulation in layer 2/3 neurons of the primary visual cortex in mice using two-photon calcium imaging. We found that surround suppression was significantly asymmetric in 17% of the visually responsive neurons examined. Furthermore, the magnitude of asymmetry was correlated with orientation selectivity. Our results demonstrate that neurons in mouse primary visual cortex are differentially sensitive to the addition of elements in the surround and that individual neurons can be described as being either uniformly suppressed by the surround, end-stopped, or side-stopped. NEW & NOTEWORTHY Perception of visual scenes requires active integration of both local and global features to successfully segment objects from the background. Although the underlying circuitry and development of perceptual inference is not well understood, converging evidence indicates that asymmetry and diversity in surround modulation are likely fundamental for these computations. We determined that these key features are present in the mouse. Our results support the mouse as a model to explore the neural basis and development of surround modulation as it relates to perceptual inference.

Seeing via Miniature Eye Movements: A Dynamic Hypothesis for Vision

During natural viewing, the eyes are never still. Even during fixation, miniature movements of the eyes move the retinal image across tens of foveal photoreceptors. Most theories of vision implicitly assume that the visual system ignores these movements and somehow overcomes the resulting smearing. However, evidence has accumulated to indicate that fixational eye movements cannot be ignored by the visual system if fine spatial details are to be resolved. We argue that the only way the visual system can achieve its high resolution given its fixational movements is by seeing via these movements. Seeing via eye movements also eliminates the instability of the image, which would be induced by them otherwise. Here we present a hypothesis for vision, in which coarse details are spatially encoded in gaze-related coordinates, and fine spatial details are temporally encoded in relative retinal coordinates. The temporal encoding presented here achieves its highest resolution by encoding along the elongated axes of simple-cell receptive fields and not across these axes as suggested by spatial models of vision. According to our hypothesis, fine details of shape are encoded by inter-receptor temporal phases, texture by instantaneous intra-burst rates of individual receptors, and motion by inter-burst temporal frequencies. We further describe the ability of the visual system to readout the encoded information and recode it internally. We show how reading out of retinal signals can be facilitated by neuronal phase-locked loops (NPLLs), which lock to the retinal jitter; this locking enables recoding of motion information and temporal framing of shape and texture processing. A possible implementation of this locking-and-recoding process by specific thalamocortical loops is suggested. Overall it is suggested that high-acuity vision is based primarily on temporal mechanisms of the sort presented here and low-acuity vision is based primarily on spatial mechanisms.

Integration of Contour and Terminator Signals in Visual Area MT of Alert Macaque

The integration of visual information is a critical task that is performed by neurons in the extrastriate cortex of the primate brain. For motion signals, integration is complicated by the geometry of the visual world, which renders some velocity measurements ambiguous and others incorrect. The ambiguity arises because neurons in the early stages of visual processing have small receptive fields, which can only recover the component of motion perpendicular to the orientation of a contour (the aperture problem). Unambiguous motion signals are located at end points and corners, which are referred to as terminators. However, when an object moves behind an occluding surface, motion measurements made at the terminators formed by the intersection of the object and the occluder are generally not consistent with the direction of object motion. To study how cortical neurons integrate these different motion cues, we used variations on the classic "barber pole" stimulus and measured the responses of neurons in the middle temporal area (MT or V5) of extrastriate cortex of alert macaque monkeys. Our results show that MT neurons are more strongly influenced by the unambiguous motion signals generated by terminators than to the ambiguous signals generated by contours. Furthermore, these neurons respond better to terminators that are intrinsic to a moving object than to those that are accidents of occlusion. V1 neurons show similar response patterns to local cues (contours and terminators), but for large stimuli, they do not reflect the global motion direction computed by MT neurons. These observations are consistent with psychophysical findings that show that our perception of moving objects often depends on the motion of terminators.

End-stopping and the aperture problem: two-dimensional motion signals in macaque V1

Our perception of fine visual detail relies on small receptive fields at early stages of visual processing. However, small receptive fields tend to confound the orientation and velocity of moving edges, leading to ambiguous or inaccurate motion measurements (the aperture problem). Thus, it is often assumed that neurons in primary visual cortex (V1) carry only ambiguous motion information. Here we show that a subpopulation of V1 neurons is capable of signaling motion direction in a manner that is independent of contour orientation. Specifically, end-stopped V1 neurons obtain accurate motion measurements by responding only to the endpoints of long contours, a strategy which renders them largely immune to the aperture problem. Furthermore, the time course of end-stopping is similar to the time course of motion integration by MT neurons. These results suggest that cortical neurons might represent object motion by responding selectively to two-dimensional discontinuities in the visual scene.

Segmentation in structure from motion: modeling and psychophysics

Much work has been done on the question of how the visual system extracts the three-dimensional (3D) structure and motion of an object from two-dimensional (2D) motion information, a problem known as 'Structure from Motion', or SFM. Much less is known, however, about the human ability to recover structure and motion when the optic flow field arises from multiple objects, although observations of this ability date as early as Ullman's well-known two-cylinders stimulus [The interpretation of visual motion (1979)]. In the presence of multiple objects, the SFM problem is further aggravated by the need to solve the segmentation problem, i.e. deciding which motion signal belongs to which object. Here, we present a model for how the human visual system solves the combined SFM and segmentation problems, which we term SSFM, concurrently. The model is based on computation of a simple scalar property of the optic flow field known as def, which was previously shown to be used by human observers in SFM. The def values of many triplets of moving dots are computed, and the identification of multiple objects the image is based on detecting multiple peaks in the histogram of def values. In five experiments, we show that human SSFM performance is consistent with the predictions of the model. We compare the predictions of our model to those of other theoretical approaches, in particular those that use a rigidity hypothesis, and discuss the validity of each approach as a model for human SSFM.

Pattern and component motion selectivity in cortical area PMLS of the cat

Visual motion perception is one of the most prominent functions performed by the mammalian cerebral cortex. The moving images are commonly considered to be processed in two stages. The first-stage neurons are sensitive to the motion of one-dimensional orientated components, and their outputs are combined at the second stage to perceive the global motion of the whole pattern. Alternatively, the pattern motion may be signalled by monitoring a distinctive feature of the image, such as a line-end or a corner. In the present study, a series of 'random-line' patterns were used to measure the direction-tuning responses of 138 neurons in the posteromedial lateral suprasylvian area of the cat. The novel stimuli comprised identical thin line segments, with a length : width ratio no less than 10 : 1, which were moved perpendicularly or obliquely to their common orientation during the recordings. When the component lines were much shorter than the size of receptive field, the majority of cells were selective to the direction of pattern motion while only a small subset was sensitive to the direction of component motion. However, the response profiles of most cells became more component-motion selective with the increment of orientation element in stimulus by elongating the component lines in the patterns. These findings imply that the two-stage theory might be incomplete for modelling the visual motion analysis. Even at relatively low levels of the visual system, some kind of nonorientation-based processing may coexist with the orientation-sensitive processing in a dynamic competition, where one rises as the other falls depending upon the strength of the orientation element in the stimulus, so that under some circumstances it becomes possible to signal the veridical direction of pattern motion.

Discriminating the volume of motion-defined solids

We investigated the ability of human observers to discriminate an important global 3-D structural property, namely volume, of motion-defined objects. We used convex transparent wire-frame objects consisting of about 12 planar triangular facets. Two objects, vertically separated by 7 degrees, were shown simultaneously on a computer display. Both revolved at 67 degrees/sec around a common vertical axis through their centers of mass. Observers watched the objects monocularly for an average of three full rotations before they responded. We measured volume discrimination as a function of absolute volume (3-48 cm3; 1 m viewing distance) and shape (cubes, rods, and slabs of different regularity). We found that (1) volume discrimination performance can be described by Weber's law, (2) Weber fractions depend strongly on the particular combination of shapes used (regular shapes, especially cubes, are easiest to compare, and similar shapes are easier to compare than different shapes), and (3) humans use a representation of volume that is more veridical and stable in the sense of repeatability than a strategy based on the average visible (2-D) area would yield.

Disambiguating velocity estimates across image space

A translating homogeneous edge viewed through an aperture is an ambiguous stimulus, while a translating edge discontinuity is unambiguous. Under what conditions does the visual system use unambiguous velocity estimates to interpret ambiguous velocity estimates? We considered a translating rectangle visible through a set of stationary apertures. One aperture displayed a rectangle edge while the other apertures displayed corners. Observers reported the direction in which the edge appeared to translate. The results suggest that collinearity and terminator proximity determine whether the unambiguous corner velocity was used to interpret the ambiguous edge velocity. These results suggest some of the ways in which the visual system controls the integration of velocity estimates across image space.

Extraction of relief from visual motion

We quantified the ability of human subjects to discriminate the relative distance of two points from a slanted plane when viewing the projected velocities of this scene (orthographic projection). The relative distance from a plane (called relief) is a 3-D property that is invariant under linear (affine) transformations. As such, relief can in principle be extracted from the instantaneous projected velocity field; a metric representation, which requires the extraction of visual acceleration, is not required. The stimulus consisted of a slanted plane P (specified by three points) and two points Q1 and Q2 that are non-coplanar with P. This configuration of points oscillated rigidly around the vertical axis. We have measured the systematic error and accuracy with which human subjects estimate the relative distance of points Q1 and Q2 from plane P as a function of the slant of P. The systematic error varies with slant: it is low for small slant values, reaches a maximum for medium slant values, and drops again for high slant values. The accuracy covaries with the systematic error and is thus high for small and large slant values and low for medium slant values. These results are successfully modeled by a simple relief-from-motion computation based on local estimates of projected velocities. The data are well predicted by assuming (1) a measurement error in velocity estimation that varies proportionally to velocity (Weber's law) and (2) an eccentricity-dependent underestimation of velocity.

Restricted ability to recover three-dimensional global motion from one-dimensional motion signals: psychophysical observations

We tested human ability to recover the 3D structure and motion information from time-varying images where only 1D motion cues were available. Under these conditions, observers exhibit poor performance in discriminating between two perpendicular axes of rotation, or discriminating between rigid and non-rigid 3D motion. This behavior of the visual system is to be contrasted with the good depth from motion performance exhibited when 2D motion cues are given in the image, as was found previously in numerous studies, and also in the work presented here. In a related paper, we suggest a theoretical framework in which to understand this differential performance on the basis of the two types of motion cues (1D vs 2D). Our findings are consistent with those of previous studies of frontoparallel motion, where it was shown that in many cases, the 1D cues alone were not integrated by the visual system into the correct global motion percept. This accumulating evidence suggests that oriented (1D) motion detectors alone cannot account for observed human performance of global motion perception, and that the role of units such as point or endpoint detectors should be studied further.