A Motion-from-Form Mechanism Contributes to Extracting Pattern Motion from Plaids

Transformation of Motion Pattern Selectivity from Retina to Superior Colliculus

The superior colliculus (SC) is a prominent and conserved visual center in all vertebrates. In mice, the most superficial lamina of the SC is enriched with neurons that are selective for the moving direction of visual stimuli. Here, we study how these direction selective neurons respond to complex motion patterns known as plaids, using two-photon calcium imaging in awake male and female mice. The plaid pattern consists of two superimposed sinusoidal gratings moving in different directions, giving an apparent pattern direction that lies between the directions of the two component gratings. Most direction selective neurons in the mouse SC respond robustly to the plaids and show a high selectivity for the moving direction of the plaid pattern but not of its components. Pattern motion selectivity is seen in both excitatory and inhibitory SC neurons and is especially prevalent in response to plaids with large cross angles between the two component gratings. However, retinal inputs to the SC are ambiguous in their selectivity to pattern versus component motion. Modeling suggests that pattern motion selectivity in the SC can arise from a nonlinear transformation of converging retinal inputs. In contrast, the prevalence of pattern motion selective neurons is not seen in the primary visual cortex (V1). These results demonstrate an interesting difference between the SC and V1 in motion processing and reveal the SC as an important site for encoding pattern motion.

A dynamical model of visual motion processing for arbitrary stimuli including type II plaids

To explore the operating principle of visual motion processing in the brain underlying perception and eye movements, we model the information processing of velocity estimate of the visual stimulus at the algorithmic level using the dynamical system approach. In this study, we formulate the model as an optimization process of an appropriately defined objective function. The model is applicable to arbitrary visual stimuli. We find that our theoretical predictions qualitatively agree with time evolution of eye movement reported by previous works across various types of stimulus. Our results suggest that the brain implements the present framework as the internal model of motion vision. We anticipate our model to be a promising building block for more profound understanding of visual motion processing as well as for the development of robotics.

Pattern Motion Direction Is Encoded in the Population Activity of Macaque Area MT

Direction selective neurons in macaque primary visual cortex are narrowly tuned for orientation, and are thus afflicted by the aperture problem. At the next stage of motion processing, in the middle temporal (MT) area, some cells appear to solve this problem, responding to the pattern motion direction of plaids. Models have been proposed to account for this computation, but they do not replicate the diversity of responses observed in MT. We recorded from 386 cells in area MT of two male macaques, while presenting a wide range of random-line stimuli and their compositions into noise plaids. As we broadened the range of stimuli used to probe the cells, yielding ever more challenging conditions for extracting pattern motion, the diversity of the responses observed increased, and the fraction of cells that faithfully encoded pattern motion direction shrank. However, we show here that a pattern motion signal is present at the population level. We identified four mechanisms, one never proposed before, that together might account for the observed diversity in single-cell responses. Pattern motion is thus extracted in area MT, but it is encoded across the population, and not in a small subset of pattern neurons.SIGNIFICANCE STATEMENT Some neurons in the middle temporal area of macaques solve the aperture problem, signaling the direction of motion of complex patterns. As the number of pattern types used to probe this mechanism is increased, fewer and fewer cells retain this capability. We show here that different cells fail in different ways, and that simply summing their responses averages away their failures, yielding a clear pattern motion signal. Similar encodings, which unequivocally violate the "neuron as a feature detector" hypothesis that has dominated sensory processing theories for the past 50 years, might apply throughout the brain.

Responses of neurons in macaque MT to unikinetic plaids

Response properties of MT neurons are often studied with "bikinetic" plaid stimuli, which consist of two superimposed sine wave gratings moving in different directions. Oculomotor studies using "unikinetic plaids" in which only one of the two superimposed gratings moves suggest that the eyes first move reflexively in the direction of the moving grating and only later converge on the perceived direction of the moving pattern. MT has been implicated as the source of visual signals that drives these responses. We wanted to know whether stationary gratings, which have little effect on MT cells when presented alone, would influence MT responses when paired with a moving grating. We recorded extracellularly from neurons in area MT and measured responses to stationary and moving gratings, and to their sums: bikinetic and unikinetic plaids. As expected, stationary gratings presented alone had a very modest influence on the activity of MT neurons. Responses to moving gratings and bikinetic plaids were similar to those previously reported and revealed cells selective for the motion of plaid patterns and of their components (pattern and component cells). When these neurons were probed with unikinetic plaids, pattern cells shifted their direction preferences in a way that revealed the influence of the static grating. Component cell preferences shifted little or not at all. These results support the notion that pattern-selective neurons in area MT integrate component motions that differ widely in speed, and that they do so in a way that is consistent with an intersection-of-constraints model.NEW & NOTEWORTHY Human perceptual and eye movement responses to moving gratings are influenced by adding a second, static grating to create a "unikinetic" plaid. Cells in MT do not respond to static gratings, but those gratings still influence the direction selectivity of some MT cells. The cells influenced by static gratings are those tuned for the motion of global patterns, but not those tuned only for the individual components of moving targets.

Motion and binocular disparity processing: Two sides of two different coins

From a mathematical point of view, extracting motion and disparity signals from a binocular visual stream requires very similar operations, applied over time for motion and across eyes for disparity. This similarity is reflected in the theories that have been proposed to describe the neural mechanisms used by the brain to extract these signals. At the behavioral level there are, however, several differences in how humans react to these stimuli, which presumably reflect differences in how these signals are processed by the brain. Here we highlight three such differences: the degree to which different axes of motion/disparity are treated isotropically, the importance of reference signals, and the rules that underlie the combination of 1D signals to extract 2D signals.

Binocular summation for reflexive eye movements

Psychophysical studies and our own subjective experience suggest that, in natural viewing conditions (i.e., at medium to high contrasts), monocularly and binocularly viewed scenes appear very similar, with the exception of the improved depth perception provided by stereopsis. This phenomenon is usually described as a lack of binocular summation. We show here that there is an exception to this rule: Ocular following eye movements induced by the sudden motion of a large stimulus, which we recorded from three human subjects, are much larger when both eyes see the moving stimulus, than when only one eye does. We further discovered that this binocular advantage is a function of the interocular correlation between the two monocular images: It is maximal when they are identical, and reduced when the two eyes are presented with different images. This is possible only if the neurons that underlie ocular following are sensitive to binocular disparity.

Suppression and Contrast Normalization in Motion Processing

Sensory neurons are activated by a range of stimuli to which they are said to be tuned. Usually, they are also suppressed by another set of stimuli that have little effect when presented in isolation. The interactions between preferred and suppressive stimuli are often quite complex and vary across neurons, even within a single area, making it difficult to infer their collective effect on behavioral responses mediated by activity across populations of neurons. Here, we investigated this issue by measuring, in human subjects (three males), the suppressive effect of static masks on the ocular following responses induced by moving stimuli. We found a wide range of effects, which depend in a nonlinear and nonseparable manner on the spatial frequency, contrast, and spatial location of both stimulus and mask. Under some conditions, the presence of the mask can be seen as scaling the contrast of the driving stimulus. Under other conditions, the effect is more complex, involving also a direct scaling of the behavioral response. All of this complexity at the behavioral level can be captured by a simple model in which stimulus and mask interact nonlinearly at two stages, one monocular and one binocular. The nature of the interactions is compatible with those observed at the level of single neurons in primates, usually broadly described as divisive normalization, without having to invoke any scaling mechanism.SIGNIFICANCE STATEMENT The response of sensory neurons to their preferred stimulus is often modulated by stimuli that are not effective when presented alone. Individual neurons can exhibit multiple modulatory effects, with considerable variability across neurons even in a single area. Such diversity has made it difficult to infer the impact of these modulatory mechanisms on behavioral responses. Here, we report the effects of a stationary mask on the reflexive eye movements induced by a moving stimulus. A model with two stages, each incorporating a divisive modulatory mechanism, reproduces our experimental results and suggests that qualitative variability of masking effects in cortical neurons might arise from differences in the extent to which such effects are inherited from earlier stages.

Combining 1-D components to extract pattern information: It is about more than component similarity

At least under some conditions, plaid stimuli are processed by combining information first extracted in orientation and scale-selective channels. The rules that govern this combination across channels are only partially understood. Although the available data suggests that only components having similar spatial frequency and contrast are combined, the extent to which this holds has not been firmly established. To address this question, we measured, in human subjects, the short-latency reflexive vergence eye movements induced by stereo plaids in which spatial frequency and contrast of the components are independently varied. We found that, although similarity in component spatial frequency and contrast matter, they interact in a nonseparable way. One way in which this relationship might arise is if the internal estimate of contrast is not a faithful representation of stimulus contrast but is instead spatial frequency–dependent (with higher spatial frequencies being boosted). We propose that such weighting might have been put in place by a mechanism that, in an effort of achieve contrast constancy and/or coding efficiency, regulates the gain of detectors in early visual cortex to equalize their long-term average response to natural images.