A processing stream in mammalian visual cortex neurons for non-Fourier responses

Influences of local and global context on local orientation perception

Visual context modulates perception of local orientation attributes. These spatially very localised effects are considered to correspond to specific excitatory-inhibitory connectivity patterns of early visual areas as V1, creating perceptual tilt repulsion and attraction effects. Here, orientation misperception of small Gabor stimuli was used as a probe of this computational structure by sampling a large spatio-orientation space to reveal expected asymmetries due to the underlying neuronal processing. Surprisingly, the results showed a regular iso-orientation pattern of nearby location effects whose reference point was globally modulated by the spatial structure, without any complex interactions between local positions and orientation. This pattern of results was confirmed by the two perceptual parameters of bias and discrimination ability. Furthermore, the response times to stimulus configuration displayed variations that further provided evidence of how multiple early visual stages affect perception of simple stimuli.

Resting state dynamics in people with varying degrees of anxiety and mindfulness: A nonlinear and nonstationary perspective

Anxiety and mindfulness are two inversely linked traits shown to be involved in various physiological domains. The current study used resting state electroencephalography (EEG) to explore differences between people with low mindfulness-high anxiety (LMHA) (n = 29) and high mindfulness-low anxiety (HMLA) (n = 27). The resting EEG was collected for a total of 6 min, with a randomized sequence of eyes closed and eyes opened conditions. Two advanced EEG analysis methods, Holo-Hilbert Spectral Analysis and Holo-Hilbert cross-frequency phase clustering (HHCFPC) were employed to estimate the power-based amplitude modulation of carrier frequencies, and cross-frequency coupling between low and high frequencies, respectively. The presence of higher oscillation power across the delta and theta frequencies in the LMHA group than the HMLA group might have been due to the similarity between the resting state and situations of uncertainty, which reportedly triggers motivational and emotional arousal. Although these two groups were formed based on their trait anxiety and trait mindfulness scores, it was anxiety that was found to be significant predictor of the EEG power, not mindfulness. It led us to conclude that it might be anxiety, not mindfulness, which might have contributed to higher electrophysiological arousal. Additionally, a higher δ-β and δ-γ CFC in LMHA suggested greater local-global neural integration, consequently a greater functional association between cortex and limbic system than in the HMLA group. The present cross-sectional study may guide future longitudinal studies on anxiety aiming with interventions such as mindfulness to characterize the individuals based on their resting state physiology.

Second-order texture gratings produce overestimation of height in depictions of rectangles and steps

The horizontal-vertical illusion (HVI) has been proposed as a method to increase the perceived height of steps, increase toe clearance and prevent falls. High contrast vertical stripes are placed on the step riser abutting a horizontal edge-highlighter creating 'T' junctions which are thought to promote the illusion. Various configurations of the HVI were tested including luminance gratings (L) and second-order modulations of contrast (CM), spatial frequency (FM) and orientation (OM). Observers were asked to compare the apparent height of gratings with that of either filled, unmodulated rectangles or unfilled rectangles. Rectangles were presented alone or as part of a step with a highlighter. In some conditions highlighters matched the properties of the grating; in others or not. In one critical experiment, the HVI was compared for steps with highlighters that were separated from the riser by a thin line and those where the risers and highlighters were continuous. All gratings except FM appeared taller when presented in the step configuration with a continuous, matching highlighter. This effect was greatly reduced when a thin line separated the grating from the highlighter and abolished for mis-matched highlighters and risers. In the rectangle conditions, all cues appeared taller than blank rectangles and L and CM appeared taller than filled-unmodulated rectangles. In conclusion, second-order cues may be useful for inducing the HVI onto steps. However, the ability of vertical stripes and edge-highlighters to accentuate perceived step height may be due to aggregation of the highlighter into the grating rather than the normal horizontal-vertical illusion.

Proto-Object Based Saliency Model With Texture Detection Channel

The amount of visual information projected from the retina to the brain exceeds the information processing capacity of the latter. Attention, therefore, functions as a filter to highlight important information at multiple stages of the visual pathway that requires further and more detailed analysis. Among other functions, this determines where to fixate since only the fovea allows for high resolution imaging. Visual saliency modeling, i.e. understanding how the brain selects important information to analyze further and to determine where to fixate next, is an important research topic in computational neuroscience and computer vision. Most existing bottom-up saliency models use low-level features such as intensity and color, while some models employ high-level features, like faces. However, little consideration has been given to mid-level features, such as texture, for visual saliency models. In this paper, we extend a biologically plausible proto-object based saliency model by adding simple texture channels which employ nonlinear operations that mimic the processing performed by primate visual cortex. The extended model shows statistically significant improved performance in predicting human fixations compared to the previous model. Comparing the performance of our model with others on publicly available benchmarking datasets, we find that our biologically plausible model matches the performance of other models, even though those were designed entirely for maximal performance with little regard to biological realism.

Integration of contours defined by second-order contrast-modulation of texture

Boundaries in the visual world can be defined by changes in luminance and texture in the input image. A "contour integration" process joins together local changes into percepts of lines or edges. A previous study tested the integration of contours defined by second-order contrast-modulation. Their contours were placed in a background of random wavelets. Participants performed near chance. We re-visited second-order contour integration with a different task. Participants distinguished contours with "good continuation" from distractors. We measured thresholds in different amounts of external orientation or position noise. This gave two noise-masking functions. We also measured thresholds for contours with a baseline curvature to assess performance with more curvy targets. Our participants were able to discriminate the good continuation of second-order contours. Thresholds were higher than for first-order contours. In our modelling, we found this was due to multiple factors. There was a doubling of equivalent internal noise between first- and second-order contour integration. There was also a reduction in efficiency. The efficiency difference was only significant in our orientation noise condition. For both first- and second-order stimuli, participants were also able to perform our task with more curved contours. We conclude that humans can integrate second-order contours, even when they are curved. There is however reduced performance compared to first-order contours. We find both an impaired input to the integrating mechanism, and reduced efficiency seem responsible. Second-order contour integration may be more affected by the noise background used in the previous study. Difficulty segregating that background may explain their result.

Unraveling nonlinear electrophysiologic processes in the human visual system with full dimension spectral analysis

Natural sensory signals have nonlinear structures dynamically composed of the carrier frequencies and the variation of the amplitude (i.e., envelope). How the human brain processes the envelope information is still poorly understood, largely due to the conventional analysis failing to quantify it directly. Here, we used a recently developed method, Holo-Hilbert spectral analysis, and steady-state visually evoked potential collected using electroencephalography (EEG) recordings to investigate how the human visual system processes the envelope of amplitude-modulated signals, in this case with a 14 Hz carrier and a 2 Hz envelope. The EEG results demonstrated that in addition to the fundamental stimulus frequencies, 4 Hz amplitude modulation residing in 14 Hz carrier and a broad range of carrier frequencies covering from 8 to 32 Hz modulated by 2 Hz amplitude modulation are also found in the two-dimensional frequency spectrum, which have not yet been recognized before. The envelope of the stimulus is also found to dominantly modulate the response to the incoming signal. The findings thus reveal that the electrophysiological response to amplitude-modulated stimuli is more complex than could be revealed by, for example, Fourier analysis. This highlights the dynamics of neural processes in the visual system.

Stimulus vignetting and orientation selectivity in human visual cortex

Neural selectivity to orientation is one of the simplest and most thoroughly-studied cortical sensory features. Here, we show that a large body of research that purported to measure orientation tuning may have in fact been inadvertently measuring sensitivity to second-order changes in luminance, a phenomenon we term 'vignetting'. Using a computational model of neural responses in primary visual cortex (V1), we demonstrate the impact of vignetting on simulated V1 responses. We then used the model to generate a set of predictions, which we confirmed with functional MRI experiments in human observers. Our results demonstrate that stimulus vignetting can wholly determine the orientation selectivity of responses in visual cortex measured at a macroscopic scale, and suggest a reinterpretation of a well-established literature on orientation processing in visual cortex.

More superimposition for contrast-modulated than luminance-modulated stimuli during binocular rivalry

Luminance-modulated noise (LM) and contrast-modulated noise (CM) gratings were presented with interocularly correlated, uncorrelated and anti-correlated binary noise to investigate their contributions to mixed percepts, specifically piecemeal and superimposition, during binocular rivalry. Stimuli were sine-wave gratings of 2 c/deg presented within 2 deg circular apertures. The LM stimulus contrast was 0.1 and the CM stimulus modulation depth was 1.0, equating to approximately 5 and 7 times detection threshold, respectively. Twelve 45 s trials, per noise configuration, were carried out. Fifteen participants with normal vision indicated via button presses whether an exclusive, piecemeal or superimposed percept was seen. For all noise conditions LM stimuli generated more exclusive visibility, and lower proportions of superimposition. CM stimuli led to greater proportions and longer periods of superimposition. For both stimulus types, correlated interocular noise generated more superimposition than did anti- or uncorrelated interocular noise. No significant effect of stimulus type (LM vs CM) or noise configuration (correlated, uncorrelated, anti-correlated) on piecemeal perception was found. Exclusive visibility was greater in proportion, and perceptual changes more numerous, during binocular rivalry for CM stimuli when interocular noise was not correlated. This suggests that mutual inhibition, initiated by non-correlated noise CM gratings, occurs between neurons processing luminance noise (first-order component), as well as those processing gratings (second-order component). Therefore, first- and second-order components can contribute to overall binocular rivalry responses. We suggest the addition of a new well to the current energy landscape model for binocular rivalry that takes superimposition into account.

Nonlinear Y-Like Receptive Fields in the Early Visual Cortex: An Intermediate Stage for Building Cue-Invariant Receptive Fields from Subcortical Y Cells

Many of the neurons in early visual cortex are selective for the orientation of boundaries defined by first-order cues (luminance) as well as second-order cues (contrast, texture). The neural circuit mechanism underlying this selectivity is still unclear, but some studies have proposed that it emerges from spatial nonlinearities of subcortical Y cells. To understand how inputs from the Y-cell pathway might be pooled to generate cue-invariant receptive fields, we recorded visual responses from single neurons in cat Area 18 using linear multielectrode arrays. We measured responses to drifting and contrast-reversing luminance gratings as well as contrast modulation gratings. We found that a large fraction of these neurons have nonoriented responses to gratings, similar to those of subcortical Y cells: they respond at the second harmonic (F2) to high-spatial frequency contrast-reversing gratings and at the first harmonic (F1) to low-spatial frequency drifting gratings ("Y-cell signature"). For a given neuron, spatial frequency tuning for linear (F1) and nonlinear (F2) responses is quite distinct, similar to orientation-selective cue-invariant neurons. Also, these neurons respond to contrast modulation gratings with selectivity for the carrier (texture) spatial frequency and, in some cases, orientation. Their receptive field properties suggest that they could serve as building blocks for orientation-selective cue-invariant neurons. We propose a circuit model that combines ON- and OFF-center cortical Y-like cells in an unbalanced push-pull manner to generate orientation-selective, cue-invariant receptive fields.

SIGNIFICANCE STATEMENT

A significant fraction of neurons in early visual cortex have specialized receptive fields that allow them to selectively respond to the orientation of boundaries that are invariant to the cue (luminance, contrast, texture, motion) that defines them. However, the neural mechanism to construct such versatile receptive fields remains unclear. Using multielectrode recording, we found a large fraction of neurons in early visual cortex with receptive fields not selective for orientation that have spatial nonlinearities like those of subcortical Y cells. These are strong candidates for building cue-invariant orientation-selective neurons; we present a neural circuit model that pools such neurons in an imbalanced "push-pull" manner, to generate orientation-selective cue-invariant receptive fields.

Invisible noise obscures visible signal in insect motion detection

The motion energy model is the standard account of motion detection in animals from beetles to humans. Despite this common basis, we show here that a difference in the early stages of visual processing between mammals and insects leads this model to make radically different behavioural predictions. In insects, early filtering is spatially lowpass, which makes the surprising prediction that motion detection can be impaired by "invisible" noise, i.e. noise at a spatial frequency that elicits no response when presented on its own as a signal. We confirm this prediction using the optomotor response of praying mantis Sphodromantis lineola. This does not occur in mammals, where spatially bandpass early filtering means that linear systems techniques, such as deriving channel sensitivity from masking functions, remain approximately valid. Counter-intuitive effects such as masking by invisible noise may occur in neural circuits wherever a nonlinearity is followed by a difference operation.

Envelope statistics of self-motion signals experienced by human subjects during everyday activities: Implications for vestibular processing

There is accumulating evidence that the brain's neural coding strategies are constrained by natural stimulus statistics. Here we investigated the statistics of the time varying envelope (i.e. a second-order stimulus attribute that is related to variance) of rotational and translational self-motion signals experienced by human subjects during everyday activities. We found that envelopes can reach large values across all six motion dimensions (~450 deg/s for rotations and ~4 G for translations). Unlike results obtained in other sensory modalities, the spectral power of envelope signals decreased slowly for low (< 2 Hz) and more sharply for high (>2 Hz) temporal frequencies and thus was not well-fit by a power law. We next compared the spectral properties of envelope signals resulting from active and passive self-motion, as well as those resulting from signals obtained when the subject is absent (i.e. external stimuli). Our data suggest that different mechanisms underlie deviation from scale invariance in rotational and translational self-motion envelopes. Specifically, active self-motion and filtering by the human body cause deviation from scale invariance primarily for translational and rotational envelope signals, respectively. Finally, we used well-established models in order to predict the responses of peripheral vestibular afferents to natural envelope stimuli. We found that irregular afferents responded more strongly to envelopes than their regular counterparts. Our findings have important consequences for understanding the coding strategies used by the vestibular system to process natural second-order self-motion signals.

Binocular functional architecture for detection of contrast-modulated gratings

Combination of signals from the two eyes is the gateway to stereo vision. To gain insight into binocular signal processing, we studied binocular summation for luminance-modulated gratings (L or LM) and contrast-modulated gratings (CM). We measured 2AFC detection thresholds for a signal grating (0.75c/deg, 216ms) shown to one eye, both eyes, or both eyes out-of-phase. For LM and CM, the carrier noise was in both eyes, even when the signal was monocular. Mean binocular thresholds for luminance gratings (L) were 5.4dB better than monocular thresholds - close to perfect linear summation (6dB). For LM and CM the binocular advantage was again 5-6dB, even when the carrier noise was uncorrelated, anti-correlated, or at orthogonal orientations in the two eyes. Binocular combination for CM probably arises from summation of envelope responses, and not from summation of these conflicting carrier patterns. Antiphase signals produced no binocular advantage, but thresholds were about 1-3dB higher than monocular ones. This is not consistent with simple linear summation, which should give complete cancellation and unmeasurably high thresholds. We propose a three-channel model in which noisy monocular responses to the envelope are binocularly combined in a contrast-weighted sum, but also remain separately available to perception via a max operator. Vision selects the largest of the three responses. With in-phase gratings the binocular channel dominates, but antiphase gratings cancel in the binocular channel and the monocular channels mediate detection. The small antiphase disadvantage might be explained by a subtle influence of background responses on binocular and monocular detection.

Encoding and estimation of first- and second-order binocular disparity in natural images

•

First- and second-order responses to natural binocular images are correlated.

•

Second-order mechanisms can improve the accuracy of disparity estimation.

•

Second-order mechanisms can extend the depth range of binocular stereopsis.

The first stage of processing of binocular information in the visual cortex is performed by mechanisms that are bandpass-tuned for spatial frequency and orientation. Psychophysical and physiological evidence have also demonstrated the existence of second-order mechanisms in binocular processing, which can encode disparities that are not directly accessible to first-order mechanisms. We compared the responses of first- and second-order binocular filters to natural images. We found that the responses of the second-order mechanisms are to some extent correlated with the responses of the first-order mechanisms, and that they can contribute to increasing both the accuracy, and depth range, of binocular stereopsis.

Coding stimulus amplitude by correlated neural activity

While correlated activity is observed ubiquitously in the brain, its role in neural coding has remained controversial. Recent experimental results have demonstrated that correlated but not single-neuron activity can encode the detailed time course of the instantaneous amplitude (i.e., envelope) of a stimulus. These have furthermore demonstrated that such coding required and was optimal for a nonzero level of neural variability. However, a theoretical understanding of these results is still lacking. Here we provide a comprehensive theoretical framework explaining these experimental findings. Specifically, we use linear response theory to derive an expression relating the correlation coefficient to the instantaneous stimulus amplitude, which takes into account key single-neuron properties such as firing rate and variability as quantified by the coefficient of variation. The theoretical prediction was in excellent agreement with numerical simulations of various integrate-and-fire type neuron models for various parameter values. Further, we demonstrate a form of stochastic resonance as optimal coding of stimulus variance by correlated activity occurs for a nonzero value of noise intensity. Thus, our results provide a theoretical explanation of the phenomenon by which correlated but not single-neuron activity can code for stimulus amplitude and how key single-neuron properties such as firing rate and variability influence such coding. Correlation coding by correlated but not single-neuron activity is thus predicted to be a ubiquitous feature of sensory processing for neurons responding to weak input.

The efficiency of second order orientation coherence detection

Neurons in early visual cortex respond to both luminance- (1st order) and contrast-modulated (2nd order) local features in the visual field. In later extra-striate areas neurons with larger receptive fields integrate information across the visual field. For example, local luminance-defined features can be integrated into contours and shapes. Evidence for the global integration of features defined by contrast-modulation is less well established. While good performance in some shape tasks has been demonstrated with 2nd order stimuli, the integration of contours fails with 2nd order elements. Recently we developed a global orientation coherence task that is more basic than contour integration, bearing similarity to the well-established global motion coherence task. Similar to our previous 1st order result for this task, we find 2nd order coherence detection to be scale-invariant. There was a small but significant threshold elevation for 2nd order relative to 1st order. We used a noise masking approach to compare the efficiency of orientation integration for the 1st and 2nd order. We find a significant deficit for 2nd order detection at both the local and global level, however the small size of this effect stands in stark contrast against previous results from contour-integration experiments, which are almost impossible with 2nd order stimuli.

Processing deficits of motion of contrast-modulated gratings in anisometropic amblyopia

Several studies have indicated substantial processing deficits for static second-order stimuli in amblyopia. However, less is known about the perception of second-order moving gratings. To investigate this issue, we measured the contrast sensitivity for second-order (contrast-modulated) moving gratings in seven anisometropic amblyopes and ten normal controls. The measurements were performed with non-equated carriers and a series of equated carriers. For comparison, the sensitivity for first-order motion and static second-order stimuli was also measured. Most of the amblyopic eyes (AEs) showed reduced sensitivity for second-order moving gratings relative to their non-amblyopic eyes (NAEs) and the dominant eyes (CEs) of normal control subjects, even when the detectability of the noise carriers was carefully controlled, suggesting substantial processing deficits of motion of contrast-modulated gratings in anisometropic amblyopia. In contrast, the non-amblyopic eyes of the anisometropic amblyopes were relatively spared. As a group, NAEs showed statistically comparable performance to CEs. We also found that contrast sensitivity for static second-order stimuli was strongly impaired in AEs and part of the NAEs of anisometropic amblyopes, consistent with previous studies. In addition, some amblyopes showed impaired performance in perception of static second-order stimuli but not in that of second-order moving gratings. These results may suggest a dissociation between the processing of static and moving second-order gratings in anisometropic amblyopia.

Form-cue invariant second-order neuronal responses to contrast modulation in primate area V2

A fundamental task of the visual system is to extract figure-ground boundaries between images of objects, which in natural scenes are often defined not only by luminance differences but also by "second-order" contrast or texture differences. Responses to contrast modulation (CM) and other second-order stimuli have been extensively studied in human psychophysics, but the neuronal substrates of second-order responses in nonhuman primates remain poorly understood. In this study, we have recorded single neurons in area V2 of macaque monkeys, using both CM patterns as well as conventional luminance modulation (LM) gratings. CM stimuli were constructed from stationary sine wave grating carrier patterns, which were modulated by drifting envelope gratings of a lower spatial frequency. We found approximately one-third of visually responsive V2 neurons responded to CM stimuli with a pronounced selectivity to carrier spatial frequencies, and often orientations, that were clearly outside the neurons' passbands for LM gratings. These neurons were "form-cue invariant" in that their tuning to CM envelope spatial frequency and orientation was very similar to that for LM gratings. Neurons were tuned to carrier spatial frequencies that were typically 2-4 octaves higher than their optimal envelope spatial frequencies, similar to results from human psychophysics. These results are distinct from CM responses arising from surround suppression, but could be understood in terms of a filter-rectify-filter model. Such neurons could provide a functionally useful and explicit representation of segmentation boundaries as well as a plausible neural substrate for human perception of second-order boundaries.

What can fish brains tell us about visual perception?

Fish are a complex taxonomic group, whose diversity and distance from other vertebrates well suits the comparative investigation of brain and behavior: in fish species we observe substantial differences with respect to the telencephalic organization of other vertebrates and an astonishing variety in the development and complexity of pallial structures. We will concentrate on the contribution of research on fish behavioral biology for the understanding of the evolution of the visual system. We shall review evidence concerning perceptual effects that reflect fundamental principles of the visual system functioning, highlighting the similarities and differences between distant fish groups and with other vertebrates. We will focus on perceptual effects reflecting some of the main tasks that the visual system must attain. In particular, we will deal with subjective contours and optical illusions, invariance effects, second order motion and biological motion and, finally, perceptual binding of object properties in a unified higher level representation.

Dependence of V2 illusory contour response on V1 cell properties and topographic organization

An illusory contour is an image that is perceived as a contour in the absence of typical contour characteristics, such as a change in luminance or chromaticity across the stimulus. In cats and primates, cells that respond to illusory contours are sparse in cortical area V1, but are found in greater numbers in cortical area V2. We propose a model capable of illusory contour detection that is based on a realistic topographic organization of V1 cells, which reproduces the responses of individual cell types measured experimentally. The model allows us to explain several experimentally observed properties of V2 cells including variability in orientation tuning and inducer spacing preference. As a practical application, the model can be used to estimate the relationship between the severity of a cortical injury in the primary visual cortex and the deterioration of V2 cell responses to real and illusory contours.

Binocular summation of second-order global motion signals in human vision

Although many studies have examined the principles governing first-order global motion perception, the mechanisms that mediate second-order global motion perception remain unresolved. This study investigated the existence, nature and extent of the binocular advantage for encoding second-order (contrast-defined) global motion. Motion coherence thresholds (79.4% correct) were assessed for determining the direction of radial, rotational and translational second-order motion trajectories as a function of local element modulation depth (contrast) under monocular and binocular viewing conditions. We found a binocular advantage for second-order global motion processing for all motion types. This advantage was mainly one of enhanced modulation sensitivity, rather than of motion-integration. However, compared to findings for first-order motion where the binocular advantage was in the region of a factor of around 1.7 (Hess et al., 2007), the binocular advantage for second-order global motion was marginal, being in the region of around 1.2. This weak enhancement in sensitivity with binocular viewing is considerably less than would be predicted by conventional models of either probability summation or neural summation.

Functional organization of envelope-responsive neurons in early visual cortex: organization of carrier tuning properties

It is well established that visual cortex neurons having similar selectivity for orientation, direction of motion, ocular dominance, and other properties of first-order (luminance-defined) stimuli are clustered into a columnar organization. However, the cortical architecture of neuronal responses to second-order (contrast/texture-defined) stimuli is poorly understood. A useful second-order stimulus is a contrast envelope, consisting of a finely detailed pattern (carrier) whose contrast varies on a coarse spatial scale (envelope). In this study, we analyzed the cortical organization of carrier tuning properties of neurons, which responded to contrast-modulated stimuli. We examined whether neurons tuned to similar carrier properties are clustered spatially and whether such spatial clusters are arranged in columns. To address these questions, we recorded single-unit activity, multiunit activity, and local field potentials simultaneously from area 18 of anesthetized cats, using single-channel microelectrodes and multielectrode arrays. Our data showed that neurons tuned to similar carrier spatial frequency are distributed in a highly clustered manner; neurons tuned to similar carrier orientation are also significantly clustered. Neurons along linear arrays perpendicular to the brain surface always exhibited similar optimal carrier spatial frequency, indicating a columnar organization. Multi-pronged tetrode recordings indicated that the diameter of these columns is ≥450 μm. Optimal carrier orientation was also significantly clustered but with finer-grain organization and greater scatter. These results indicate a fine anatomical structure of cortical organization of second-order information processing and suggest that there are probably more maps in cat area 18 than previously believed.

Responses to second-order texture modulations undergo surround suppression

First-order (contrast) surround suppression has been well characterized both psychophysically and physiologically,but relatively little is known as to whether the perception of second-order visual stimuli exhibits analogous center–surround interactions. Second-order surround suppression was characterized by requiring subjects to detect second-order modulation in stimuli presented alone or embedded in a surround.Both contrast- (CM) and orientation-modulated (OM) stimuli were used. For most subjects and both OM and CM stimuli, second-order surrounds caused thresholds to be higher, indicative of second-order suppression. For CM stimuli, suppression was orientation-specific, i.e., higher thresholds for parallel than for orthogonal surrounds. However, the evidence for orientation specificity of suppression for OM stimuli was weaker. These results suggest that normalization, leading to surround suppression, operates at multiple stages in cortical processing.

Interaction of first- and second-order signals in the extraction of global-motion and optic-flow

The intention of this series of experiments was to determine the extent to which the pathways sensitive to first-order and second-order motion are independent of one another at, and above, the level of global motion integration. We used translational, radial and rotational motion stimuli containing luminance-modulated dots, contrast-modulated dots, or a mixture of both. Our results show that the two classes of motion stimuli interact perceptually in a global motion coherence task, and the extent of this interaction is governed by whether the two varieties of local motion signal produce an equivalent response in the pathways that encode each type of motion. This provides strong psychophysical evidence that global motion and optic flow processing are cue-invariant. The fidelity of the first-order motion signal was moderated by either reducing the luminance of the dots or by increasing the displacement of the dots on each positional update. The experiments were carried out with two different types of second-order elements (contrast-modulated dots and flicker-modulated dots) and the results were comparable, suggesting that these findings are generalisable to a variety of second-order stimuli. In addition, the interaction between the two different types of second-order stimuli was investigated and we found that the relative modulation depth was also crucial to whether the two populations interacted. We conclude that the relative output of local motion sensors sensitive to either first-order or second-order motion dictates their weight in subsequent cue-invariant global motion computations.

Motion-defined contour processing in the early visual cortex

From our daily experience, it is very clear that relative motion cues can contribute to correctly identifying object boundaries and perceiving depth. Motion-defined contours are not only generated by the motion of objects in a scene but also by the movement of an observer's head and body (motion parallax). However, the neural mechanism involved in detecting these contours is still unknown. To explore this mechanism, we extracellularly recorded visual responses of area 18 neurons in anesthetized and paralyzed cats. The goal of this study was to determine if motion-defined contours could be detected by neurons that have been previously shown to detect luminance-, texture-, and contrast-defined contours cue invariantly. Motion-defined contour stimuli were generated by modulating the velocity of high spatial frequency sinusoidal luminance gratings (carrier gratings) by a moving squarewave envelope. The carrier gratings were outside the luminance passband of a neuron, such that presence of the carrier alone within the receptive field did not elicit a response. Most neurons that responded to contrast-defined contours also responded to motion-defined contours. The orientation and direction selectivity of these neurons for motion-defined contours was similar to that of luminance gratings. A given neuron also exhibited similar selectivity for the spatial frequency of the carrier gratings of contrast- and motion-defined contours. These results suggest that different second-order contours are detected in a form-cue invariant manner, through a common neural mechanism in area 18.

Figure tracking by flies is supported by parallel visual streams

Visual figures may be distinguished based on elementary motion or higher-order non-Fourier features, and flies track both. The canonical elementary motion detector, a compact computation for Fourier motion direction and amplitude, can also encode higher-order signals provided elaborate preprocessing. However, the way in which a fly tracks a moving figure containing both elementary and higher-order signals has not been investigated. Using a novel white noise approach, we demonstrate that (1) the composite response to an object containing both elementary motion (EM) and uncorrelated higher-order figure motion (FM) reflects the linear superposition of each component; (2) the EM-driven component is velocity-dependent, whereas the FM component is driven by retinal position; (3) retinotopic variation in EM and FM responses are different from one another; (4) the FM subsystem superimposes saccadic turns upon smooth pursuit; and (5) the two systems in combination are necessary and sufficient to predict the full range of figure tracking behaviors, including those that generate no EM cues at all. This analysis requires an extension of the model that fly motion vision is based on simple elementary motion detectors and provides a novel method to characterize the subsystems responsible for the pursuit of visual figures.

Visual chimaeras obtained with the Riesz transform

Similar to an auditory chimaera (Smith et al, 2002 Nature 416 87-90), a visual chimaera can be defined as a synthetic image which has the fine spatial structure of one natural image and the envelope of another image in each spatial frequency band. Visual chimaeras constructed in this way could be useful to vision scientists interested in the study of interactions between first-order and second-order visual processing. Although it is almost trivial to generate 1-D chimaeras by means of the Hilbert transform and the analytic signal, problems arise in multidimensional signals like images given that the partial directional Hilbert transform and current 2-D demodulation algorithms are anisotropic or orientation-variant procedures. Here, we present a computational procedure to synthesise visual chimaeras by means of the Riesz transform--an isotropic generalisation of the Hilbert transform for multidimensional signals--and the associated monogenic signal--the vector-valued function counterpart of the analytic signal in which the Riesz transform replaces the Hilbert transform. Examples of visual chimaeras are shown for same/different category images.

Direction-selective patterns of activity in human visual cortex suggest common neural substrates for different types of motion

A sense of motion can be elicited by the movement of both luminance- and texture-defined patterns, what is commonly referred to as first- and second-order, respectively. Although there are differences in the perception of these two classes of motion stimuli, including differences in temporal and spatial sensitivity, it is debated whether common or separate direction-selective mechanisms are responsible for processing these two types of motion. Here, we measured direction-selective responses to luminance- and texture-defined motion in the human visual cortex by using functional MRI (fMRI) in conjunction with multivariate pattern analysis (MVPA). We found evidence of direction selectivity for both types of motion in all early visual areas (V1, V2, V3, V3A, V4, and MT+), implying that none of these early visual areas is specialized for processing a specific type of motion. More importantly, linear classifiers trained with cortical activity patterns to one type of motion (e.g., first-order motion) could reliably classify the direction of motion defined by the other type (e.g., second-order motion). Our results suggest that the direction-selective mechanisms that respond to these two types of motion share similar spatial distributions in the early visual cortex, consistent with the possibility that common mechanisms are responsible for processing both types of motion.

Visual responses to contrast-defined contours with equally spatial-scaled carrier in cat area 18

Contrast-defined contours are one type of second-order contours, across which there are no differences in luminance. Although they can be always perceived, their responses have been only investigated when the spatial frequency of carrier, the background texture whose contrast is modulated to form contours, is much higher than that of contrast-defined contours, due to the interference of responses to luminance contours in other cases. In the present study, we examined visual responses in cat area 18 to the contrast-defined contours with carrier at same spatial frequency equal to neuron's preferred value for luminance contours, by establishing a control stimulus including all the luminance components but lack of the contrast contour information. Using single unit recording and intrinsic optical imaging, we demonstrated that contrast gratings with equally spatial-scaled carrier induced responses in a proportion of cat area 18 neurons with the preferred orientation similar to that for luminance contours, and the responses generated orientation maps similar to those for luminance contours. Our finding suggests that early visual cortex can process second-order contours regardless of the spatial frequency of carriers, in a way similar to the processing of luminance contours. This uniform manner of early visual processing might underlie the visual detection of both luminance contours and non-luminance second-order contours.

Sensitivity to first- and second-order drifting gratings in 3-month-old infants

In two experiments, we investigated 3-month-old infants' sensitivity to first- and second-order drifting gratings. In Experiment 1 we used forced-choice preferential looking with drifting versus stationary gratings to estimate depth modulation thresholds for 3-month-old infants and a similar task for a comparison group of adults. Thresholds for infants were more adult-like for second-order than first-order gratings. In Experiment 2, 3-month-olds dishabituated to a change in first-order orientation, but not to a change in direction of first- or second-order motion. Hence, results from Experiment 1 were likely driven by the perception of flicker rather than motion. Thus, infants' sensitivity to uniform motion is slow to develop and appears to be driven initially by flicker-sensitive mechanisms. The underlying mechanisms have more mature tuning for second-order than for first-order information.

Visual deficits in amblyopia constrain normal models of second-order motion processing

It is well established that amblyopes exhibit deficits in processing first-order (luminance-defined) patterns. This is readily manifest by measuring spatiotemporal sensitivity (i.e. the "window of visibility") to moving luminance gratings. However the window of visibility to moving second-order (texture-defined) patterns has not been systematically studied in amblyopia. To address this issue monocular modulation sensitivity (1/threshold) to first-order motion and four different varieties of second-order motion (modulations of either the contrast, flicker, size or orientation of visual noise) was measured over a five-octave range of spatial and temporal frequencies. Compared to normals amblyopes are not only impaired in the processing of first-order motion, but overall they exhibit both higher thresholds and a much narrower window of visibility to second-order images. However amblyopia can differentially impair the perception of some types of second-order motion much more than others and crucially the precise pattern of deficits varies markedly between individuals (even for those with the same conventional visual acuity measures). For the most severely impaired amblyopes certain second-order (texture) cues to movement in the environment are effectively invisible. These results place important constraints on the possible architecture of models of second-order motion perception in human vision.

Spatial summation of first-order and second-order motion in human vision

This study assessed spatial summation of first-order (luminance-defined) and second-order (contrast-defined) motion. Thresholds were measured for identifying the drift direction of 1c/deg., luminance-modulated and contrast-modulated dynamic noise drifting at temporal frequencies of 0.5, 2 and 8Hz. Image size varied from 0.125 degrees to 16 degrees . The effects of increasing image size on thresholds for luminance-modulated noise were also compared to those for luminance-defined gratings. In all cases, performance improved as image size increased. The rate at which performance improved with increasing image size was similar for all stimuli employed although the slopes corresponding to the initial improvement were steeper for first-order compared to second-order motion. The image sizes at which performance for first-order motion asymptote were larger than for second-order motion. In addition, findings showed that the minimum image size required to support reliable identification of the direction of moving stimuli is greater for second-order than first-order motion. Thus, although first-order and second-order motion processing have a number of properties in common, the visual system's sensitivity to each type of motion as a function of image size is quite different.

Neural responses to one- and two-tone stimuli in the hearing organ of the dengue vector mosquito

Recent studies demonstrate that mosquitoes listen to each other's wing beats just prior to mating in flight. Field potentials from sound-transducing neurons in the antennae contain both sustained and oscillatory components to pure and paired tone stimuli. Described here is a direct comparison of these two types of response in the dengue vector mosquito, Aedes aegypti. Across a wide range of frequencies and intensities, sustained responses to one- and two-tone stimuli are about equal in magnitude to oscillatory responses to the beats produced by two-tone stimuli. All of these responses are much larger than the oscillatory responses to one-tone stimuli. Similarly, the frequency range extends up to at least the fifth harmonic of the male flight tone for sustained responses to one- and two-tone stimuli and oscillatory responses at the beat frequency of two-tone stimuli, whereas the range of oscillatory response to a one-tone stimulus is limited to, at most, the third harmonic. Thresholds near the fundamental of the flight tone are lower for oscillatory responses than for sustained deflections, lower for males than for females, and within the behaviorally relevant range. A simple model of the transduction process can qualitatively account for both oscillatory and sustained responses to pure and paired tones. These data leave open the question as to which of several alternative strategies underlie flight tone matching behavior in mosquitoes.

Application of Riesz transforms to the isotropic AM-PM decomposition of geometrical-optical illusion images

The existence of a special second-order mechanism in the human visual system, able to demodulate the envelope of visual stimuli, suggests that spatial information contained in the image envelope may be perceptually relevant. The Riesz transform, a natural isotropic extension of the Hilbert transform to multidimensional signals, was used here to demodulate band-pass filtered images of well-known visual illusions of length, size, direction, and shape. We show that the local amplitude of the monogenic signal or envelope of each illusion image conveys second-order information related to image holistic spatial structure, whereas the local phase component conveys information about the spatial features. Further low-pass filtering of the illusion image envelopes creates physical distortions that correspond to the subjective distortions perceived in the illusory images. Therefore the envelope seems to be the image component that physically carries the spatial information about these illusions. This result contradicts the popular belief that the relevant spatial information to perceive geometrical-optical illusions is conveyed only by the lower spatial frequencies present in their Fourier spectrum.

Coarse-fine dichotomies in human stereopsis

There is a long history of research into depth percepts from very large disparities, beyond the fusion limit. Such diplopic stimuli have repeatedly been shown to provide reliable depth percepts. A number of researchers have pointed to differences between the processing of small and large disparities, arguing that they are subserved by distinct neural mechanisms. Other studies have pointed to a dichotomy between the processing of 1st- and 2nd-order stimuli. Here we review literature on the full range of disparity processing to determine how well different proposed dichotomies map onto one another, and to identify unresolved issues.

Surround suppression of V1 neurons mediates orientation-based representation of high-order visual features

Neurons with surround suppression have been implicated in processing high-order visual features such as contrast- or texture-defined boundaries and subjective contours. However, little is known regarding how these neurons encode high-order visual information in a systematic manner as a population. To address this issue, we have measured detailed spatial structures of classical center and suppressive surround regions of receptive fields of primary visual cortex (V1) neurons and examined how a population of such neurons allow encoding of various high-order features and shapes in visual scenes. Using a novel method to reconstruct structures, we found that the center and surround regions are often both elongated parallel to each other, reminiscent of on and off subregions of simple cells without surround suppression. These structures allow V1 neurons to extract high-order contours of various orientations and spatial frequencies, with a variety of optimal values across neurons. The results show that a wide range of orientations and widths of the high-order features are systematically represented by the population of V1 neurons with surround suppression.

Distinct perceptual grouping pathways revealed by temporal carriers and envelopes

S. E. Guttman, L. A. Gilroy, and R. Blake (2005) investigated whether observers could perform temporal grouping in multi-element displays where each local element was stochastically modulated over time along one of several potential dimensions--or "messenger types"--such as contrast, position, orientation, or spatial scale. Guttman et al.'s data revealed that grouping discards messenger type and therefore support a single-pathway model that groups elements with similar temporal waveforms. In the current study, we carried out three experiments in which temporal-grouping information resided either in the carrier, the envelope, or the combined carrier and envelope of each messenger's timecourse. Results revealed that grouping is highly specific for messenger type if carrier envelopes lack grouping information but largely messenger nonspecific if carrier envelopes contain grouping information. These imply that temporal grouping is mediated by several messenger-specific carrier pathways as well as by a messenger-nonspecific envelope pathways. Findings also challenge simple temporal-filtering accounts of perceptual grouping (E. H. Adelson & H. Farid, 1999).

Eye Movements in Response to Dichoptic Motion: Evidence for a Parallel-Hierarchical Structure of Visual Motion Processing in Primates

Brief movements of a large-field visual stimulus elicit short-latency tracking eye movements termed “ocular following responses” (OFRs). To address the question of whether OFRs can be elicited by purely binocular motion signals in the absence of monocular motion cues, we measured OFRs from monkeys using dichoptic motion stimuli, the monocular inputs of which were flickering gratings in spatiotemporal quadrature, and compared them with OFRs to standard motion stimuli including monocular motion cues. Dichoptic motion did elicit OFRs, although with longer latencies and smaller amplitudes. In contrast to these findings, we observed that other types of motion stimuli categorized as non-first-order motion, which is undetectable by detectors for standard luminance-defined (first-order) motion, did not elicit OFRs, although they did evoke the sensation of motion. These results indicate that OFRs can be driven solely by cortical visual motion processing after binocular integration, which is distinct from the process incorporating non-first-order motion for elaborated motion perception. To explore the nature of dichoptic motion processing in terms of interaction with monocular motion processing, we further recorded OFRs from both humans and monkeys using our novel motion stimuli, the monocular and dichoptic motion signals of which move in opposite directions with a variable motion intensity ratio. We found that monocular and dichoptic motion signals are processed in parallel to elicit OFRs, rather than suppressing each other in a winner-take-all fashion, and the results were consistent across the species.

Choice reaction times for identifying the direction of first-order motion and different varieties of second-order motion

This study sought to quantify the temporal properties of the human visual system by measuring forced-choice reaction times for discriminating the drift direction of first-order motion (luminance-modulated noise) and a variety of second-order motion patterns (modulations of either the contrast, polarity, orientation or spatial length of a noise carrier) over a range of stimulus modulation depths. In general, reaction times for all types of second-order motion were slower than those for first-order motion. Specifically, reaction times were similar for modulations of image contrast, polarity and orientation but were markedly slower for modulations of spatial length. There was also a tendency for reaction times to decrease as stimulus modulation depth increased. The rate of this decrease was shallowest for first-order, luminance-defined patterns. For second-order motion reaction times decreased at a similar rate for contrast, polarity and orientation but this decrease was steepest for spatial length. However, when equated in terms of visibility (multiples of direction-discrimination threshold), the rate at which reaction times decreased as modulation depth increased became comparable for patterns defined by luminance, contrast, polarity and orientation. For patterns defined by spatial length, performance could not be equated in this manner. These findings demonstrate that the time taken to encode the direction of each pattern is not an invariant response metric. The results are consistent with psychophysical and electrophysiological evidence for longer response latencies for second-order motion and may reflect the additional processing stages (e.g. filter-rectify-filter) required for its extraction.

Human cortical response to various apparent motions: A magnetoencephalographic study

The human visual system is considered to have at least two different mechanisms for perceiving motions: one for luminance-based (first-order) motions and another for non-luminance-based (second-order) motions. In this study, we examined the perception of first- and second-order motions using four different types of stimulus cues (luminance, contrast, texture, and flicker) while using whole head magnetoencephalography (MEG) to measure human brain responses to those apparent motions. MEG responses to all stimuli were recorded from the occipito-temporal area (possibly human MT/V5+), and response properties (peak latency and amplitude) varied with stimulus cues. Further, we observed various effects of luminance-addition to the non-luminance cues on the response properties that could not be explained by the magnetic field distribution and/or the visibility of the stationary object. The results indicate that differences in response properties elicited by various stimulus cues represent differences in the neural processes underlying apparent motions with various cues. We suggest that the distinct "preprocessing" of each stimulus cue occurs before the common process for apparent motion, and the response property changes associated with different cues are related to differences in preprocessing that may occur in a distributed cortical network that include the striate and extrastriate visual cortex.

Neuronal response to texture- and contrast-defined boundaries in early visual cortex

Natural scenes contain a variety of visual cues that facilitate boundary perception (e.g., luminance, contrast, and texture). Here we explore whether single neurons in early visual cortex can process both contrast and texture cues. We recorded neural responses in cat A18 to both illusory contours formed by abutting gratings (ICs, texture-defined) and contrast-modulated gratings (CMs, contrast-defined). We found that if a neuron responded to one of the two stimuli, it also responded to the other. These neurons signaled similar contour orientation, spatial frequency, and movement direction of the two stimuli. A given neuron also exhibited similar selectivity for spatial frequency of the fine, stationary grating components (carriers) of the stimuli. These results suggest that the cue-invariance of early cortical neurons extends to different kinds of texture or contrast cues, and might arise from a common nonlinear mechanism.

Implications of motion detection for early non-linearities

Analysis of motion may be accomplished using the spatiotemporal variations produced when a spatially varying luminance waveform moves across linear receptive fields. Moving contrast-modulated patterns which consist of coarse-scale spatial variations in the contrast of fine-scale luminance patterns cannot be analysed in this way. The human visual system can analyse the motion of contrast-modulated patterns and this suggests it may contain mechanisms that use non-linear transformations. Non-linear transformation of contrast-modulated patterns would give rise to a component (a distortion product) that varies on the same spatial scale as the contrast variation; this can be analysed to extract motion. Is the non-linearity simply an inherent part of the transduction process or is it a characteristic of a mechanism specialized for the analysis of the motion of such patterns? Comparisons of the spatial and temporal limitations of motion discrimination using luminance and contrast-modulated patterns suggest that the mechanisms which analyse the two types of patterns are different, although recent physiological evidence suggests that they may have common elements.

Critical spatial frequencies for illusory contour processing in early visual cortex

Single neurons in primate V2 and cat A18 exhibit identical orientation tuning for sinewave grating and illusory contour stimuli. This cue invariance is also manifested in similar orientation maps to these stimuli, but in V1/A17 the illusory contour maps appear reversed. We hypothesized that this map reversal depends upon the spatial frequencies of the inducers in the illusory contours, relative to the spatial selectivities of these brain areas. We employed intrinsic signal optical imaging to measure orientation maps in cat A17/18 to illusory contours with inducers at spatial frequencies from 0.15 to 1.6 cpd. A17 illusory contour maps were indeed reversed compared with grating-driven maps for inducer spatial frequencies <1.3 cpd, whereas A18 maps were invariant. Simulations based on known neurophysiology demonstrated that map reversal can arise from linear filtering, and map invariance can be explained by a nonlinear (filter-rectify-filter) mechanism. The simulation also correctly predicted that A17 could show invariant maps when the inducer spatial frequency is sufficiently high (1.6 cpd), and that A18 maps could reverse at lower inducer frequencies (0.18 cpd). Thus, the map reversal or invariance to illusory contours depends critically on the relationship of the inducer spatial frequencies to the spatial filtering properties of neurons in each brain area.

Separate first- and second-order processing is supported by spatial summation estimates at the fovea and eccentrically

We estimated spatial summation areas for the detection of luminance-modulated (LM) and contrast-modulated (CM) blobs at the fovea, 2.5, 5 and 10 deg eccentrically. Gaussian profiles were added or multiplied to binary white noise to create LM and CM blob stimuli and these were used to psychophysically estimate detection thresholds and spatial summation areas. The results reveal significantly larger summation areas for detecting CM than LM blobs across eccentricity. These differences are comparable to receptive field size estimates made in V1 and V2. They support the notion that separate spatial processing occurs for the detection of LM and CM stimuli.