Local sensitivity to stimulus orientation and spatial frequency within the receptive fields of neurons in visual area 2 of macaque monkeys

Foundations of visual form selectivity for neurons in macaque V1 and V2

We have measured the visually evoked activity of single neurons recorded in areas V1 and V2 of awake, fixating macaque monkeys, and captured their responses with a common computational model. We used a stimulus set composed of "droplets" of localized contrast, band-limited in orientation and spatial frequency; each brief stimulus contained a random superposition of droplets presented in and near the mapped receptive field. We accounted for neuronal responses with a 2-layer linear-nonlinear model, representing each receptive field by a combination of orientation- and scale-selective filters. We fit the data by jointly optimizing the model parameters to enforce sparsity and to prevent overfitting. We visualized and interpreted the fits in terms of an "afferent field" of nonlinearly combined inputs, dispersed in the 4 dimensions of space and spatial frequency. The resulting fits generally give a good account of the responses of neurons in both V1 and V2, capturing an average of 40% of the explainable variance in neuronal firing. Moreover, the resulting models predict neuronal responses to image families outside the test set, such as gratings of different orientations and spatial frequencies. Our results offer a common framework for understanding processing in the early visual cortex, and also demonstrate the ways in which the distributions of neuronal responses in V1 and V2 are similar but not identical.

Normalization by orientation-tuned surround in human V1-V3

An influential account of neuronal responses in primary visual cortex is the normalized energy model. This model is often implemented as a multi-stage computation. The first stage is linear filtering. The second stage is the extraction of contrast energy, whereby a complex cell computes the squared and summed outputs of a pair of the linear filters in quadrature phase. The third stage is normalization, in which a local population of complex cells mutually inhibit one another. Because the population includes cells tuned to a range of orientations and spatial frequencies, the result is that the responses are effectively normalized by the local stimulus contrast. Here, using evidence from human functional MRI, we show that the classical model fails to account for the relative responses to two classes of stimuli: straight, parallel, band-passed contours (gratings), and curved, band-passed contours (snakes). The snakes elicit fMRI responses that are about twice as large as the gratings, yet a traditional divisive normalization model predicts responses that are about the same. Motivated by these observations and others from the literature, we implement a divisive normalization model in which cells matched in orientation tuning ("tuned normalization") preferentially inhibit each other. We first show that this model accounts for differential responses to these two classes of stimuli. We then show that the model successfully generalizes to other band-pass textures, both in V1 and in extrastriate cortex (V2 and V3). We conclude that even in primary visual cortex, complex features of images such as the degree of heterogeneity, can have large effects on neural responses.

Visual response characteristics of neurons in the second visual area of marmosets

The physiological characteristics of the marmoset second visual area (V2) are poorly understood compared with those of the primary visual area (V1). In this study, we observed the physiological response characteristics of V2 neurons in four healthy adult marmosets using intracortical tungsten microelectrodes. We recorded 110 neurons in area V2, with receptive fields located between 8° and 15° eccentricity. Most (88.2%) of these neurons were orientation selective, with half-bandwidths typically ranging between 10° and 30°. A significant proportion of neurons (28.2%) with direction selectivity had a direction index greater than 0.5. The vast majority of V2 neurons had separable spatial frequency and temporal frequency curves and, according to this criterion, they were not speed selective. The basic functional response characteristics of neurons in area V2 resemble those found in area V1. Our findings show that area V2 together with V1 are important in primate visual processing, especially in locating objects in space and in detecting an object’s direction of motion. The methods used in this study were approved by the Monash University Animal Ethics Committee, Australia (MARP 2009-2011) in 2009.

Spiking Noise and Information Density of Neurons in Visual Area V2 of Infant Monkeys

Encoding of visual information requires precisely timed spiking activity in the network of cortical neurons; irregular spiking can interfere with information processing especially for low-contrast images. The vision of newborn infants is impoverished. An infant's contrast sensitivity is low and the ability to discriminate complex stimuli is poor. The neural mechanisms that limit the visual capacities of infants are a matter of debate. Here we asked whether noisy spiking and/or crude information processing in visual cortex limit infant vision. Since neurons beyond the primary visual cortex (V1) have rarely been studied in neonates or infants, we focused on the firing pattern of neurons in visual area V2, the earliest extrastriate visual area of both male and female macaque monkeys (Maccaca mulatta). For eight stimulus contrasts ranging from 0% to 80%, we analyzed spiking irregularity by calculating the square of the coefficient of variation (CV²) in interspike intervals, the trial-to-trial fluctuation in spiking (Fano factor), and the amount of information on contrast conveyed by each spiking (information density). While the contrast sensitivity of infant neurons was reduced as expected, spiking noise, both the magnitude of spiking irregularity and the trial-to-trial fluctuations, was much lower in the spike trains of infant V2 neurons compared with those of adults. However, information density for V2 neurons was significantly lower in infants. Our results suggest that poor contrast sensitivity combined with lower information density of extrastriate neurons, despite their lower spiking noise, may limit behaviorally determined contrast sensitivity soon after birth.SIGNIFICANCE STATEMENT Despite >50 years of investigations on the postnatal development of the primary visual cortex (V1), cortical mechanisms that may limit infant vision are still unclear. We investigated the quality and strength of neuronal firing in primate visual area V2 by analyzing contrast sensitivity, spiking variability, and the amount of information on contrast conveyed by each action potential (information density). Here we demonstrate that the firing rate, contrast sensitivity, and dynamic range of V2 neurons were depressed in infants compared with adults. Although spiking noise was less, information density was lower in infant V2. Impoverished neuronal drive and lower information density in extrastriate visual areas, despite lower spiking noise, largely explain the impoverished visual sensitivity of primates near birth.

Joint coding of shape and blur in area V4

Edge blur, a prevalent feature of natural images, is believed to facilitate multiple visual processes including segmentation and depth perception. Furthermore, image descriptions that explicitly combine blur and shape information provide complete representations of naturalistic scenes. Here we report the first demonstration of blur encoding in primate visual cortex: neurons in macaque V4 exhibit tuning for both object shape and boundary blur, with observed blur tuning not explained by potential confounds including stimulus size, intensity, or curvature. A descriptive model wherein blur selectivity is cast as a distinct neural process that modulates the gain of shape-selective V4 neurons explains observed data, supporting the hypothesis that shape and blur are fundamental features of a sufficient neural code for natural image representation in V4.

Blurred edges of objects can aid in depth perception and segmentation, yet how it is combined with shape information in the visual pathway is unknown. Here the authors report that neurons in higher visual area V4 represent both object shape and boundary blur, controlling for stimulus size, intensity and curvature.

Noisy Spiking in Visual Area V2 of Amblyopic Monkeys

Interocular decorrelation of input signals in developing visual cortex can cause impaired binocular vision and amblyopia. Although increased intrinsic noise is thought to be responsible for a range of perceptual deficits in amblyopic humans, the neural basis for the elevated perceptual noise in amblyopic primates is not known. Here, we tested the idea that perceptual noise is linked to the neuronal spiking noise (variability) resulting from developmental alterations in cortical circuitry. To assess spiking noise, we analyzed the contrast-dependent dynamics of spike counts and spiking irregularity by calculating the square of the coefficient of variation in interspike intervals (CV²) and the trial-to-trial fluctuations in spiking, or mean matched Fano factor (m-FF) in visual area V2 of monkeys reared with chronic monocular defocus. In amblyopic neurons, the contrast versus response functions and the spike count dynamics exhibited significant deviations from comparable data for normal monkeys. The CV² was pronounced in amblyopic neurons for high-contrast stimuli and the m-FF was abnormally high in amblyopic neurons for low-contrast gratings. The spike count, CV², and m-FF of spontaneous activity were also elevated in amblyopic neurons. These contrast-dependent spiking irregularities were correlated with the level of binocular suppression in these V2 neurons and with the severity of perceptual loss for individual monkeys. Our results suggest that the developmental alterations in normalization mechanisms resulting from early binocular suppression can explain much of these contrast-dependent spiking abnormalities in V2 neurons and the perceptual performance of our amblyopic monkeys.

SIGNIFICANCE STATEMENT

Amblyopia is a common developmental vision disorder in humans. Despite the extensive animal studies on how amblyopia emerges, we know surprisingly little about the neural basis of amblyopia in humans and nonhuman primates. Although the vision of amblyopic humans is often described as being noisy by perceptual and modeling studies, the exact nature or origin of this elevated perceptual noise is not known. We show that elevated and noisy spontaneous activity and contrast-dependent noisy spiking (spiking irregularity and trial-to-trial fluctuations in spiking) in neurons of visual area V2 could limit the visual performance of amblyopic primates. Moreover, we discovered that the noisy spiking is linked to a high level of binocular suppression in visual cortex during development.

Eye movement-invariant representations in the human visual system

During natural vision, humans make frequent eye movements but perceive a stable visual world. It is therefore likely that the human visual system contains representations of the visual world that are invariant to eye movements. Here we present an experiment designed to identify visual areas that might contain eye-movement-invariant representations. We used functional MRI to record brain activity from four human subjects who watched natural movies. In one condition subjects were required to fixate steadily, and in the other they were allowed to freely make voluntary eye movements. The movies used in each condition were identical. We reasoned that the brain activity recorded in a visual area that is invariant to eye movement should be similar under fixation and free viewing conditions. In contrast, activity in a visual area that is sensitive to eye movement should differ between fixation and free viewing. We therefore measured the similarity of brain activity across repeated presentations of the same movie within the fixation condition, and separately between the fixation and free viewing conditions. The ratio of these measures was used to determine which brain areas are most likely to contain eye movement-invariant representations. We found that voxels located in early visual areas are strongly affected by eye movements, while voxels in ventral temporal areas are only weakly affected by eye movements. These results suggest that the ventral temporal visual areas contain a stable representation of the visual world that is invariant to eye movements made during natural vision.

Neurons in Macaque Area V4 Are Tuned for Complex Spatio-Temporal Patterns

To deepen our understanding of object recognition, it is critical to understand the nature of transformations that occur in intermediate stages of processing in the ventral visual pathway, such as area V4. Neurons in V4 are selective to local features of global shape, such as extended contours. Previously, we found that V4 neurons selective for curved elements exhibit a high degree of spatial variation in their preference. If spatial variation in curvature selectivity was also marked by distinct temporal response patterns at different spatial locations, then it might be possible to untangle this information in subsequent processing based on temporal responses. Indeed, we find that V4 neurons whose receptive fields exhibit intricate selectivity also show variation in their temporal responses across locations. A computational model that decodes stimulus identity based on population responses benefits from using this temporal information, suggesting that it could provide a multiplexed code for spatio-temporal features.

Spatial structure of neuronal receptive field in awake monkey secondary visual cortex (V2)

Visual processing depends critically on the receptive field (RF) properties of visual neurons. However, comprehensive characterization of RFs beyond the primary visual cortex (V1) remains a challenge. Here we report fine RF structures in secondary visual cortex (V2) of awake macaque monkeys, identified through a projection pursuit regression analysis of neuronal responses to natural images. We found that V2 RFs could be broadly classified as V1-like (typical Gabor-shaped subunits), ultralong (subunits with high aspect ratios), or complex-shaped (subunits with multiple oriented components). Furthermore, single-unit recordings from functional domains identified by intrinsic optical imaging showed that neurons with ultralong RFs were primarily localized within pale stripes, whereas neurons with complex-shaped RFs were more concentrated in thin stripes. Thus, by combining single-unit recording with optical imaging and a computational approach, we identified RF subunits underlying spatial feature selectivity of V2 neurons and demonstrated the functional organization of these RF properties.

A Hierarchical Statistical Model of Natural Images Explains Tuning Properties in V2

Previous theoretical and experimental studies have demonstrated tight relationships between natural image statistics and neural representations in V1. In particular, receptive field properties similar to simple and complex cells have been shown to be inferable from sparse coding of natural images. However, whether such a relationship exists in higher areas has not been clarified. To address this question for V2, we trained a sparse coding model that took as input the output of a fixed V1-like model, which was in its turn fed a large variety of natural image patches as input. After the training, the model exhibited response properties that were qualitatively and quantitatively compatible with three major neurophysiological results on macaque V2, as follows: (1) homogeneous and heterogeneous integration of local orientations (Anzai et al., 2007); (2) a wide range of angle selectivities with biased sensitivities to one component orientation (Ito and Komatsu, 2004); and (3) exclusive length and width suppression (Schmid et al., 2014). The reproducibility was stable across variations in several model parameters. Further, a formal classification of the internal representations of the model units offered detailed interpretations of the experimental data, emphasizing that a novel type of model cell that could detect a combination of local orientations converging toward a single spatial point (potentially related to corner-like features) played an important role in reproducing tuning properties compatible with V2. These results are consistent with the idea that V2 uses a sparse code of natural images. Significance statement: Sparse coding theory has successfully explained a number of receptive field properties in V1; but how about in V2? This question has recently become important since a variety of properties distinct from V1 have been discovered in V2, and thus a more integrative understanding is called for. Our study shows that a hierarchical sparse coding model of natural images explains three major response properties known in the macaque V2. We further provide a detailed analysis revealing the roles of different kinds of model cells in explaining the V2-specific properties. Our results thus offer the first sparse coding account for receptive field properties in V2 that has extensive biological relevance.

Early monocular defocus disrupts the normal development of receptive-field structure in V2 neurons of macaque monkeys

Experiencing different quality images in the two eyes soon after birth can cause amblyopia, a developmental vision disorder. Amblyopic humans show the reduced capacity for judging the relative position of a visual target in reference to nearby stimulus elements (position uncertainty) and often experience visual image distortion. Although abnormal pooling of local stimulus information by neurons beyond striate cortex (V1) is often suggested as a neural basis of these deficits, extrastriate neurons in the amblyopic brain have rarely been studied using microelectrode recording methods. The receptive field (RF) of neurons in visual area V2 in normal monkeys is made up of multiple subfields that are thought to reflect V1 inputs and are capable of encoding the spatial relationship between local stimulus features. We created primate models of anisometropic amblyopia and analyzed the RF subfield maps for multiple nearby V2 neurons of anesthetized monkeys by using dynamic two-dimensional noise stimuli and reverse correlation methods. Unlike in normal monkeys, the subfield maps of V2 neurons in amblyopic monkeys were severely disorganized: subfield maps showed higher heterogeneity within each neuron as well as across nearby neurons. Amblyopic V2 neurons exhibited robust binocular suppression and the strength of the suppression was positively correlated with the degree of hereogeneity and the severity of amblyopia in individual monkeys. Our results suggest that the disorganized subfield maps and robust binocular suppression of amblyopic V2 neurons are likely to adversely affect the higher stages of cortical processing resulting in position uncertainty and image distortion.

Form-cue invariant second-order neuronal responses to contrast modulation in primate area V2

A fundamental task of the visual system is to extract figure-ground boundaries between images of objects, which in natural scenes are often defined not only by luminance differences but also by "second-order" contrast or texture differences. Responses to contrast modulation (CM) and other second-order stimuli have been extensively studied in human psychophysics, but the neuronal substrates of second-order responses in nonhuman primates remain poorly understood. In this study, we have recorded single neurons in area V2 of macaque monkeys, using both CM patterns as well as conventional luminance modulation (LM) gratings. CM stimuli were constructed from stationary sine wave grating carrier patterns, which were modulated by drifting envelope gratings of a lower spatial frequency. We found approximately one-third of visually responsive V2 neurons responded to CM stimuli with a pronounced selectivity to carrier spatial frequencies, and often orientations, that were clearly outside the neurons' passbands for LM gratings. These neurons were "form-cue invariant" in that their tuning to CM envelope spatial frequency and orientation was very similar to that for LM gratings. Neurons were tuned to carrier spatial frequencies that were typically 2-4 octaves higher than their optimal envelope spatial frequencies, similar to results from human psychophysics. These results are distinct from CM responses arising from surround suppression, but could be understood in terms of a filter-rectify-filter model. Such neurons could provide a functionally useful and explicit representation of segmentation boundaries as well as a plausible neural substrate for human perception of second-order boundaries.

Contextual modulation and stimulus selectivity in extrastriate cortex

Contextual modulation is observed throughout the visual system, using techniques ranging from single-neuron recordings to behavioral experiments. Its role in generating feature selectivity within the retina and primary visual cortex has been extensively described in the literature. Here, we describe how similar computations can also elaborate feature selectivity in the extrastriate areas of both the dorsal and ventral streams of the primate visual system. We discuss recent work that makes use of normalization models to test specific roles for contextual modulation in visual cortex function. We suggest that contextual modulation renders neuronal populations more selective for naturalistic stimuli. Specifically, we discuss contextual modulation's role in processing optic flow in areas MT and MST and for representing naturally occurring curvature and contours in areas V4 and IT. We also describe how the circuitry that supports contextual modulation is robust to variations in overall input levels. Finally, we describe how this theory relates to other hypothesized roles for contextual modulation.

Oblique effect in visual area 2 of macaque monkeys

The neural basis of an oblique effect, a reduced visual sensitivity for obliquely oriented stimuli, has been a matter of considerable debate. We have analyzed the orientation tuning of a relatively large number of neurons in the primary visual cortex (V1) and visual area 2 (V2) of anesthetized and paralyzed macaque monkeys. Neurons in V2 but not V1 of macaque monkeys showed clear oblique effects. This orientation anisotropy in V2 was more robust for those neurons that preferred higher spatial frequencies. We also determined whether V1 and V2 neurons exhibit a similar orientation anisotropy soon after birth. The oblique effect was absent in V1 of 4- and 8-week-old infant monkeys, but their V2 neurons showed a significant oblique effect. This orientation anisotropy in infant V2 was milder than that in adults. The results suggest that the oblique effect emerges in V2 based on the pattern of the connections that are established before birth and enhanced by the prolonged experience-dependent modifications of the neural circuitry in V2.

The fine structure of shape tuning in area V4

Previous studies have shown that neurons in area V4 are involved in the processing of shapes of intermediate complexity and are sensitive to curvature. These studies also suggest that curvature-tuned neurons are position invariant. We sought to examine the mechanisms that endow V4 neurons with these properties. Consistent with previous studies, we found that response rank order to the most- and least-preferred stimuli was preserved throughout the receptive field. However, a fine-grained analysis of shape tuning revealed a surprising result: V4 neurons tuned to highly curved shapes exhibit very limited translation invariance. At a fine spatial scale, these neurons exhibit local variation in orientation. In contrast, neurons that prefer straight contours exhibit spatially invariant orientation-tuning and homogenous fine-scale orientation maps. Both of these patterns are consistent with a simple orientation-pooling model, with tuning for straight or curved shapes resulting, respectively, from pooling of homogenous or heterogeneous orientation signals inherited from early visual areas.

Receptive-field subfields of V2 neurons in macaque monkeys are adult-like near birth

Infant primates can discriminate texture-defined form despite their relatively low visual acuity. The neuronal mechanisms underlying this remarkable visual capacity of infants have not been studied in nonhuman primates. Since many V2 neurons in adult monkeys can extract the local features in complex stimuli that are required for form vision, we used two-dimensional dynamic noise stimuli and local spectral reverse correlation to measure whether the spatial map of receptive-field subfields in individual V2 neurons is sufficiently mature near birth to capture local features. As in adults, most V2 neurons in 4-week-old monkeys showed a relatively high degree of homogeneity in the spatial matrix of facilitatory subfields. However, ∼25% of V2 neurons had the subfield map where the neighboring facilitatory subfields substantially differed in their preferred orientations and spatial frequencies. Over 80% of V2 neurons in both infants and adults had "tuned" suppressive profiles in their subfield maps that could alter the tuning properties of facilitatory profiles. The differences in the preferred orientations between facilitatory and suppressive profiles were relatively large but extended over a broad range. Response immaturities in infants were mild; the overall strength of facilitatory subfield responses was lower than that in adults, and the optimal correlation delay ("latency") was longer in 4-week-old infants. These results suggest that as early as 4 weeks of age, the spatial receptive-field structure of V2 neurons is as complex as in adults and the ability of V2 neurons to compare local features of neighboring stimulus elements is nearly adult like.

A three-layer model of natural image statistics

An important property of visual systems is to be simultaneously both selective to specific patterns found in the sensory input and invariant to possible variations. Selectivity and invariance (tolerance) are opposing requirements. It has been suggested that they could be joined by iterating a sequence of elementary selectivity and tolerance computations. It is, however, unknown what should be selected or tolerated at each level of the hierarchy. We approach this issue by learning the computations from natural images. We propose and estimate a probabilistic model of natural images that consists of three processing layers. Two natural image data sets are considered: image patches, and complete visual scenes downsampled to the size of small patches. For both data sets, we find that in the first two layers, simple and complex cell-like computations are performed. In the third layer, we mainly find selectivity to longer contours; for patch data, we further find some selectivity to texture, while for the downsampled complete scenes, some selectivity to curvature is observed.