Developmentally stable representations of naturalistic image structure in macaque visual cortex

We studied visual development in macaque monkeys using texture stimuli, matched in local spectral content but varying in "naturalistic" structure. In adult monkeys, naturalistic textures preferentially drive neurons in areas V2 and V4, but not V1. We paired behavioral measurements of naturalness sensitivity with separately-obtained neuronal population recordings from neurons in areas V1, V2, V4, and inferotemporal cortex (IT). We made behavioral measurements from 16 weeks of age and physiological measurements as early as 20 weeks, and continued through 56 weeks. Behavioral sensitivity reached half of maximum at roughly 25 weeks of age. Neural sensitivities remained stable from the earliest ages tested. As in adults, neural sensitivity to naturalistic structure increased from V1 to V2 to V4. While sensitivities in V2 and IT were similar, the dimensionality of the IT representation was more similar to V4's than to V2's.

Foundations of visual form selectivity for neurons in macaque V1 and V2

We have measured the visually evoked activity of single neurons recorded in areas V1 and V2 of awake, fixating macaque monkeys, and captured their responses with a common computational model. We used a stimulus set composed of "droplets" of localized contrast, band-limited in orientation and spatial frequency; each brief stimulus contained a random superposition of droplets presented in and near the mapped receptive field. We accounted for neuronal responses with a 2-layer linear-nonlinear model, representing each receptive field by a combination of orientation- and scale-selective filters. We fit the data by jointly optimizing the model parameters to enforce sparsity and to prevent overfitting. We visualized and interpreted the fits in terms of an "afferent field" of nonlinearly combined inputs, dispersed in the 4 dimensions of space and spatial frequency. The resulting fits generally give a good account of the responses of neurons in both V1 and V2, capturing an average of 40% of the explainable variance in neuronal firing. Moreover, the resulting models predict neuronal responses to image families outside the test set, such as gratings of different orientations and spatial frequencies. Our results offer a common framework for understanding processing in the early visual cortex, and also demonstrate the ways in which the distributions of neuronal responses in V1 and V2 are similar but not identical.

Neuronal and behavioral responses to naturalistic texture images in macaque monkeys

The visual world is richly adorned with texture, which can serve to delineate important elements of natural scenes. In anesthetized macaque monkeys, selectivity for the statistical features of natural texture is weak in V1, but substantial in V2, suggesting that neuronal activity in V2 might directly support texture perception. To test this, we investigated the relation between single cell activity in macaque V1 and V2 and simultaneously measured behavioral judgments of texture. We generated stimuli along a continuum between naturalistic texture and phase-randomized noise and trained two macaque monkeys to judge whether a sample texture more closely resembled one or the other extreme. Analysis of responses revealed that individual V1 and V2 neurons carried much less information about texture naturalness than behavioral reports. However, the sensitivity of V2 neurons, especially those preferring naturalistic textures, was significantly closer to that of behavior compared with V1. The firing of both V1 and V2 neurons predicted perceptual choices in response to repeated presentations of the same ambiguous stimulus in one monkey, despite low individual neural sensitivity. However, neither population predicted choice in the second monkey. We conclude that neural responses supporting texture perception likely continue to develop downstream of V2. Further, combined with neural data recorded while the same two monkeys performed an orientation discrimination task, our results demonstrate that choice-correlated neural activity in early sensory cortex is unstable across observers and tasks, untethered from neuronal sensitivity, and thus unlikely to reflect a critical aspect of the formation of perceptual decisions.

Significance statement

As visual signals propagate along the cortical hierarchy, they encode increasingly complex aspects of the sensory environment and likely have a more direct relationship with perceptual experience. We replicate and extend previous results from anesthetized monkeys differentiating the selectivity of neurons along the first step in cortical vision from area V1 to V2. However, our results further complicate efforts to establish neural signatures that reveal the relationship between perception and the neuronal activity of sensory populations. We find that choice-correlated activity in V1 and V2 is unstable across different observers and tasks, and also untethered from neuronal sensitivity and other features of nonsensory response modulation.

DREDge: robust motion correction for high-density extracellular recordings across species

High-density microelectrode arrays (MEAs) have opened new possibilities for systems neuroscience in human and non-human animals, but brain tissue motion relative to the array poses a challenge for downstream analyses, particularly in human recordings. We introduce DREDge (Decentralized Registration of Electrophysiology Data), a robust algorithm which is well suited for the registration of noisy, nonstationary extracellular electrophysiology recordings. In addition to estimating motion from spikes in the action potential (AP) frequency band, DREDge enables automated tracking of motion at high temporal resolution in the local field potential (LFP) frequency band. In human intraoperative recordings, which often feature fast (period <1s) motion, DREDge correction in the LFP band enabled reliable recovery of evoked potentials, and significantly reduced single-unit spike shape variability and spike sorting error. Applying DREDge to recordings made during deep probe insertions in nonhuman primates demonstrated the possibility of tracking probe motion of centimeters across several brain regions while simultaneously mapping single unit electrophysiological features. DREDge reliably delivered improved motion correction in acute mouse recordings, especially in those made with an recent ultra-high density probe. We also implemented a procedure for applying DREDge to recordings made across tens of days in chronic implantations in mice, reliably yielding stable motion tracking despite changes in neural activity across experimental sessions. Together, these advances enable automated, scalable registration of electrophysiological data across multiple species, probe types, and drift cases, providing a stable foundation for downstream scientific analyses of these rich datasets.

Shared mechanisms drive ocular following and motion perception

How features of complex visual patterns combine to drive perception and eye movements is not well understood. We simultaneously assessed human observers' perceptual direction estimates and ocular following responses (OFR) evoked by moving plaids made from two summed gratings with varying contrast ratios. When the gratings were of equal contrast, observers' eye movements and perceptual reports followed the motion of the plaid pattern. However, when the contrasts were unequal, eye movements and reports during early phases of the OFR were biased toward the direction of the high-contrast grating component; during later phases, both responses more closely followed the plaid pattern direction. The shift from component- to pattern-driven behavior resembles the shift in tuning seen under similar conditions in neuronal responses recorded from monkey MT. Moreover, for some conditions, pattern tracking and perceptual reports were correlated on a trial-by-trial basis. The OFR may therefore provide a precise behavioural read-out of the dynamics of neural motion integration for complex visual patterns.

Contrast and Luminance Gain Control in the Macaque's Lateral Geniculate Nucleus

There is substantial variation in the mean and variance of light levels (luminance and contrast) in natural visual scenes. Retinal ganglion cells maintain their sensitivity despite this variation using two adaptive mechanisms, which control how responses depend on luminance and on contrast. However, the nature of each mechanism and their interactions downstream of the retina are unknown. We recorded neurons in the magnocellular and parvocellular layers of the lateral geniculate nucleus (LGN) in anesthetized adult male macaques and characterized how their responses adapt to changes in contrast and luminance. As contrast increases, neurons in the magnocellular layers maintain sensitivity to high temporal frequency stimuli but attenuate sensitivity to low-temporal frequency stimuli. Neurons in the parvocellular layers do not adapt to changes in contrast. As luminance increases, both magnocellular and parvocellular cells increase their sensitivity to high-temporal frequency stimuli. Adaptation to luminance is independent of adaptation to contrast, as previously reported for LGN neurons in the cat. Our results are similar to those previously reported for macaque retinal ganglion cells, suggesting that adaptation to luminance and contrast result from two independent mechanisms that are retinal in origin.

Sensitivity to naturalistic texture relies primarily on high spatial frequencies

Natural images contain information at multiple spatial scales. Though we understand how early visual mechanisms split multiscale images into distinct spatial frequency channels, we do not know how the outputs of these channels are processed further by mid-level visual mechanisms. We have recently developed a texture discrimination task that uses synthetic, multi-scale, "naturalistic" textures to isolate these mid-level mechanisms. Here, we use three experimental manipulations (image blur, image rescaling, and eccentric viewing) to show that perceptual sensitivity to naturalistic structure is strongly dependent on features at high object spatial frequencies (measured in cycles/image). As a result, sensitivity depends on a texture acuity limit, a property of the visual system that sets the highest retinal spatial frequency (measured in cycles/degree) at which observers can detect naturalistic features. Analysis of the texture images using a model observer analysis shows that naturalistic image features at high object spatial frequencies carry more task-relevant information than those at low object spatial frequencies. That is, the dependence of sensitivity on high object spatial frequencies is a property of the texture images, rather than a property of the visual system. Accordingly, we find human observers' ability to extract naturalistic information (their efficiency) is similar for all object spatial frequencies. We conclude that the mid-level mechanisms that underlie perceptual sensitivity effectively extract information from all image features below the texture acuity limit, regardless of their retinal and object spatial frequency.

Altered functional interactions between neurons in primary visual cortex of macaque monkeys with experimental amblyopia

Amblyopia, a disorder in which vision through one of the eyes is degraded, arises because of defective processing of information by the visual system. Amblyopia often develops in humans after early misalignment of the eyes (strabismus) and can be simulated in macaque monkeys by artificially inducing strabismus. In such amblyopic animals, single-unit responses in primary visual cortex (V1) are appreciably reduced when evoked by the amblyopic eye compared with the other (fellow) eye. However, this degradation in single V1 neuron responsivity is not commensurate with the marked losses in visual sensitivity and resolution measured behaviorally. Here we explored the idea that changes in patterns of coordinated activity across populations of V1 neurons may contribute to degraded visual representations in amblyopia, potentially making it more difficult to read out evoked activity to support perceptual decisions. We studied the visually evoked activity of V1 neuronal populations in three macaques (Macaca nemestrina) with strabismic amblyopia and in one control animal. Activity driven through the amblyopic eye was diminished, and these responses also showed more interneuronal correlation at all stimulus contrasts than responses driven through the fellow eye or responses in the control animal. A decoding analysis showed that responses driven through the amblyopic eye carried less visual information than other responses. Our results suggest that part of the reduced visual capacity of amblyopes may be due to changes in the patterns of functional interaction among neurons in V1.NEW & NOTEWORTHY Previous work on the neurophysiological basis of amblyopia has largely focused on relating behavioral deficits to changes in visual processing by single neurons in visual cortex. In this study, we recorded simultaneously from populations of primary visual cortical (V1) neurons in macaques with amblyopia. We found changes in the strength and pattern of shared response variability between neurons. These changes in neuronal interactions could impair the visual representations of V1 populations driven by the amblyopic eye.

A Conversation with Jacob Nachmias

We are sad to report that Professor Jacob (Jack) Nachmias passed away on March 2, 2019. Nachmias was born in Athens, Greece, on June 9, 1928. To escape the Nazis, he and his family came to the United States in 1939. He received his undergraduate degree from Cornell University and then an MA from Swarthmore College, where he worked with Hans Wallach and Wolfgang Kohler; his PhD in Psychology was from Harvard University. Nachmias spent the majority of his career as a Professor of Psychology at the University of Pennsylvania. He made fundamental contributions to our understanding of vision, most notably through the study of eye movements, the development of signal detection theory and forced-choice psychophysical methods, and the psychophysical characterization of spatial-frequency-selective visual channels. Nachmias' work was recognized by his election to the National Academy of Sciences and receipt of the Optical Society's Tillyer Award.

Responses of neurons in macaque MT to unikinetic plaids

Response properties of MT neurons are often studied with "bikinetic" plaid stimuli, which consist of two superimposed sine wave gratings moving in different directions. Oculomotor studies using "unikinetic plaids" in which only one of the two superimposed gratings moves suggest that the eyes first move reflexively in the direction of the moving grating and only later converge on the perceived direction of the moving pattern. MT has been implicated as the source of visual signals that drives these responses. We wanted to know whether stationary gratings, which have little effect on MT cells when presented alone, would influence MT responses when paired with a moving grating. We recorded extracellularly from neurons in area MT and measured responses to stationary and moving gratings, and to their sums: bikinetic and unikinetic plaids. As expected, stationary gratings presented alone had a very modest influence on the activity of MT neurons. Responses to moving gratings and bikinetic plaids were similar to those previously reported and revealed cells selective for the motion of plaid patterns and of their components (pattern and component cells). When these neurons were probed with unikinetic plaids, pattern cells shifted their direction preferences in a way that revealed the influence of the static grating. Component cell preferences shifted little or not at all. These results support the notion that pattern-selective neurons in area MT integrate component motions that differ widely in speed, and that they do so in a way that is consistent with an intersection-of-constraints model.NEW & NOTEWORTHY Human perceptual and eye movement responses to moving gratings are influenced by adding a second, static grating to create a "unikinetic" plaid. Cells in MT do not respond to static gratings, but those gratings still influence the direction selectivity of some MT cells. The cells influenced by static gratings are those tuned for the motion of global patterns, but not those tuned only for the individual components of moving targets.

Slow gain fluctuations limit benefits of temporal integration in visual cortex

Sensory neurons represent stimulus information with sequences of action potentials that differ across repeated measurements. This variability limits the information that can be extracted from momentary observations of a neuron's response. It is often assumed that integrating responses over time mitigates this limitation. However, temporal response correlations can reduce the benefits of temporal integration. We examined responses of individual orientation-selective neurons in the primary visual cortex of two macaque monkeys performing an orientation-discrimination task. The signal-to-noise ratio of temporally integrated responses increased for durations up to a few hundred milliseconds but saturated for longer durations. This was true even when cells exhibited little or no adaptation in their response levels. These observations are well explained by a statistical response model in which spikes arise from a Poisson process whose stimulus-dependent rate is modulated by slow, stimulus-independent fluctuations in gain. The response variability arising from the Poisson process is reduced by temporal integration, but the slow modulatory nature of variability due to gain fluctuations is not. Slow gain fluctuations therefore impose a fundamental limit on the benefits of temporal integration.

Contextual modulation of sensitivity to naturalistic image structure in macaque V2

The stimulus selectivity of neurons in V1 is well known, as is the finding that their responses can be affected by visual input to areas outside of the classical receptive field. Less well understood are the ways selectivity is modified as signals propagate to visual areas beyond V1, such as V2. We recently proposed a role for V2 neurons in representing the higher order statistical dependencies found in images of naturally occurring visual texture. V2 neurons, but not V1 neurons, respond more vigorously to "naturalistic" images that contain these dependencies than to "noise" images that lack them. In this work, we examine the dependency of these effects on stimulus size. For most V2 neurons, the preference for naturalistic over noise stimuli was modest when presented in small patches and gradually strengthened with increasing size, suggesting that the mechanisms responsible for this enhanced sensitivity operate over regions of the visual field that are larger than the classical receptive field. Indeed, we found that surround suppression was stronger for noise than for naturalistic stimuli and that the preference for large naturalistic stimuli developed over a delayed time course consistent with lateral or feedback connections. These findings are compatible with a spatially broad facilitatory mechanism that is absent in V1 and suggest that a distinct role for the receptive field surround emerges in V2 along with sensitivity for more complex image structure. NEW & NOTEWORTHY The responses of neurons in visual cortex are often affected by visual input delivered to regions of the visual field outside of the conventionally defined receptive field, but the significance of such contextual modulations are not well understood outside of area V1. We studied the importance of regions beyond the receptive field in establishing a novel form of selectivity for the statistical dependencies contained in natural visual textures that first emerges in area V2.

Development of visual cortical function in infant macaques: A BOLD fMRI study

Functional brain development is not well understood. In the visual system, neurophysiological studies in nonhuman primates show quite mature neuronal properties near birth although visual function is itself quite immature and continues to develop over many months or years after birth. Our goal was to assess the relative development of two main visual processing streams, dorsal and ventral, using BOLD fMRI in an attempt to understand the global mechanisms that support the maturation of visual behavior. Seven infant macaque monkeys (Macaca mulatta) were repeatedly scanned, while anesthetized, over an age range of 102 to 1431 days. Large rotating checkerboard stimuli induced BOLD activation in visual cortices at early ages. Additionally we used static and dynamic Glass pattern stimuli to probe BOLD responses in primary visual cortex and two extrastriate areas: V4 and MT-V5. The resulting activations were analyzed with standard GLM and multivoxel pattern analysis (MVPA) approaches. We analyzed three contrasts: Glass pattern present/absent, static/dynamic Glass pattern presentation, and structured/random Glass pattern form. For both GLM and MVPA approaches, robust coherent BOLD activation appeared relatively late in comparison to the maturation of known neuronal properties and the development of behavioral sensitivity to Glass patterns. Robust differential activity to Glass pattern present/absent and dynamic/static stimulus presentation appeared first in V1, followed by V4 and MT-V5 at older ages; there was no reliable distinction between the two extrastriate areas. A similar pattern of results was obtained with the two analysis methods, although MVPA analysis showed reliable differential responses emerging at later ages than GLM. Although BOLD responses to large visual stimuli are detectable, our results with more refined stimuli indicate that global BOLD activity changes as behavioral performance matures. This reflects an hierarchical development of the visual pathways. Since fMRI BOLD reflects neural activity on a population level, our results indicate that, although individual neurons might be adult-like, a longer maturation process takes place on a population level.

Opportunities and challenges in modeling human brain disorders in transgenic primates

Molecular genetic tools have had a profound impact on neuroscience, but until recently their application has largely been confined to a few model species, most notably mouse, zebrafish, Drosophila melanogaster and Caenorhabditis elegans. With the development of new genome engineering technologies such as CRISPR, it is becoming increasingly feasible to apply these molecular tools in a wider range of species, including nonhuman primates. This will lead to many opportunities for brain research, but it will also pose challenges. Here we identify some of these opportunities and challenges in light of recent and foreseeable technological advances and offer some suggestions. Our main focus is on the creation of new primate disease models for understanding the pathological mechanisms of brain disorders and for developing new approaches to effective treatment. However, we also emphasize that primate genetic models have great potential to address many fundamental questions about brain function, providing an essential foundation for future progress in disease research.

Saccadic latency in amblyopia

We measured saccadic latencies in a large sample (total n = 459) of individuals with amblyopia or risk factors for amblyopia, e.g., strabismus or anisometropia, and normal control subjects. We presented an easily visible target randomly to the left or right, 3.5° from fixation. The interocular difference in saccadic latency is highly correlated with the interocular difference in LogMAR (Snellen) acuity-as the acuity difference increases, so does the latency difference. Strabismic and strabismic-anisometropic amblyopes have, on average, a larger difference between their eyes in LogMAR acuity than anisometropic amblyopes and thus their interocular latency difference is, on average, significantly larger than anisometropic amblyopes. Despite its relation to LogMAR acuity, the longer latency in strabismic amblyopes cannot be attributed either to poor resolution or to reduced contrast sensitivity, because their interocular differences in grating acuity and in contrast sensitivity are roughly the same as for anisometropic amblyopes. The correlation between LogMAR acuity and saccadic latency arises because of the confluence of two separable effects in the strabismic amblyopic eye-poor letter recognition impairs LogMAR acuity while an intrinsic sluggishness delays reaction time. We speculate that the frequent microsaccades and the accompanying attentional shifts, made while strabismic amblyopes struggle to maintain fixation with their amblyopic eyes, result in all types of reactions being irreducibly delayed.

Properties of pattern and component direction-selective cells in area MT of the macaque

Neurons in area MT/V5 of the macaque visual cortex encode visual motion. Some cells are selective for the motion of oriented features (component direction-selective, CDS); others respond to the true direction of complex patterns (pattern-direction selective, PDS). There is a continuum of selectivity in MT, with CDS cells at one extreme and PDS cells at the other; we compute a pattern index that captures this variation. It is unknown how a neuron's pattern index is related to its other tuning characteristics. We therefore analyzed the responses of 792 MT cells recorded in the course of other experiments from opiate-anesthetized macaque monkeys, as a function of the direction, spatial frequency, drift rate, size, and contrast of sinusoidal gratings and of the direction and speed of random-dot textures. We also compared MT responses to those of 718 V1 cells. As expected, MT cells with higher pattern index tended to have stronger direction selectivity and broader direction tuning to gratings, and they responded better to plaids than to gratings. Strongly PDS cells also tended to have smaller receptive fields and stronger surround suppression. Interestingly, they also responded preferentially to higher drift rates and higher speeds of moving dots. The spatial frequency preferences of PDS cells depended strongly on their preferred temporal frequencies, whereas these preferences were independent in component-selective cells. Pattern direction selectivity is statistically associated with many response properties of MT cells but not strongly associated with any particular property. Pattern-selective signals are thus available in association with most other signals exported by MT.

Origin and Function of Tuning Diversity in Macaque Visual Cortex

Neurons in visual cortex vary in their orientation selectivity. We measured responses of V1 and V2 cells to orientation mixtures and fit them with a model whose stimulus selectivity arises from the combined effects of filtering, suppression, and response nonlinearity. The model explains the diversity of orientation selectivity with neuron-to-neuron variability in all three mechanisms, of which variability in the orientation bandwidth of linear filtering is the most important. The model also accounts for the cells' diversity of spatial frequency selectivity. Tuning diversity is matched to the needs of visual encoding. The orientation content found in natural scenes is diverse, and neurons with different selectivities are adapted to different stimulus configurations. Single orientations are better encoded by highly selective neurons, while orientation mixtures are better encoded by less selective neurons. A diverse population of neurons therefore provides better overall discrimination capabilities for natural images than any homogeneous population.

Brains, genes, and primates

One of the great strengths of the mouse model is the wide array of genetic tools that have been developed. Striking examples include methods for directed modification of the genome, and for regulated expression or inactivation of genes. Within neuroscience, it is now routine to express reporter genes, neuronal activity indicators, and opsins in specific neuronal types in the mouse. However, there are considerable anatomical, physiological, cognitive, and behavioral differences between the mouse and the human that, in some areas of inquiry, limit the degree to which insights derived from the mouse can be applied to understanding human neurobiology. Several recent advances have now brought into reach the goal of applying these tools to understanding the primate brain. Here we describe these advances, consider their potential to advance our understanding of the human brain and brain disorders, discuss bioethical considerations, and describe what will be needed to move forward.

Representation of Naturalistic Image Structure in the Primate Visual Cortex

The perception of complex visual patterns emerges from neuronal activity in a cascade of areas in the primate cerebral cortex. We have probed the early stages of this cascade with "naturalistic" texture stimuli designed to capture key statistical features of natural images. Humans can recognize and classify these synthetic images and are insensitive to distortions that do not alter the local values of these statistics. The responses of neurons in the primary visual cortex, V1, are relatively insensitive to the statistical information in these textures. However, in the area immediately downstream, V2, cells respond more vigorously to these stimuli than to matched control stimuli. Humans show blood-oxygen-level-dependent functional magnetic resonance imaging (BOLD fMRI responses in V1 and V2) that are consistent with the neuronal measurements in macaque. These fMRI measurements, as well as neurophysiological work by others, show that true natural scenes become a more prominent driving feature of cortex downstream from V2. These results suggest a framework for thinking about how information about elementary visual features is transformed into the specific representations of scenes and objects found in areas higher in the visual pathway.

Population representation of visual information in areas V1 and V2 of amblyopic macaques

Amblyopia is a developmental disorder resulting in poor vision in one eye. The mechanism by which input to the affected eye is prevented from reaching the level of awareness remains poorly understood. We recorded simultaneously from large populations of neurons in the supragranular layers of areas V1 and V2 in 6 macaques that were made amblyopic by rearing with artificial strabismus or anisometropia, and 1 normally reared control. In agreement with previous reports, we found that cortical neuronal signals driven through the amblyopic eyes were reduced, and that cortical neurons were on average more strongly driven by the non-amblyopic than by the amblyopic eyes. We analyzed multiunit recordings using standard population decoding methods, and found that visual signals from the amblyopic eye, while weakened, were not degraded enough to explain the behavioral deficits. Thus additional losses must arise in downstream processing. We tested the idea that under monocular viewing conditions, only signals from neurons dominated by - rather than driven by - the open eye might be used. This reduces the proportion of neuronal signals available from the amblyopic eye, and amplifies the interocular difference observed at the level of single neurons. We conclude that amblyopia might arise in part from degradation in the neuronal signals from the amblyopic eye, and in part from a reduction in the number of signals processed by downstream areas.

Putting big data to good use in neuroscience

Big data has transformed fields such as physics and genomics. Neuroscience is set to collect its own big data sets, but to exploit its full potential, there need to be ways to standardize, integrate and synthesize diverse types of data from different levels of analysis and across species. This will require a cultural shift in sharing data across labs, as well as to a central role for theorists in neuroscience research.

Temporal and spatial limits of pattern motion sensitivity in macaque MT neurons

Many neurons in visual cortical area MT signal the direction of motion of complex visual patterns, such as plaids composed of two superimposed drifting gratings. To compute the direction of pattern motion, MT neurons combine component motion signals over time and space. To determine the spatial and temporal limits of signal integration, we measured the responses of single MT neurons to a novel set of "pseudoplaid" stimuli in which the component gratings were alternated in time or space. As the temporal or spatial separation of the component gratings increased, neuronal selectivity for the direction of pattern motion decreased. Using descriptive models of signal integration, we inferred the temporal and spatial structure of the mechanisms that compute pattern direction selectivity. The median time constant for integration was roughly 10 ms, a timescale characteristic of integration by single cortical pyramidal neurons. The median spatial integration field was roughly one-third of the MT receptive field diameter, suggesting that the spatial limits are set by stages of processing in earlier areas of visual cortex where receptive fields are smaller than in MT. Interestingly, pattern direction-selective neurons had shorter temporal integration times than component direction-selective neurons but similar spatial integration windows. We conclude that pattern motion can only be signaled by MT neurons when the component motion signals co-occur within relatively narrow spatial and temporal limits. We interpret these results in the framework of recent hierarchical models of MT.

A functional and perceptual signature of the second visual area in primates

There is no generally accepted account of the function of the second visual cortical area (V2), partly because no simple response properties robustly distinguish V2 neurons from those in primary visual cortex (V1). We constructed synthetic stimuli replicating the higher-order statistical dependencies found in natural texture images and used them to stimulate macaque V1 and V2 neurons. Most V2 cells responded more vigorously to these textures than to control stimuli lacking naturalistic structure; V1 cells did not. Functional magnetic resonance imaging (fMRI) measurements in humans revealed differences between V1 and V2 that paralleled the neuronal measurements. The ability of human observers to detect naturalistic structure in different types of texture was well predicted by the strength of neuronal and fMRI responses in V2 but not in V1. Together, these results reveal a particular functional role for V2 in the representation of natural image structure.

Efficient and direct estimation of a neural subunit model for sensory coding

Many visual and auditory neurons have response properties that are well explained by pooling the rectified responses of a set of spatially shifted linear filters. These filters cannot be estimated using spike-triggered averaging (STA). Subspace methods such as spike-triggered covariance (STC) can recover multiple filters, but require substantial amounts of data, and recover an orthogonal basis for the subspace in which the filters reside rather than the filters themselves. Here, we assume a linear-nonlinear-linear-nonlinear (LN-LN) cascade model in which the first linear stage is a set of shifted ('convolutional') copies of a common filter, and the first nonlinear stage consists of rectifying scalar nonlinearities that are identical for all filter outputs. We refer to these initial LN elements as the 'subunits' of the receptive field. The second linear stage then computes a weighted sum of the responses of the rectified subunits. We present a method for directly fitting this model to spike data, and apply it to both simulated and real neuronal data from primate V1. The subunit model significantly outperforms STA and STC in terms of cross-validated accuracy and efficiency.

Dissociation of neuronal and psychophysical responses to local and global motion

Most neurons in cortical area MT (V5) are strongly direction selective, and their activity is closely associated with the perception of visual motion. These neurons have large receptive fields built by combining inputs with smaller receptive fields that respond to local motion. Humans integrate motion over large areas and can perceive what has been referred to as global motion. The large size and direction selectivity of MT receptive fields suggests that MT neurons may represent global motion. We have explored this possibility by measuring responses to a stimulus in which the directions of simultaneously presented local and global motion are independently controlled. Surprisingly, MT responses depended only on the local motion and were unaffected by the global motion. Yet, under similar conditions, human observers perceive global motion and are impaired in discriminating local motion. Although local motion perception might depend on MT signals, global motion perception depends on mechanisms qualitatively different from those in MT. Motion perception therefore does not depend on a single cortical area but reflects the action and interaction of multiple brain systems.

Visual deficits in anisometropia

Amblyopia is usually associated with the presence of anisometropia, strabismus or both early in life. We set out to explore quantitative relationships between the degree of anisometropia and the loss of visual function, and to examine how the presence of strabismus affects visual function in observers with anisometropia. We measured optotype acuity, Pelli-Robson contrast sensitivity and stereoacuity in 84 persons with anisometropia and compared their results with those of 27 persons with high bilateral refractive error (isoametropia) and 101 persons with both strabismus and anisometropia. All subjects participated in a large-scale study of amblyopia (McKee et al., 2003). We found no consistent visual abnormalities in the strong eye, and therefore report only on vision in the weaker, defined as the eye with lower acuity. LogMAR acuity falls off markedly with increasing anisometropia in non-strabismic anisometropes, while contrast sensitivity is much less affected. Acuity degrades rapidly with increases in both hyperopic and myopic anisometropia, but the risk of amblyopia is about twice as great in hyperopic than myopic anisometropes of comparable refractive imbalance. For a given degree of refractive imbalance, strabismic anisometropes perform considerably worse than anisometropes without strabismus--visual acuity for strabismics was on average 2.5 times worse than for non-strabismics with similar anisometropia. For observers with equal refractive error in the two eyes there is very little change in acuity or sensitivity with increasing (bilateral) refractive error except for one extreme individual (bilaterally refractive error of -15 D). Most pure anisometropes with interocular differences less than 4D retain some stereopsis, and the degree is correlated with the acuity of the weak eye. We conclude that even modest interocular differences in refractive error can influence visual function.

Does the noise matter? Effects of different kinematogram types on smooth pursuit eye movements and perception

We investigated how the human visual system and the pursuit system react to visual motion noise. We presented three different types of random-dot kinematograms at five different coherence levels. For transparent motion, the signal and noise labels on each dot were preserved throughout each trial, and noise dots moved with the same speed as the signal dots but in fixed random directions. For white noise motion, every 20 ms the signal and noise labels were randomly assigned to each dot and noise dots appeared at random positions. For Brownian motion, signal and noise labels were also randomly assigned, but the noise dots moved at the signal speed in a direction that varied randomly from moment to moment. Neither pursuit latency nor early eye acceleration differed among the different types of kinematograms. Late acceleration, pursuit gain, and perceived speed all depended on kinematogram type, with good agreement between pursuit gain and perceived speed. For transparent motion, pursuit gain and perceived speed were independent of coherence level. For white and Brownian motions, pursuit gain and perceived speed increased with coherence but were higher for white than for Brownian motion. This suggests that under our conditions, the pursuit system integrates across all directions of motion but not across all speeds.

Development of sensitivity to visual texture modulation in macaque monkeys

In human and non-human primates, higher form vision matures substantially later than spatial acuity and contrast sensitivity, as revealed by performance on such tasks as figure-ground segregation and contour integration. Our goal was to understand whether delayed maturation on these tasks was intrinsically form-dependent or, rather, related to the nature of spatial integration necessary for extracting task-relevant cues. We used an intermediate-level form task that did not call for extensive spatial integration. We trained monkeys (6-201 weeks) to discriminate the orientation of pattern modulation in a two-alternative forced choice paradigm. We presented two families of form patterns, defined by texture or contrast variations, and luminance-defined patterns for comparison. Infant monkeys could discriminate texture- and contrast-defined form as early as 6 weeks; sensitivity improved up to 40 weeks. Surprisingly, sensitivity for texture- and contrast-defined form matured earlier than for luminance-defined form. These results suggest that intermediate-level form vision develops in concert with basic spatial vision rather than following sequentially. Comparison with earlier results reveals that different aspects of form vision develop over different time courses, with processes that depend on comparing local image content maturing earlier than those requiring "global" linking of multiple visual elements across a larger spatial extent.