Organization of disparity-selective neurons in macaque area MT

Neurons selective for binocular disparity are found in a number of visual cortical areas in primates, but there is little evidence that any of these areas are specialized for disparity processing. We have examined the organization of disparity-selective neurons in the middle temporal visual area (MT), an area shown previously to contain an abundance of disparity-sensitive neurons. We recorded extracellularly from MT neurons at regularly spaced intervals along electrode penetrations that passed through MT either normal to the cortical surface or at a shallow oblique angle. Comparison of multiunit and single-unit recordings shows that neurons are clustered in MT according to their disparity selectivity. Across the surface of MT, disparity-selective neurons are found in discrete patches that are separated by regions of MT that exhibit poor disparity tuning. Within disparity-selective patches of MT, we typically observe a smooth progression of preferred disparities (e.g. , near to far) as our electrode travels parallel to the cortical surface. In electrode penetrations normal to the cortical surface, on the other hand, MT neurons generally have similar disparity tuning, with little variation from one recording site to the next. Thus disparity-tuned neurons are organized into cortical columns by preferred disparity, and preferred disparity is mapped systematically within larger, disparity-tuned patches of MT. Combined with other recent findings, the data suggest that MT plays an important role in stereoscopic depth perception in addition to its well known role in motion perception.

Cortical area MT and the perception of stereoscopic depth

Stereopsis is the perception of depth based on small positional differences between images formed on the two retinae (known as binocular disparity). Neurons that respond selectively to binocular disparity were first described three decades ago, and have since been observed in many visual areas of the primate brain, including V1, V2, V3, MT and MST. Although disparity-selective neurons are thought to form the neural substrate for stereopsis, the mere existence of disparity-selective neurons does not guarantee that they contribute to stereoscopic depth perception. Some disparity-selective neurons may play other roles, such as guiding vergence eye movements. Thus, the roles of different visual areas in stereopsis remain poorly defined. Here we show that visual area MT is important in stereoscopic vision: electrical stimulation of clusters of disparity-selective MT neurons can bias perceptual judgements of depth, and the bias is predictable from the disparity preference of neurons at the stimulation site. These results show that behaviourally relevant signals concerning stereoscopic depth are present in MT.

Tuning bandwidths for near-threshold stimuli in area MT

It is not known whether psychophysical performance depends primarily on small numbers of neurons optimally tuned to specific visual stimuli, or on larger populations of neurons that vary widely in their properties. Tuning bandwidths of single cells can provide important insight into this issue, yet most bandwidth measurements have been made using suprathreshold visual stimuli, whereas psychophysical measurements are frequently obtained near threshold. We therefore examined the directional tuning of cells in the middle temporal area (MT, or V5) using perithreshold, stochastic motion stimuli that we have employed extensively in combined psychophysical and physiological studies. The strength of the motion signal (coherence) in these displays can be varied independently of its direction. For each MT neuron, we characterized the directional bandwidth by fitting Gaussian functions to directional tuning data obtained at each of several motion coherences. Directional bandwidth increased modestly as the coherence of the stimulus was reduced. We then assessed the ability of MT neurons to discriminate opposed directions of motion along six equally spaced axes of motion spanning 180 degrees. A signal detection analysis yielded neurometric functions for each axis of motion, from which neural thresholds could be extracted. Neural thresholds remained surprisingly low as the axis of motion diverged from the neuron's preferred-null axis, forming a plateau of high to medium sensitivity that extended approximately 45 degrees on either side of the preferred-null axis. We conclude that directional tuning remains broad in MT when motion signals are reduced to near-threshold values. Thus directional information is widely distributed in MT, even near the limits of psychophysical performance. These observations support models in which relatively large numbers of signals are pooled to inform psychophysical decisions.

Temporal gating of neural signals during performance of a visual discrimination task

The flow of neural signals within the cerebral cortex must be subject to multiple controls as behaviour unfolds in time. In a visual discrimination task that includes a delay period, the transmission of sensory signals to circuitry that mediates memory, decision-making and motor-planning must be governed closely by 'filtering' or 'gating' mechanisms so that extraneous events occurring before, during or after presentation of the critical visual stimulus have little or no effect on the subject's behavioural responses. Here we study one such mechanism physiologically by applying electrical microstimulation to columns of directionally selective neurons in the middle temporal visual area at varying times during single trials of a direction-discrimination task. The behavioural effects of microstimulation varied strikingly according to the timing of delivery within the trial, indicating that signals produced by microstimulation may be subject to active 'gating'. Our results show several important features of this gating process: first, signal flow is modulated upwards on onset of the visual stimulus and downwards, typically with a slower time course, after stimulus offset; second, gating efficacy can be modified by behavioural training; and third, gating is implemented primarily downstream of the middle temporal visual area.

Neurophysiology: sensing and categorizing

Objects differ along many stimulus dimensions, but observers typically group them into fewer 'categories' according to their potential use or behavioral relevance. New experiments in awake, behaving monkeys open a window onto the process of stimulus categorization within the central nervous system.

The variable discharge of cortical neurons: implications for connectivity, computation, and information coding

Cortical neurons exhibit tremendous variability in the number and temporal distribution of spikes in their discharge patterns. Furthermore, this variability appears to be conserved over large regions of the cerebral cortex, suggesting that it is neither reduced nor expanded from stage to stage within a processing pathway. To investigate the principles underlying such statistical homogeneity, we have analyzed a model of synaptic integration incorporating a highly simplified integrate and fire mechanism with decay. We analyzed a "high-input regime" in which neurons receive hundreds of excitatory synaptic inputs during each interspike interval. To produce a graded response in this regime, the neuron must balance excitation with inhibition. We find that a simple integrate and fire mechanism with balanced excitation and inhibition produces a highly variable interspike interval, consistent with experimental data. Detailed information about the temporal pattern of synaptic inputs cannot be recovered from the pattern of output spikes, and we infer that cortical neurons are unlikely to transmit information in the temporal pattern of spike discharge. Rather, we suggest that quantities are represented as rate codes in ensembles of 50-100 neurons. These column-like ensembles tolerate large fractions of common synaptic input and yet covary only weakly in their spike discharge. We find that an ensemble of 100 neurons provides a reliable estimate of rate in just one interspike interval (10-50 msec). Finally, we derived an expression for the variance of the neural spike count that leads to a stable propagation of signal and noise in networks of neurons-that is, conditions that do not impose an accumulation or diminution of noise. The solution implies that single neurons perform simple algebra resembling averaging, and that more sophisticated computations arise by virtue of the anatomical convergence of novel combinations of inputs to the cortical column from external sources.

Sense and the single neuron: probing the physiology of perception

The newly defined field of cognitive neuroscience attempts to draw together the study of all brain mechanisms that underlie our mental life. Historically, the major sensory pathways have provided the most trustworthy insights into how the brain supports cognitive functions such as perception, attention, and short-term memory. The links between neural activity and perception, in particular, have been studied revealingly in recent decades. Here we review the striking progress in this area, giving particular emphasis to the kinds of neural events that underlie the perceptual judgments of conscious observers.

How is a sensory map read Out? Effects of microstimulation in visual area MT on saccades and smooth pursuit eye movements

To generate behavioral responses based on sensory input, motor areas of the brain must interpret, or "read out," signals from sensory maps. Our experiments tested several algorithms for how the motor systems for smooth pursuit and saccadic eye movements might extract a usable signal of target velocity from the distributed representation of velocity in the middle temporal visual area (MT or V5). Using microstimulation, we attempted to manipulate the velocity information within MT while monkeys tracked a moving visual stimulus. We examined the effects of the microstimulation on smooth pursuit and on the compensation for target velocity shown by saccadic eye movements. Microstimulation could alter both the speed and direction of the motion estimates of both types of eye movements and could also cause monkeys to generate pursuit even when the visual target was actually stationary. The pattern of alterations suggests that microstimulation can introduce an additional velocity signal into MT and that the pursuit and saccadic systems usually compute a vector average of the visually evoked and microstimulation-induced velocity signals (pursuit, 55 of 122 experiments; saccades, 70 of 122). Microstimulation effects in a few experiments were consistent with vector summation of these two signals (pursuit, 6 of 122; saccades, 2 of 122). In the remainder of the experiments, microstimulation caused either an apparent impairment in motion processing (pursuit, 47 of 122; saccades, 41 of 122) or had no effect (pursuit, 14 of 122; saccades, 9 of 122). Within individual experiments, the effects on pursuit and saccades were usually similar, but the occasional striking exception suggests that the two eye movement systems may perform motion computations somewhat independently.

Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys

We have previously shown that some neurons in extrastriate area MT are capable of signaling the global motion of complex patterns; neurons randomly sampled from V1, on the other hand, respond only to the motion of individual oriented components. Because only a small fraction of V1 neurons projects to MT, we wished to establish the processing hierarchy more precisely by studying the properties of those neurons projecting to MT, identified by antidromic responses to electrical stimulation of MT. The neurons that project from V1 to MT were directionally selective and, like other V1 neurons, responded only to the motion of the components of complex patterns. The projection neurons were predominantly "special complex," responsive to a broad range of spatial and temporal frequencies, and sensitive to very low stimulus contrasts. The projection neurons thus comprise a homogeneous and highly specialized subset of V1 neurons, consistent with the notion that V1 acts as clearing house of basic visual measurements, distributing information appropriately to higher cortical areas for specialized analysis.

Neurophysiology: neural fingerprints of visual attention

Pronounced effects of attention have been demonstrated in a region of visual cortex previously thought to be devoid of such influences; identifying the features critical for eliciting these effects should teach us a great deal about the neural underpinnings of visual attention.

A computational analysis of the relationship between neuronal and behavioral responses to visual motion

We have documented previously a close relationship between neuronal activity in the middle temporal visual area (MT or V5) and behavioral judgments of motion (Newsome et al., 1989; Salzman et al., 1990; Britten et al., 1992; Britten et al., 1996). We have now used numerical simulations to try to understand how neural signals in area MT support psychophysical decisions. We developed a model that pools neuronal responses drawn from our physiological data set and compares average responses in different pools to produce psychophysical decisions. The structure of the model allows us to assess the relationship between "neuronal" input signals and simulated psychophysical performance using the same methods we have applied to real experimental data. We sought to reconcile three experimental observations: psychophysical performance (threshold sensitivity to motion stimuli embedded in noise), a trial-by-trial covariation between the neural response and the monkey's choices, and a modest correlation between pairs of MT neurons in their variable responses to identical visual stimuli. Our results can be most accurately simulated if psychophysical decisions are based on pools of at least 100 weakly correlated sensory neurons. The neurons composing the pools must include a broader range of sensitivities than we encountered in our MT recordings, presumably because of the inclusion of neurons whose optimal stimulus is different from the one being discriminated. Central sources of noise degrade the signal-to-noise ratio of the pooled signal, but this degradation is relatively small compared with the noise typically carried by single cortical neurons. This suggests that our monkeys base near-threshold psychophysical judgments on signals carried by populations of weakly interacting neurons; these populations include many neurons that are not tuned optimally for the particular stimuli being discriminated.

Motion perception: seeing and deciding

The primate visual system offers unprecedented opportunities for investigating the neural basis of cognition. Even the simplest visual discrimination task requires processing of sensory signals, formation of a decision, and orchestration of a motor response. With our extensive knowledge of the primate visual and oculomotor systems as a base, it is now possible to investigate the neural basis of simple visual decisions that link sensation to action. Here we describe an initial study of neural responses in the lateral intraparietal area (LIP) of the cerebral cortex while alert monkeys discriminated the direction of motion in a visual display. A subset of LIP neurons carried high-level signals that may comprise a neural correlate of the decision process in our task. These signals are neither sensory nor motor in the strictest sense; rather they appear to reflect integration of sensory signals toward a decision appropriate for guiding movement. If this ultimately proves to be the case, several fascinating issues in cognitive neuroscience will be brought under rigorous physiological scrutiny.

A relationship between behavioral choice and the visual responses of neurons in macaque MT

We have previously documented the exquisite motion sensitivity of neurons in extrastriate area MT by studying the relationship between their responses and the direction and strength of visual motion signals delivered to their receptive fields. These results suggested that MT neurons might provide the signals supporting behavioral choice in visual discrimination tasks. To approach this question from another direction, we have now studied the relationship between the discharge of MT neurons and behavioral choice, independently of the effects of visual stimulation. We found that trial-to-trial variability in neuronal signals was correlated with the choices the monkey made. Therefore, when a directionally selective neuron in area MT fires more vigorously, the monkey is more likely to make a decision in favor of the preferred direction of the cell. The magnitude of the relationship was modest, on average, but was highly significant across a sample of 299 cells from four monkeys. The relationship was present for all stimuli (including those without a net motion signal), and for all but the weakest responses. The relationship was reduced or eliminated when the demands of the task were changed so that the directional signal carried by the cell was less informative. The relationship was evident within 50 ms of response onset, and persisted throughout the stimulus presentation. On average, neurons that were more sensitive to weak motion signals had a stronger relationship to behavior than those that were less sensitive. These observations are consistent with the idea that neuronal signals in MT are used by the monkey to determine the direction of stimulus motion. The modest relationship between behavioral choice and the discharge of any one neuron, and the prevalence of the relationship across the population, make it likely that signals from many neurons are pooled to form the data on which behavioral choices are based.

Microstimulation of extrastriate area MST influences performance on a direction discrimination task

1. Evidence from single-unit recordings suggests that neurons in the medial superior temporal visual area (MST) carry directional signals that influence psychophysical judgements of motion direction. We tested this hypothesis by electrically stimulating clusters of directionally selective neurons in MST (the dorsomedial subdivision, primarily) while rhesus monkeys performed a two-alternative, forced-choice direction discrimination task. 2. We performed forty-six microstimulation experiments on two rhesus monkeys. The visual stimuli were dynamic random dot patterns in which the strength of a coherent motion signal could be varied continuously about psychophysical threshold. The monkeys were rewarded for reporting correctly the direction of the coherent motion signal. Microstimulation was applied on half the trials, selected randomly, and the psychophysical data were analyzed to determine whether stimulation of MST neurons influenced the monkeys' choices. 3. Microstimulation influenced the monkeys' performance in a statistically significant manner in 67% of the experiments. In all but one of the significant experiments, microstimulation biased the monkeys' choices toward the direction of motion encoded by MST neurons at the stimulation site. Microstimulation had little effect on the slopes of the psychometric functions, suggesting that the stimulation-induced neural activity resembled a relatively pure motion "signal" rather than "noise." 4. Microstimulation exerted strong effects on the monkeys' behavior only when the visual stimulus was located within the multiunit receptive field measured at the stimulation site. This kind of spatial specificity has also been observed in the middle temporal visual area (MT), but receptive fields in MST are typically much larger than those in MT. Thus MST microstimulation effects are characterized by a coarser spatial scale: stimulation of a single site in MST can influence judgements over a much larger portion of the visual field than equivalent stimulation in MT. 5. Microstimulation was often most effective when visual stimuli were placed within a particularly responsive subregion of the receptive field (a "hot spot"). 6. The results show that MST neurons, like MT neurons, can strongly influence performance on a direction discrimination task. Whether MT and MST influence the decision process in parallel or in series remains to be determined.

On Neural Codes and Perception

Bill Newsome is a professor in the Department of Neurobiology at the Stanford University School of Medicine. He received his B.S. in physics from Stetson University in 1974 and his Ph.D. in biology from Caltech in 1980. Following postdoctoral work at MH, he served on the faculty at SUNY Stony Brook before joining the Stanford faculty in 1988. His research has focused on the neural mechanisms underlying visual perception and visually guided behavior. Bill was a corecipient of the Rank Prize in optoelectronics in 1992, and received the Minerva Foundation's Golden Brain Award in the same year. This fall he received the Spencer Award, granted yearly by the College of Physicians and Surgeons at Columbia University for highly original contributions to research in neurobiology. In addition, he won the Kaiser Award for excellence in preclinical teaching granted annually by the Stanford School of Medicine.

Noise, neural codes and cortical organization

Cortical circuitry must facilitate information transfer in accordance with a neural code. In this article we examine two candidate neural codes: information is represented in the spike rate of neurons, or information is represented in the precise timing of individual spikes. These codes can be distinguished by examining the physiological basis of the highly irregular interspike intervals typically observed in cerebral cortex. Recent advances in our understanding of cortical microcircuitry suggest that the timing of neuronal spikes conveys little, if any, information. The cortex is likely to propagate a noisy rate code through redundant, patchy interconnections.

Correlated neuronal discharge rate and its implications for psychophysical performance

Single neurons can signal subtle changes in the sensory environment with surprising fidelity, often matching the perceptual sensitivity of trained psychophysical observers. This similarity poses an intriguing puzzle: why is psychophysical sensitivity not greater than that of single neurons? Pooling responses across neurons should average out noise in the activity of single cells, leading to substantially improved psychophysical performance. If, however, noise is correlated among these neurons, the beneficial effects of pooling would be diminished. To assess correlation within a pool, the responses of pairs of neurons were recorded simultaneously during repeated stimulus presentations. We report here that the observed covariation in spike count was relatively weak, the correlation coefficient averaging 0.12. A theoretical analysis revealed, however, that weak correlation can limit substantially the signalling capacity of the pool. In addition, theory suggests a relationship between neuronal responses and psychophysical decisions which may prove useful for identifying cell populations underlying specific perceptual capacities.

Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey

We recorded the responses of single neurons in extrastriate area MST while rhesus monkeys discriminated the direction of motion in a set of stochastic visual displays. By varying systematically the strength of a coherent motion signal within the visual display, we were able to measure simultaneously the monkeys' psychophysical thresholds for direction discrimination and the responses of single neurons to the same motion signals. Neuronal thresholds for reliably signaling the direction of motion in the visual display were calculated from the measured responses using a method based in signal detection theory. Neurons in MST were exquisitely sensitive to motion signals in the display, having thresholds for discriminating the direction of coherent motion that were, on average, equal to the psychophysical thresholds of the monkeys. For many MST neurons, the intensity of the response was correlated with the monkey's psychophysical judgements for repeated presentations of a given near-threshold stimulus; the monkey tended to choose the preferred direction of the neuron under study when that neuron responded more strongly to the stimulus. In both of these respects, MST neurons were indistinguishable from neurons in extrastriate area MT, a major source of afferent input to MST. In a second set of experiments, we found that both of these results held true in the face of pronounced manipulations of the visual stimulus. Severe reductions in stimulus size and speed, for example, compromised neuronal and psychophysical sensitivities by similar amounts so that the average neuronal and psychophysical thresholds remained approximately equal. In addition, the trial-to-trial covariation of neuronal response and perceptual decision was unaffected by our stimulus manipulations. Thus, MST neurons carry signals appropriate for supporting psychophysical performance on our task over an impressively wide range of stimulus configurations.

Neural mechanisms for forming a perceptual decision

Cognitive and behavioral responses to environmental stimuli depend on an evaluation of sensory signals within the cerebral cortex. The mechanism by which this occurs in a specific visual task was investigated with a combination of physiological and psychophysical techniques. Rhesus monkeys discriminated among eight possible directions of motion while directional signals were manipulated in visual area MT. One directional signal was generated by a visual stimulus and a second signal was introduced by electrically stimulating neurons that encoded a specific direction of motion. The decisions made by the monkeys in response to the two signals allowed a distinction to be made between two possible mechanisms for interpreting directional signals in MT. The monkeys tended to cast decisions in favor of one or the other signal, indicating that the signals exerted independent effects on performance and that an interactive mechanism such as vector averaging of the two signals was not operative. Thus, the data suggest a mechanism in which monkeys chose the direction encoded by the largest signal in the representation of motion direction, a "winner-take-all" decision process.

Neuronal plasticity that underlies improvement in perceptual performance

The electrophysiological properties of sensory neurons in the adult cortex are not immutable but can change in response to alterations of sensory input caused by manipulation of afferent pathways in the nervous system or by manipulation of the sensory environment. Such plasticity creates great potential for flexible processing of sensory information, but the actual effects of neuronal plasticity on perceptual performance are poorly understood. The link between neuronal plasticity and performance was explored here by recording the responses of directionally selective neurons in the visual cortex while rhesus monkeys practiced a familiar task involving discrimination of motion direction. Each animal experienced a short-term improvement in perceptual sensitivity during daily experiments; sensitivity increased by an average of 19 percent over a few hundred trials. The increase in perceptual sensitivity was accompanied by a short-term improvement in neuronal sensitivity that mirrored the perceptual effect both in magnitude and in time course, which suggests that improved psychophysical performance can result directly from increased neuronal sensitivity within a sensory pathway.

Responses of neurons in macaque MT to stochastic motion signals

Dynamic random-dot stimuli have been widely used to explore central mechanisms of motion processing. We have measured the responses of neurons in area MT of the alert monkey while we varied the strength and direction of the motion signal in such displays. The strength of motion is controlled by the proportion of spatiotemporally correlated dots, which we term the correlation of the stimulus. For many MT cells, responses varied approximately linearly with stimulus correlation. When they occurred, nonlinearities were equally likely to be either positively or negatively accelerated. We also explored the relationship between response magnitude and response variance for these cells and found, in general agreement with other investigators, that this relationship conforms to a power law with an exponent slightly greater than 1. The variance of the cells' discharge is little influenced by the trial-to-trial fluctuations inherent in our stochastic display, and is therefore likely to be of neural origin. Linear responses to these stochastic motion stimuli are predicted by simple, low-level motion models incorporating sensors having relatively broad spatial and temporal frequency tuning.