Temporal precision of neuronal information in a rapid perceptual judgment

Dynamic divisive normalization circuits explain and predict change detection in monkey area MT

Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.

Correlated structure of neuronal firing in macaque visual cortex limits information for binocular depth discrimination

Variability in cortical neural activity potentially limits sensory discriminations. Theoretical work shows that information required to discriminate two similar stimuli is limited by the correlation structure of cortical variability. We investigated these information-limiting correlations by recording simultaneously from visual cortical areas primary visual cortex (V1) and extrastriate area V4 in macaque monkeys performing a binocular, stereo depth discrimination task. Within both areas, noise correlations on a rapid temporal scale (20–30 ms) were stronger for neuron pairs with similar selectivity for binocular depth, meaning that these correlations potentially limit information for making the discrimination. Between-area correlations (V1 to V4) were different, being weaker for neuron pairs with similar tuning and having a slower temporal scale (100+ ms). Fluctuations in these information-limiting correlations just prior to the detection event were associated with changes in behavioral accuracy. Although these correlations limit the recovery of information about sensory targets, their impact may be curtailed by integrative processing of signals across multiple brain areas.

NEW & NOTEWORTHY Correlated noise reduces the stimulus information in visual cortical neurons during experimental performance of binocular depth discriminations. The temporal scale of these correlations is important. Rapid (20–30 ms) correlations reduce information within and between areas V1 and V4, whereas slow (>100 ms) correlations between areas do not. Separate cortical areas appear to act together to maintain signal fidelity. Rapid correlations reduce the neuronal signal difference between stimuli and adversely affect perceptual discrimination.

Attention amplifies neural representations of changes in sensory input at the expense of perceptual accuracy

Attention enhances the neural representations of behaviorally relevant stimuli, typically by a push-pull increase of the neuronal response gain to attended vs. unattended stimuli. This selectively improves perception and consequently behavioral performance. However, to enhance the detectability of stimulus changes, attention might also distort neural representations, compromising accurate stimulus representation. We test this hypothesis by recording neural responses in the visual cortex of rhesus monkeys during a motion direction change detection task. We find that attention indeed amplifies the neural representation of direction changes, beyond a similar effect of adaptation. We further show that humans overestimate such direction changes, providing a perceptual correlate of our neurophysiological observations. Our results demonstrate that attention distorts the neural representations of abrupt sensory changes and consequently perceptual accuracy. This likely represents an evolutionary adaptive mechanism that allows sensory systems to flexibly forgo accurate representation of stimulus features to improve the encoding of stimulus change.

Micropools of reliable area MT neurons explain rapid motion detection

Many models of perceptually based decisions postulate that actions are initiated when accumulated sensory signals reach a threshold level of activity. These models have received considerable neurophysiological support from recordings of individual neurons while animals are engaged in motion discrimination tasks. These experiments have found that the activity of neurons in a particular visual area strongly associated with motion processing (MT), when pooled over hundreds of milliseconds, is sufficient to explain behavioral timing and performance. However, this level of pooling may be problematic for urgent perceptual decisions in which rapid detection dictates temporally precise integration. In this paper, we explore the physiological basis of one such task in which macaques detected brief (~70 ms) transients of coherent motion within ~240 ms. We find that a simple linear summation model based on realistic stimulus responses of as few as 40 correlated neurons can predict the reliability and timing of rapid motion detection. The model naturally reproduces a distinctive physiological relationship observed in rapid detection tasks in which the individual neurons with the most reliable stimulus responses are also the most predictive of impending behavioral choices. Remarkably, we observed this relationship across our simulated neuronal populations even when all neurons within the pool were weighted equally with respect to readout. These results demonstrate that small numbers of reliable sensory neurons can dominate perceptual judgments without any explicit reliability based weighting and are sufficient to explain the accuracy, latency, and temporal precision of rapid detection. NEW & NOTEWORTHY Computational and psychophysical models suggest that performance in many perceptual tasks may be based on the preferential sampling of reliable neurons. Recent studies of MT neurons during rapid motion detection, in which only those neurons with the most reliable sensory responses were strongly predictive of the animals' decisions, seemingly support this notion. Here we show that a simple threshold model without explicit reliability biases can explain both the behavioral accuracy and precision of these detections and the distribution of sensory- and choice-related signals across neurons.

Spatially tuned normalization explains attention modulation variance within neurons

Spatial attention improves perception of attended parts of a scene, a behavioral enhancement accompanied by modulations of neuronal firing rates. These modulations vary in size across neurons in the same brain area. Models of normalization explain much of this variance in attention modulation with differences in tuned normalization across neurons (Lee J, Maunsell JHR. PLoS One 4: e4651, 2009; Ni AM, Ray S, Maunsell JHR. Neuron 73: 803-813, 2012). However, recent studies suggest that normalization tuning varies with spatial location both across and within neurons (Ruff DA, Alberts JJ, Cohen MR. J Neurophysiol 116: 1375-1386, 2016; Verhoef BE, Maunsell JHR. eLife 5: e17256, 2016). Here we show directly that attention modulation and normalization tuning do in fact covary within individual neurons, in addition to across neurons as previously demonstrated. We recorded the activity of isolated neurons in the middle temporal area of two rhesus monkeys as they performed a change-detection task that controlled the focus of spatial attention. Using the same two drifting Gabor stimuli and the same two receptive field locations for each neuron, we found that switching which stimulus was presented at which location affected both attention modulation and normalization in a correlated way within neurons. We present an equal-maximum-suppression spatially tuned normalization model that explains this covariance both across and within neurons: each stimulus generates equally strong suppression of its own excitatory drive, but its suppression of distant stimuli is typically less. This new model specifies how the tuned normalization associated with each stimulus location varies across space both within and across neurons, changing our understanding of the normalization mechanism and how attention modulations depend on this mechanism.NEW & NOTEWORTHY Tuned normalization studies have demonstrated that the variance in attention modulation size seen across neurons from the same cortical area can be largely explained by between-neuron differences in normalization strength. Here we demonstrate that attention modulation size varies within neurons as well and that this variance is largely explained by within-neuron differences in normalization strength. We provide a new spatially tuned normalization model that explains this broad range of observed normalization and attention effects.

A null model for cortical representations with grandmothers galore

There has been extensive discussion in the literature about the extent to which cortical representations can be described as localist or distributed. Here we discuss a simple null model that encompasses a family of related architectures describing the transformation of signals throughout the parts of the visual system involved in object recognition. This family of models constitutes a rigorous first approximation to explain the neurophysiological properties of ventral visual cortex. This null model contains both distributed and local representations throughout the entire hierarchy of computations and the responses of individual units are meaningful and interpretable when encoding is adequately defined for each computational stage.

The random dot tachistogram: a novel task that elucidates the functional architecture of decision

Reaction times are long and variable, almost certainly because they result from a process that accumulates noisy decision signals over time, rising to a threshold. But the origin of the variability is still disputed: is it because the incoming sensory signals are themselves noisy? Or does it arise within the brain? Here we use a stimulus – the random dot tachistogram – which demands spatial integration of information presented essentially instantaneously; with it, we demonstrate three things. First, that the latency distributions still show the variability characteristic of LATER, implying that there must be two integrators in series. Secondly, that since this variability persists despite removal of all temporal noise from the stimulus, or even trial-to-trial spatial variation, it must come from within the nervous system. Finally, that the average rate of rise of the decision signal depends linearly on how many dots move in a given direction. Taken together, this suggests a rather simple, two-stage model of the overall process. The first, detection, stage performs local temporal integration of stimuli; the local, binary, outcomes are linearly summed and integrated by LATER units in the second stage, that perform the final global decision by a process of racing competition.

Neuronal Mechanisms of Visual Attention

Advances on several fronts have refined our understanding of the neuronal mechanisms of attention. This review focuses on recent progress in understanding visual attention through single-neuron recordings made in behaving subjects. Simultaneous recordings from populations of individual cells have shown that attention is associated with changes in the correlated firing of neurons that can enhance the quality of sensory representations. Other work has shown that sensory normalization mechanisms are important for explaining many aspects of how visual representations change with attention, and these mechanisms must be taken into account when evaluating attention-related neuronal modulations. Studies comparing different brain structures suggest that attention is composed of several cognitive processes, which might be controlled by different brain regions. Collectively, these and other recent findings provide a clearer picture of how representations in the visual system change when attention shifts from one target to another.

Insights into decision making using choice probability

A long-standing question in systems neuroscience is how the activity of single neurons gives rise to our perceptions and actions. Critical insights into this question occurred in the last part of the 20th century when scientists began linking modulations of neuronal activity directly to perceptual behavior. A significant conceptual advance was the application of signal detection theory to both neuronal activity and behavior, providing a quantitative assessment of the relationship between brain and behavior. One metric that emerged from these efforts was choice probability (CP), which provides information about how well an ideal observer can predict the choice an animal makes from a neuron's discharge rate distribution. In this review, we describe where CP has been studied, locational trends in the values found, and why CP values are typically so low. We discuss its dependence on correlated activity among neurons of a population, assess whether it arises from feedforward or feedback mechanisms, and investigate what CP tells us about how many neurons are required for a decision and how they are pooled to do so.

How Can Single Sensory Neurons Predict Behavior?

Single sensory neurons can be surprisingly predictive of behavior in discrimination tasks. We propose this is possible because sensory information extracted from neural populations is severely restricted, either by near-optimal decoding of a population with information-limiting correlations or by suboptimal decoding that is blind to correlations. These have different consequences for choice correlations, the correlations between neural responses and behavioral choices. In the vestibular and cerebellar nuclei and the dorsal medial superior temporal area, we found that choice correlations during heading discrimination are consistent with near-optimal decoding of neuronal responses corrupted by information-limiting correlations. In the ventral intraparietal area, the choice correlations are also consistent with the presence of information-limiting correlations, but this area does not appear to influence behavior, although the choice correlations are particularly large. These findings demonstrate how choice correlations can be used to assess the efficiency of the downstream readout and detect the presence of information-limiting correlations.

Perceptual decision related activity in the lateral geniculate nucleus

Fundamental to neuroscience is the understanding of how the language of neurons relates to behavior. In the lateral geniculate nucleus (LGN), cells show distinct properties such as selectivity for particular wavelengths, increments or decrements in contrast, or preference for fine detail versus rapid motion. No studies, however, have measured how LGN cells respond when an animal is challenged to make a perceptual decision using information within the receptive fields of those LGN cells. In this study we measured neural activity in the macaque LGN during a two-alternative, forced-choice (2AFC) contrast detection task or during a passive fixation task and found that a small proportion (13.5%) of single LGN parvocellular (P) and magnocellular (M) neurons matched the psychophysical performance of the monkey. The majority of LGN neurons measured in both tasks were not as sensitive as the monkey. The covariation between neural response and behavior (quantified as choice probability) was significantly above chance during active detection, even when there was no external stimulus. Interneuronal correlations and task-related gain modulations were negligible under the same condition. A bottom-up pooling model that used sensory neural responses to compute perceptual choices in the absence of interneuronal correlations could fully explain these results at the level of the LGN, supporting the hypothesis that the perceptual decision pool consists of multiple sensory neurons and that response fluctuations in these neurons can influence perception.

Population coding in area V4 during rapid shape detections

While previous studies have suggested that neuronal correlations are common in visual cortex over a range of timescales, the effect of correlations on rapid visually based decisions has received little attention. We trained Macaca mulatta to saccade to a peripherally presented shape embedded in dynamic noise as soon as the shape appeared. While the monkeys performed the task, we recorded from neuronal populations (5-29 cells) using a microelectrode array implanted in area V4, a visual area thought to be involved in form perception. While modest correlations were present between cells during visual stimulation, their magnitude did not change significantly subsequent to the appearance of a shape. We quantified the reliability and temporal precision with which neuronal populations signaled the appearance of the shape and predicted the animals' choices using mutual information analyses. To study the impact of correlations, we shuffled the activity from each cell across observations while retaining stimulus-dependent modulations in firing rate. We found that removing correlations by shuffling across trials minimally affected the reliability or timing with which pairs, or larger groups of cells, signaled the presence of a shape. To assess the downstream impact of correlations, we also studied how shuffling affected the ability of V4 populations to predict behavioral choices. Surprisingly, shuffling created a modest increase in the accuracy of such predictions, suggesting that the reliability of downstream neurons is slightly compromised by activity correlations. Our findings are consistent with neuronal correlations having a minimal effect on the reliability and timing of rapid perceptual decisions.

Transient activity in monkey area MT represents speed changes and is correlated with human behavioral performance

Neurons in the middle temporal area (MT) respond to motion onsets and speed changes with a transient-sustained firing pattern. The latency of the transient response has recently been shown to correlate with reaction time in a speed change detection task, but it is not known how the sign, the amplitude, and the latency of this response depend on the sign and the magnitude of a speed change, and whether these transients can be decoded to explain speed change detection behavior. To investigate this issue, we measured the neuronal representation of a wide range of positive and negative speed changes in area MT of fixating macaques and obtained three major findings. First, speed change transients not only reflect a neuron's absolute speed tuning but are shaped by an additional gain that scales the tuned response according to the magnitude of a relative speed change. Second, by means of a threshold model positive and negative population transients of a moderate number of MT neurons explain detection of both positive and negative speed changes, respectively, at a level comparable to human detection rates under identical visual stimulation. Third, like reaction times in a psychophysical model of velocity detection, speed change response latencies follow a power-law function of the absolute difference of a speed change. Both this neuronal representation and its close correlation with behavioral measures of speed change detection suggest that neuronal transients in area MT facilitate the detection of rapid changes in visual input.

Rapid shape detection signals in area V4

Vision in foveate animals is an active process that requires rapid and constant decision-making. For example, when a new object appears in the visual field, we can quickly decide to inspect it by directing our eyes to the object's location. We studied the contribution of primate area V4 to these types of rapid foveation decisions. Animals performed a reaction time task that required them to report when any shape appeared within a peripherally-located noisy stimulus by making a saccade to the stimulus location. We found that about half of the randomly sampled V4 neurons not only rapidly and precisely represented the appearance of this shape, but they were also predictive of the animal's saccades. A neuron's ability to predict the animal's saccades was not related to the specificity with which the cell represented a single type of shape but rather to its ability to signal whether any shape was present. This relationship between sensory sensitivity and behavioral predictiveness was not due to global effects such as alertness, as it was equally likely to be observed for cells with increases and decreases in firing rate. Careful analysis of the timescales of reliability in these neurons implies that they reflect both feedforward and feedback shape detecting processes. In approximately 7% of our recorded sample, individual neurons were able to predict both the delay and precision of the animal's shape detection performance. This suggests that a subset of V4 neurons may have been directly and causally contributing to task performance and that area V4 likely plays a critical role in guiding rapid, form-based foveation decisions.

ROC-based estimates of neural-behavioral covariations using matched filters

Correlations between responses in visual cortex and perceptual performance help draw a functional link between neural activity and visually guided behavior. These correlations are commonly derived with ROC-based neural-behavioral covariances (referred to as choice or detect probability) using boxcar analysis windows. Although boxcar windows capture the covariation between neural activity and behavior during steady-state stimulus presentations, they are not optimized to capture these correlations during short time-varying visual inputs. In this study, we implemented a matched-filter technique, combined with cross-validation, to improve the estimation of ROC-based neural-behavioral covariance under short and dynamic stimulus conditions. We show that this approach maximizes the area under the ROC curve and converges to the true neural-behavioral covariance using a Poisson spiking model. We also demonstrate that the matched filter, combined with cross-validation, reveals the dynamics of the neural-behavioral covariations of individual MT neurons during the detection of a brief motion stimulus.

Inattention blindness to motion in middle temporal area

Subjects naturally form and use expectations to solve familiar tasks, but the accuracy of these expectations and the neuronal mechanisms by which these expectations enhance behavior are unclear. We trained animals (Macaca mulatta) in a challenging perceptual task in which the likelihood of a very brief pulse of motion was consistently modulated over time and space. Pulse likelihood had dramatic effects on behavior: unexpected pulses were nearly invisible to the animals. To examine the neuronal basis of such inattention blindness, we recorded from single neurons in the middle temporal (MT) area, an area related to motion perception. Fluctuations in how reliably MT neurons both signaled stimulus events and predicted behavioral choices were highly correlated with changes in performance over the course of individual trials. A simple neuronal pooling model reveals that the dramatic behavioral effects of attention in this task can be completely explained by changes in the reliability of a small number of MT neurons.

Decision-related activity in sensory neurons: correlations among neurons and with behavior

Neurons in early sensory cortex show weak but systematic correlations with perceptual decisions when trained animals perform at psychophysical threshold. These correlations are observed across repeated presentations of identical stimuli and cannot be explained by variation in external factors. The relationship between the activity of individual sensory neurons and the animal's behavioral choice means that even neurons in early sensory cortex carry information about an upcoming decision. This relationship, termed choice probability, may reflect the effect of fluctuations in neuronal firing rate on the animal's decision, but it can also reflect modulation of sensory responses by cognitive factors, or network properties such as variability that is shared among populations of neurons. Here, we review recent work clarifying the relationship among fluctuations in the responses of individual neurons, correlated variability, and behavior in a variety of tasks and cortical areas. We also discuss the possibility that choice probability may in part reflect the influence of cognitive factors on sensory neurons and explore the situations in which choice probability can be used to make inferences about the role of particular sensory neurons in the decision-making process.

Ability of primary auditory cortical neurons to detect amplitude modulation with rate and temporal codes: neurometric analysis

Amplitude modulation (AM) is a common feature of natural sounds, and its detection is biologically important. Even though most sounds are not fully modulated, the majority of physiological studies have focused on fully modulated (100% modulation depth) sounds. We presented AM noise at a range of modulation depths to awake macaque monkeys while recording from neurons in primary auditory cortex (A1). The ability of neurons to detect partial AM with rate and temporal codes was assessed with signal detection methods. On average, single-cell synchrony was as or more sensitive than spike count in modulation detection. Cells are less sensitive to modulation depth if tested away from their best modulation frequency, particularly for temporal measures. Mean neural modulation detection thresholds in A1 are not as sensitive as behavioral thresholds, but with phase locking the most sensitive neurons are more sensitive, suggesting that for temporal measures the lower-envelope principle cannot account for thresholds. Three methods of preanalysis pooling of spike trains (multiunit, similar to convergence from a cortical column; within cell, similar to convergence of cells with matched response properties; across cell, similar to indiscriminate convergence of cells) all result in an increase in neural sensitivity to modulation depth for both temporal and rate codes. For the across-cell method, pooling of a few dozen cells can result in detection thresholds that approximate those of the behaving animal. With synchrony measures, indiscriminate pooling results in sensitive detection of modulation frequencies between 20 and 60 Hz, suggesting that differences in AM response phase are minor in A1.

The functional link between area MT neural fluctuations and detection of a brief motion stimulus

Fluctuations of neural firing rates in visual cortex are known to be correlated with variations in perceptual performance. It is important to know whether these fluctuations are functionally linked to perception in a causal manner or instead reflect non-causal processes that arise after the perceptual decision is made. We recorded from middle temporal (MT) neurons from monkey subjects while they detected the random occurrence of a brief 50 ms motion pulse that occurred in either of two (or simultaneously in both) random dot patches located in the same hemisphere. The receptive field parameters of the motion pulse were matched to that preferred by each MT neuron under study. This task contained uncertainty in both space and time because, on any given trial, the subjects did not know which patch would contain the motion pulse or when the motion pulse would occur. Covariations between MT activity and behavior began just before the motion pulse onset and peaked at the maximum neural response. These neural-behavioral covariations were strongest when only one patch contained the motion pulse and were still weakly present when a patch did not contain a motion pulse. A feedforward temporal integration model with two independent detector channels captured both the detection performance and evolution of the neural-behavior covariations over time and stimulus condition. The results suggest that, when detecting a brief visual stimulus, there is a causal relationship between fluctuations in neural activity and variations in behavior across trials.

Neurally constrained modeling of perceptual decision making

Stochastic accumulator models account for response time in perceptual decision-making tasks by assuming that perceptual evidence accumulates to a threshold. The present investigation mapped the firing rate of frontal eye field (FEF) visual neurons onto perceptual evidence and the firing rate of FEF movement neurons onto evidence accumulation to test alternative models of how evidence is combined in the accumulation process. The models were evaluated on their ability to predict both response time distributions and movement neuron activity observed in monkeys performing a visual search task. Models that assume gating of perceptual evidence to the accumulating units provide the best account of both behavioral and neural data. These results identify discrete stages of processing with anatomically distinct neural populations and rule out several alternative architectures. The results also illustrate the use of neurophysiological data as a model selection tool and establish a novel framework to bridge computational and neural levels of explanation.

Timescales of sensory- and decision-related activity in the middle temporal and medial superior temporal areas

The contribution of sensory neurons to perceptual decisions about external stimulus events has received much attention, but it is less clear how sensory responses are integrated over time to produce decisions that are both rapid and reliable. To address this issue, we recorded from middle temporal area and medial superior temporal area neurons in rhesus macaques performing a task requiring the detection and discrimination of unpredictable speed changes. We examined how neuronal activity encoded the sign of the speed change and predicted the animals' behavioral judgments and reaction times, with a focus on the timescales over which neuronal activity is informative. False detection trials, on which animals reported a speed change even though none had occurred, were grouped according to the animals' discrimination judgment. By comparing the neuronal responses between the two groups of false detection trials, we were able to predict the animals' choices from the sensory activity of single neurons at levels significantly better than chance. These choice probability measurements were strongest using spike counts in an 80 ms window ending 150 ms before a choice saccade began, but significant choice probabilities were observed in windows as short as 10 ms. While the maximum deviation in spiking rate following a speed change is evident in the transient response, averaging neuronal activity in longer time windows can be more informative about both the stimulus and the animals' behavioral judgments. Thus the timescales found in this study represent a trade-off between producing rapid reactions and overcoming the noise inherent in short time windows.

Trial-to-trial variability of spike response of V1 and saccadic response time

Single neurons in the primary visual cortex (V1) show variability in spike activity in response to an identical visual stimulus. In the current study, we examined the behavioral significance of the variability in spike activity of V1 neurons for visually guided saccades. We recorded single-cell activity from V1 of monkeys trained to detect and make saccades toward visual targets of varying contrast and analyzed trial-to-trial covariation between the onset time or firing rate of neural response and saccadic response time (RT). Neural latency (NL, the time of the first spike of neural response) was correlated with RT, whereas firing rate (FR) was not. When FR was computed with respect to target onset ignoring NL, a "false" correlation between FR and RT emerged. Multiple regression and partial correlation analyses on NL and FR for predictability of RT variability, as well as a simulation with artificial Poisson spike trains, supported the conclusion that the correlation between FR with respect to target onset and RT was mediated by a correlation between NL and RT, emphasizing the role of trial-to-trial variability of NL for extracting RT-related signals. We attempted to examine laminar differences in RT-related activity. Neurons recorded in the superficial layers tended to show a higher sensitivity to stimulus contrast and a lower correlation with RT compared with those in the lower layers, suggesting a sensory-to-motor transformation within V1 that follows the order of known anatomical connections. These results demonstrate that the trial-to-trial variability of neural response in V1 propagates to the stage of saccade execution, resulting in trial-to-trial variability of RT of a visually guided saccade.

Attention directed by expectations enhances receptive fields in cortical area MT

Expectations, especially those formed on the basis of extensive training, can substantially enhance visual performance. However, it is not clear that the physiological mechanisms underlying this enhancement are identical to those examined by experiments in which attention is directed by explicit instructions rather than strong expectations. To study the changes in visual representations associated with strong expectations, we trained animals to detect a brief motion pulse that was embedded in noise. Because the nature of the pulse and the statistics of its appearance were well known to the animals, they formed strong expectations which determined their behavioral performance. We used white-noise methods to infer the receptive field structure of single neurons in area MT while they were performing this task. Incorporating non-linearities, we compared receptive fields during periods of time when the animals were expecting the motion pulse with periods of time when they were not. We found receptive field changes consistent with an increased reliability in signaling pulse occurrence. Moreover, these changes were not consistent with a simple gain modulation. The results suggest that strong expectations can create very specific changes in the visual representations at a cellular level to enhance performance.