Effect of spatial attention on the responses of area MT neurons

Inattentional aftereffects: The role of attention on the strength of the motion aftereffect

The way that attention affects the processing of visual information is one of the most intriguing fields in the study of visual perception. One way to examine this interaction is by studying the way perceptual aftereffects are modulated by attention. In the present study, we have manipulated attention during adaptation to translational motion generated by coherently moving random dots, in order to investigate the effect of the distraction of attention on the strength of the peripheral dynamic motion aftereffect (MAE). A foveal rapid serial visual presentation task (RSVP) of varying difficulty was introduced during the adaptation period while the adaptation and test stimuli were presented peripherally. Furthermore, to examine the interaction between the physical characteristics of the stimulus and attention, we have manipulated the motion coherence level of the adaptation stimuli. Our results suggested that the removal of attention through an irrelevant task modulated the MAE's magnitude moderately and that such an effect depends on the stimulus strength. We also showed that the MAE still persists with subthreshold and unattended stimuli, suggesting that perhaps attention is not required for the complete development of the MAE.

Phase of neural oscillations as a reference frame for attention-based routing in visual cortex

Selective attention allows the brain to efficiently process the image projected onto the retina, selectively focusing neural processing resources on behaviorally relevant visual information. While previous studies have documented the crucial role of the action potential rate of single neurons in relaying such information, little is known about how the activity of single neurons relative to their neighboring network contributes to the efficient representation of attended stimuli and transmission of this information to downstream areas. Here, we show in the dorsal visual pathway of monkeys (medial superior temporal area) that neurons fire spikes preferentially at a specific phase of the ongoing population beta (∼20 Hz) oscillations of the surrounding local network. This preferred spiking phase shifts towards a later phase when monkeys selectively attend towards (rather than away from) the receptive field of the neuron. This shift of the locking phase is positively correlated with the speed at which animals report a visual change. Furthermore, our computational modeling suggests that neural networks can manipulate the preferred phase of coupling by imposing differential synaptic delays on postsynaptic potentials. This distinction between the locking phase of neurons activated by the spatially attended stimulus vs. that of neurons activated by the unattended stimulus, may enable the neural system to discriminate relevant from irrelevant sensory inputs and consequently filter out distracting stimuli information by aligning the spikes which convey relevant/irrelevant information to distinct phases linked to periods of better/worse perceptual sensitivity for higher cortices. This strategy may be used to reserve the narrow windows of highest perceptual efficacy to the processing of the most behaviorally relevant information, ensuring highly efficient responses to attended sensory events.

Pre-saccadic Neural Enhancements in Marmoset Area MT

Each time we make an eye movement, attention moves before the eyes, resulting in a perceptual enhancement at the target. Recent psychophysical studies suggest that this pre-saccadic attention enhances the visual features at the saccade target, whereas covert attention causes only spatially selective enhancements. While previous nonhuman primate studies have found that pre-saccadic attention does enhance neural responses spatially, no studies have tested whether changes in neural tuning reflect an automatic feature enhancement. Here we examined pre-saccadic attention using a saccade foraging task developed for marmoset monkeys (one male and one female). We recorded from neurons in the middle temporal area with peripheral receptive fields that contained a motion stimulus, which would either be the target of a saccade or a distracter as a saccade was made to another location. We established that marmosets, like macaques, show enhanced pre-saccadic neural responses for saccades toward the receptive field, including increases in firing rate and motion information. We then examined if the specific changes in neural tuning might support feature enhancements for the target. Neurons exhibited diverse changes in tuning but predominantly showed additive and multiplicative increases that were uniformly applied across motion directions. These findings confirm that marmoset monkeys, like macaques, exhibit pre-saccadic neural enhancements during saccade foraging tasks with minimal training requirements. However, at the level of individual neurons, the lack of feature-tuned enhancements is similar to neural effects reported during covert spatial attention.

Sensory and Choice Responses in MT Distinct from Motion Encoding

The macaque middle temporal (MT) area is well known for its visual motion selectivity and relevance to motion perception, but the possibility of it also reflecting higher-level cognitive functions has largely been ignored. We tested for effects of task performance distinct from sensory encoding by manipulating subjects' temporal evidence-weighting strategy during a direction discrimination task while performing electrophysiological recordings from groups of MT neurons in rhesus macaques (one male, one female). This revealed multiple components of MT responses that were, surprisingly, not interpretable as behaviorally relevant modulations of motion encoding, or as bottom-up consequences of the readout of motion direction from MT. The time-varying motion-driven responses of MT were strongly affected by our strategic manipulation-but with time courses opposite the subjects' temporal weighting strategies. Furthermore, large choice-correlated signals were represented in population activity distinct from its motion responses, with multiple phases that lagged psychophysical readout and even continued after the stimulus (but which preceded motor responses). In summary, a novel experimental manipulation of strategy allowed us to control the time course of readout to challenge the correlation between sensory responses and choices, and population-level analyses of simultaneously recorded ensembles allowed us to identify strong signals that were so distinct from direction encoding that conventional, single-neuron-centric analyses could not have revealed or properly characterized them. Together, these approaches revealed multiple cognitive contributions to MT responses that are task related but not functionally relevant to encoding or decoding of motion for psychophysical direction discrimination, providing a new perspective on the assumed status of MT as a simple sensory area.SIGNIFICANCE STATEMENT This study extends understanding of the middle temporal (MT) area beyond its representation of visual motion. Combining multineuron recordings, population-level analyses, and controlled manipulation of task strategy, we exposed signals that depended on changes in temporal weighting strategy, but did not manifest as feedforward effects on behavior. This was demonstrated by (1) an inverse relationship between temporal dynamics of behavioral readout and sensory encoding, (2) a choice-correlated signal that always lagged the stimulus time points most correlated with decisions, and (3) a distinct choice-correlated signal after the stimulus. These findings invite re-evaluation of MT for functions outside of its established sensory role and highlight the power of experimenter-controlled changes in temporal strategy, coupled with recording and analysis approaches that transcend the single-neuron perspective.

Attention improves information flow between neuronal populations without changing the communication subspace

Visual attention allows observers to change the influence of different parts of a visual scene on their behavior, suggesting that information can be flexibly shared between visual cortex and neurons involved in decision making. We investigated the neural substrate of flexible information routing by analyzing the activity of populations of visual neurons in the medial temporal area (MT) and oculo-motor neurons in the superior colliculus (SC) while rhesus monkeys switched spatial attention. We demonstrated that attention increases the efficacy of visuomotor communication: trial-to-trial variability in SC population activity could be better predicted by the activity of the MT population (and vice versa) when attention was directed toward their joint receptive fields. Surprisingly, this improvement in prediction was not explained by changes in the dimensionality of the shared subspace or in the magnitude of local or shared pairwise noise correlations. These results lay a foundation for future theoretical and experimental studies into how visual attention can flexibly change information flow between sensory and decision neurons.

Attention amplifies neural representations of changes in sensory input at the expense of perceptual accuracy

Attention enhances the neural representations of behaviorally relevant stimuli, typically by a push-pull increase of the neuronal response gain to attended vs. unattended stimuli. This selectively improves perception and consequently behavioral performance. However, to enhance the detectability of stimulus changes, attention might also distort neural representations, compromising accurate stimulus representation. We test this hypothesis by recording neural responses in the visual cortex of rhesus monkeys during a motion direction change detection task. We find that attention indeed amplifies the neural representation of direction changes, beyond a similar effect of adaptation. We further show that humans overestimate such direction changes, providing a perceptual correlate of our neurophysiological observations. Our results demonstrate that attention distorts the neural representations of abrupt sensory changes and consequently perceptual accuracy. This likely represents an evolutionary adaptive mechanism that allows sensory systems to flexibly forgo accurate representation of stimulus features to improve the encoding of stimulus change.

Strategic deployment of feature-based attentional gain in primate visual cortex

Attending to visual stimuli enhances the gain of those neurons in primate visual cortex that preferentially respond to the matching locations and features (on-target gain). Although this is well suited to enhance the neuronal representation of attended stimuli, it is nonoptimal under difficult discrimination conditions, as in the presence of similar distractors. In such cases, directing attention to neighboring neuronal populations (off-target gain) has been shown to be the most efficient strategy, but although such a strategic deployment of attention has been shown behaviorally, its underlying neural mechanisms are unknown. Here, we investigated how attention affects the population responses of neurons in the middle temporal (MT) visual area of rhesus monkeys to bidirectional movement inside the neurons' receptive field (RF). The monkeys were trained to focus their attention onto the fixation spot or to detect a direction or speed change in one of the motion directions (the "target"), ignoring the distractor motion. Population activity profiles were determined by systematically varying the patterns' directions while maintaining a constant angle between them. As expected, the response profiles show a peak for each of the 2 motion directions. Switching spatial attention from the fixation spot into the RF enhanced the peak representing the attended stimulus and suppressed the distractor representation. Importantly, the population data show a direction-dependent attentional modulation that does not peak at the target feature but rather along the slopes of the activity profile representing the target direction. Our results show that attentional gains are strategically deployed to optimize the discriminability of target stimuli, in line with an optimal gain mechanism proposed by Navalpakkam and Itti.

Distracter suppression dominates attentional modulation of responses to multiple stimuli inside the receptive fields of middle temporal neurons

Single-cell studies in macaques have shown that attending to one of two stimuli, positioned inside a visual neuron's receptive field (RF), modulates the neuron's response to reflect the features of the attended stimulus. Such a modulation has been described as a 'push-pull' effect relative to a reference response: a neuron's response increases when attention is directed to a preferred stimulus, and decreases when attention is directed to a non-preferred stimulus. It has been further suggested that the response increase when attending to a preferred stimulus is the predominant effect. Here, we show that the observed attentional modulation depends on the reference response. We recorded neuronal responses in motion processing area middle temporal (MT) of macaques to two moving random dot patterns positioned inside neurons' RF. One pattern always moved in the neuron's antipreferred direction (null pattern), while the other moved in one of 12 directions (tuning pattern). At the beginning of a trial, a cue indicated the location and direction of the target. The animal was required to release a lever when a change in the target direction occurred, and to ignore changes in the distracter. Relative to neurons' initial responses to the dual stimuli (when attention was less likely to modulate responses), attending to the tuning pattern did not significantly modulate responses over time. However, attending to the null pattern progressively decreased responses over time. These results were quantitatively described by filter and input gain models, characterising a predominant response suppression relative to a reference response, rather than response enhancement.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Sequential sensory and decision processing in posterior parietal cortex

Decisions about the behavioral significance of sensory stimuli often require comparing sensory inference of what we are looking at to internal models of what we are looking for. Here, we test how neuronal selectivity for visual features is transformed into decision-related signals in posterior parietal cortex (area LIP). Monkeys performed a visual matching task that required them to detect target stimuli composed of conjunctions of color and motion-direction. Neuronal recordings from area LIP revealed two main findings. First, the sequential processing of visual features and the selection of target-stimuli suggest that LIP is involved in transforming sensory information into decision-related signals. Second, the patterns of color and motion selectivity and their impact on decision-related encoding suggest that LIP plays a role in detecting target stimuli by comparing bottom-up sensory inputs (what the monkeys were looking at) and top-down cognitive encoding inputs (what the monkeys were looking for).

DOI:http://dx.doi.org/10.7554/eLife.23743.001

Neurons in Primate Visual Cortex Alternate between Responses to Multiple Stimuli in Their Receptive Field

A fundamental question concerning representation of the visual world in our brain is how a cortical cell responds when presented with more than a single stimulus. We find supportive evidence that most cells presented with a pair of stimuli respond predominantly to one stimulus at a time, rather than a weighted average response. Traditionally, the firing rate is assumed to be a weighted average of the firing rates to the individual stimuli (response-averaging model) (Bundesen et al., 2005). Here, we also evaluate a probability-mixing model (Bundesen et al., 2005), where neurons temporally multiplex the responses to the individual stimuli. This provides a mechanism by which the representational identity of multiple stimuli in complex visual scenes can be maintained despite the large receptive fields in higher extrastriate visual cortex in primates. We compare the two models through analysis of data from single cells in the middle temporal visual area (MT) of rhesus monkeys when presented with two separate stimuli inside their receptive field with attention directed to one of the two stimuli or outside the receptive field. The spike trains were modeled by stochastic point processes, including memory effects of past spikes and attentional effects, and statistical model selection between the two models was performed by information theoretic measures as well as the predictive accuracy of the models. As an auxiliary measure, we also tested for uni- or multimodality in interspike interval distributions, and performed a correlation analysis of simultaneously recorded pairs of neurons, to evaluate population behavior.

Steering Transforms the Cortical Representation of Self-Movement from Direction to Destination

Steering demands rapid responses to heading deviations and uses optic flow to redirect self-movement toward the intended destination. We trained monkeys in a naturalistic steering paradigm and recorded dorsal medial superior temporal area (MSTd) cortical neuronal responses to the visual motion and spatial location cues in optic flow. We found that neuronal responses to the initial heading direction are dominated by the optic flow's global radial pattern cue. Responses to subsequently imposed heading deviations are dominated by the local direction of motion cue. Finally, as the monkey steers its heading back to the goal location, responses are dominated by the spatial location cue, the screen location of the flow field's center of motion. We conclude that MSTd responses are not rigidly linked to specific stimuli, but rather are transformed by the task relevance of cues that guide performance in learned, naturalistic behaviors.

SIGNIFICANCE STATEMENT

Unplanned heading changes trigger lifesaving steering back to a goal. Conventionally, such behaviors are thought of as cortical sensory-motor reflex arcs. We find that a more reciprocal process underlies such cycles of perception and action, rapidly transforming visual processing to suit each stage of the task. When monkeys monitor their simulated self-movement, dorsal medial superior temporal area (MSTd) neurons represent their current heading direction. When monkeys steer to recover from an unplanned change in heading direction, MSTd shifts toward representing the goal location. We hypothesize that this transformation reflects the reweighting of bottom-up visual motion signals and top-down spatial location signals, reshaping MSTd's response properties through task-dependent interactions with adjacent cortical areas.

An Eye in the Palm of Your Hand: Alterations in Visual Processing Near the Hand, a Mini-Review

Feedback within the oculomotor system improves visual processing at eye movement end points, also termed a visual grasp. We do not just view the world around us however, we also reach out and grab things with our hands. A growing body of literature suggests that visual processing in near-hand space is altered. The control systems for moving either the eyes or the hands rely on parallel networks of fronto-parietal regions, which have feedback connections to visual areas. Since the oculomotor system effects on visual processing occur through feedback, both through the motor plan and the motor efference copy, a parallel system where reaching and/or grasping motor-related activity also affects visual processing is likely. Areas in the posterior parietal cortex, for example, receive proprioceptive and visual information used to guide actions, as well as motor efference signals. This trio of information channels is all that would be necessary to produce spatial allocation of reach-related visual attention. We review evidence from behavioral and neurophysiological studies that support the hypothesis that feedback from the reaching and/or grasping motor control networks affects visual processing while noting ways in which it differs from that seen within the oculomotor system. We also suggest that object affordances may represent the neural mechanism through which certain object features are selected for preferential processing when stimuli are near the hand. Finally, we summarize the two effector-based feedback systems and discuss how having separate but parallel effector systems allows for efficient decoupling of eye and hand movements.

Evaluation of local field potential signals in decoding of visual attention

In the field of brain research, attention as one of the main issues in cognitive neuroscience is an important mechanism to be studied. The complicated structure of the brain cannot process all the information it receives at any moment. Attention, in fact, is considered as a possible useful mechanism in which brain concentrates on the processing of important information which is required at any certain moment. The main goal of this study is decoding the location of visual attention from local field potential signals recorded from medial temporal (MT) area of a macaque monkey. To this end, feature extraction and feature selection are applied in both the time and the frequency domains. After applying feature extraction methods such as the short time Fourier transform, continuous wavelet transform (CWT), and wavelet energy (scalogram), feature selection methods are evaluated. Feature selection methods used here are T-test, Entropy, receiver operating characteristic, and Bhattacharyya. Subsequently, different classifiers are utilized in order to decode the location of visual attention. At last, the performances of the employed classifiers are compared. The results show that the maximum information about the visual attention in area MT exists in the low frequency features. Interestingly, low frequency features over all the time-axis and all of the frequency features at the initial time interval in the spectrogram domain contain the most valuable information related to the decoding of spatial attention. In the CWT and scalogram domains, this information exists in the low frequency features at the initial time interval. Furthermore, high performances are obtained for these features in both the time and the frequency domains. Among different employed classifiers, the best achieved performance which is about 84.5 % belongs to the K-nearest neighbor classifier combined with the T-test method for feature selection in the time domain. Additionally, the best achieved result (82.9 %) is related to the spectrogram with the least number of selected features as large as 200 features using the T-test method and SVM classifier in the time-frequency domain.

Smooth pursuit preparation modulates neuronal responses in visual areas MT and MST

Primates are able to track small moving visual targets using smooth pursuit eye movements. Target motion for smooth pursuit is signaled by neurons in visual cortical areas MT and MST. In this study, we trained monkeys to either initiate or withhold smooth pursuit in the presence of a moving target to test whether this decision was reflected in the relative strength of "go" and "no-go" processes. We found that the gain of the motor response depended strongly on whether monkeys were instructed to initiate or withhold pursuit, thus demonstrating voluntary control of pursuit initiation. We found that the amplitude of the neuronal response to moving targets in areas MT and MST was also significantly lower on no-go trials (by 2.1 spikes/s on average). The magnitude of the neural response reduction was small compared with the behavioral gain reduction. There were no significant differences in neuronal direction selectivity, spatial selectivity, or response reliability related to pursuit initiation or the absence thereof. Variability in eye speed was negatively correlated with firing rate variability after target motion onset during go trials but not during no-go trials, suggesting that MT and MST activity represents an error signal for a negative feedback controller. We speculate that modulation of the visual motion signals in areas MT and MST may be one of the first visual cortical events in the initiation of smooth pursuit and that the small early response modulation may be amplified to produce an all-or-none motor response by downstream areas.

Hand placement near the visual stimulus improves orientation selectivity in V2 neurons

Often, the brain receives more sensory input than it can process simultaneously. Spatial attention helps overcome this limitation by preferentially processing input from a behaviorally-relevant location. Recent neuropsychological and psychophysical studies suggest that attention is deployed to near-hand space much like how the oculomotor system can deploy attention to an upcoming gaze position. Here we provide the first neuronal evidence that the presence of a nearby hand enhances orientation selectivity in early visual processing area V2. When the hand was placed outside the receptive field, responses to the preferred orientation were significantly enhanced without a corresponding significant increase at the orthogonal orientation. Consequently, there was also a significant sharpening of orientation tuning. In addition, the presence of the hand reduced neuronal response variability. These results indicate that attention is automatically deployed to the space around a hand, improving orientation selectivity. Importantly, this appears to be optimal for motor control of the hand, as opposed to oculomotor mechanisms which enhance responses without sharpening orientation selectivity. Effector-based mechanisms for visual enhancement thus support not only the spatiotemporal dissociation of gaze and reach, but also the optimization of vision for their separate requirements for guiding movements.

Effects of crowding and attention on high-levels of motion processing and motion adaptation

The motion after-effect (MAE) persists in crowding conditions, i.e., when the adaptation direction cannot be reliably perceived. The MAE originating from complex moving patterns spreads into non-adapted sectors of a multi-sector adapting display (i.e., phantom MAE). In the present study we used global rotating patterns to measure the strength of the conventional and phantom MAEs in crowded and non-crowded conditions, and when attention was directed to the adapting stimulus and when it was diverted away from the adapting stimulus. The results show that: (i) the phantom MAE is weaker than the conventional MAE, for both non-crowded and crowded conditions, and when attention was focused on the adapting stimulus and when it was diverted from it, (ii) conventional and phantom MAEs in the crowded condition are weaker than in the non-crowded condition. Analysis conducted to assess the effect of crowding on high-level of motion adaptation suggests that crowding is likely to affect the awareness of the adapting stimulus rather than degrading its sensory representation, (iii) for high-level of motion processing the attentional manipulation does not affect the strength of either conventional or phantom MAEs, neither in the non-crowded nor in the crowded conditions. These results suggest that high-level MAEs do not depend on attention and that at high-level of motion adaptation the effects of crowding are not modulated by attention.

Higher resolution stimulus facilitates depth perception: MT+ plays a significant role in monocular depth perception

Today, we human beings are facing with high-quality virtual world of a completely new nature. For example, we have a digital display consisting of a high enough resolution that we cannot distinguish from the real world. However, little is known how such high-quality representation contributes to the sense of realness, especially to depth perception. What is the neural mechanism of processing such fine but virtual representation? Here, we psychophysically and physiologically examined the relationship between stimulus resolution and depth perception, with using luminance-contrast (shading) as a monocular depth cue. As a result, we found that a higher resolution stimulus facilitates depth perception even when the stimulus resolution difference is undetectable. This finding is against the traditional cognitive hierarchy of visual information processing that visual input is processed continuously in a bottom-up cascade of cortical regions that analyze increasingly complex information such as depth information. In addition, functional magnetic resonance imaging (fMRI) results reveal that the human middle temporal (MT+) plays a significant role in monocular depth perception. These results might provide us with not only the new insight of our neural mechanism of depth perception but also the future progress of our neural system accompanied by state-of- the-art technologies.

The processing of coherent global form and motion patterns without visual awareness

In the present study we addressed whether the processing of global form and motion was dependent on visual awareness. Continuous flash suppression (CFS) was used to suppress from awareness global dot motion (GDM) and Glass pattern stimuli. We quantified the minimum time taken for both pattern types to break suppression with the signal coherence of the pattern (0, 25, 50, and 100% signal) and the type of global structure (rotational, and radial) as independent variables. For both form and motion patterns increasing signal coherence decreased the time required to break suppression. This was the same for both rotational and radial global patterns. However, GDM patterns broke suppression faster than Glass patterns. In a supplementary experiment, we confirmed that this difference in break times is not because of the temporal nature of GDM patterns in attracting attention. In Experiment 2, we examined whether the processing of dynamic Glass patterns were similarly dependent on visual awareness. The processing of dynamic Glass patterns is involves both motion and form systems, and we questioned whether the interaction of these two systems was dependent on visual awareness. The suppression of dynamic Glass patterns was also dependent on signal coherence and the time course of suppression break resembled the detection of global motion and not global form. In Experiment 3 we ruled out the possibility that faster suppression break times was because the visual system is more sensitive to highly coherent form and motion patterns. Here contrast changing GDM and Glass patterns were superimposed on the dynamic CFS mask, and the minimum time required for them to be detected was measured. We showed that there was no difference in detection times for patterns of 0 and 100% coherence. The advantage of highly coherent global motion and form patterns in breaking suppression indicated that the processing and interaction of global motion and form systems occur without visual awareness.

Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices

Computational theories propose that attention modulates the topographical landscape of spatial ‘priority’ maps in regions of visual cortex so that the location of an important object is associated with higher activation levels. While single-unit recording studies have demonstrated attention-related increases in the gain of neural responses and changes in the size of spatial receptive fields, the net effect of these modulations on the topography of region-level priority maps has not been investigated. Here, we used fMRI and a multivariate encoding model to reconstruct spatial representations of attended and ignored stimuli using activation patterns across entire visual areas. These reconstructed spatial representations reveal the influence of attention on the amplitude and size of stimulus representations within putative priority maps across the visual hierarchy. Our results suggest that attention increases the amplitude of stimulus representations in these spatial maps, particularly in higher visual areas, but does not substantively change their size.

Inattention blindness to motion in middle temporal area

Subjects naturally form and use expectations to solve familiar tasks, but the accuracy of these expectations and the neuronal mechanisms by which these expectations enhance behavior are unclear. We trained animals (Macaca mulatta) in a challenging perceptual task in which the likelihood of a very brief pulse of motion was consistently modulated over time and space. Pulse likelihood had dramatic effects on behavior: unexpected pulses were nearly invisible to the animals. To examine the neuronal basis of such inattention blindness, we recorded from single neurons in the middle temporal (MT) area, an area related to motion perception. Fluctuations in how reliably MT neurons both signaled stimulus events and predicted behavioral choices were highly correlated with changes in performance over the course of individual trials. A simple neuronal pooling model reveals that the dramatic behavioral effects of attention in this task can be completely explained by changes in the reliability of a small number of MT neurons.

Monkey area MT latencies to speed changes depend on attention and correlate with behavioral reaction times

Selective visual attention is known to be associated with characteristic modulations of neuronal activity in early visual cortex, but there is only rare evidence showing that these neuronal modulations are directly related to attention-dependent behavioral improvements. Here, we describe a strong, transient increase in the response of neurons in the mediotemporal (MT) area to behaviorally relevant speed changes that is not only modulated by attention but also highly correlated with the animal's performance. In trials with fast reaction time (RT), this transient component occurs with short latency, whereas latency increases monotonically with slower RT. Importantly, RTs are related not to the firing rate modulation during sustained attentive tracking of the target prior to the speed change but to the variability of the neuronal response. Our findings suggest a direct link between attention-dependent response modulations in early visual cortex and improved behavioral performance.

Task-dependent modulation of SI physiological responses to targets and distractors

Selective attention experimental designs have shown that neural responses to stimuli in primary somatosensory cortex are stronger when the sensory stimuli are task relevant. Other studies have used animals under no task demands for data collection. The relationship between neural responses in the brain during behavior, and while an animal has no task demands, remains underexplored. We trained two animals to perform somatosensory detection for several weeks, followed by somatosensory discrimination for several weeks. Data in response to physically identical stimuli were collected from cortical implants while the animal was under no task demands before each behavioral session and also during that behavioral session. The Fourier spectra of the field potentials during detection or discrimination compared with the no task condition demonstrated suppression of the somatosensory μ-rhythm that is associated with readiness and anticipation of cognitive use of somatosensory and motor inputs. Responses to the task target were stronger during detection and discrimination than in the no task condition. The amplitude normalized time course of the target evoked response was similar in both cases. Evoked responses to the task distractor were not significantly stronger during behavior than in recordings under no task demands. The normalized time course of the distractor responses showed a suppression that peaks 30-35 ms after the onset of the response. The selectivity of this within trial suppression is the same as the selectivity of enduring suppression evident in studies of sensory cortical plasticity, which suggests the same neural process may be responsible for both.

Segregated pathways carrying frontally derived top-down signals to visual areas MT and V4 in macaques

The bottom-up processing of visual information is strongly influenced by top-down signals, at least part of which is thought to be conveyed from the frontal cortex through the frontal eye field (FEF) and the lateral intraparietal area (LIP). Here we investigated the architecture of multisynaptic pathways from the frontal cortex to the middle temporal area (MT) of the dorsal visual stream and visual area 4 (V4) of the ventral visual stream in macaques. In the first series of experiments, the retrograde trans-synaptic tracer, rabies virus, was injected into MT or V4. Three days after rabies injections, the second-order (disynaptically connected) neuron labeling appeared in the ventral part of area 46 (area 46v), along with the first-order (monosynaptically connected) neuron labeling in FEF and LIP. In the MT-injection case, second-order neurons were also observed in the supplementary eye field (SEF). In the next series of experiments, double injections of two fluorescent dyes, fast blue and diamidino yellow, were made into MT and V4 to examine whether the frontal inputs are mediated by distinct or common neuronal populations. Virtually no double-labeled neurons were observed in FEF or LIP, indicating that separate neuronal populations mediate the frontal inputs to MT and V4. The present results define that the multisynaptic frontal input to V4 arises primarily from area 46v, whereas the input to MT arises from not only area 46v but also SEF, through distinct FEF and LIP neurons. Segregated pathways from the frontal cortex possibly carry the functionally diverse top-down signals to each visual stream.

Attentional modulations related to spatial gating but not to allocation of limited resources in primate V1

Attention can modulate neural responses in sensory cortical areas and improve behavioral performance in perceptual tasks. However, the nature and purpose of these modulations remain under debate. Here we used voltage-sensitive dye imaging (VSDI) to measure V1 population responses while monkeys performed a difficult detection task under focal or distributed attention. We found that V1 responses at attended locations are significantly elevated relative to actively ignored or irrelevant locations, consistent with the hypothesis that an important goal of attention in V1 is to highlight task-relevant information. Surprisingly, these modulations were indistinguishable under focal and distributed attention, suggesting a minor or no role for attention as a mechanism for allocating limited representational resources in V1. The response elevation at attended locations is additive, is widespread, and starts shortly before stimulus onset. This elevation could contribute to spatial gating by biasing competition in subsequent processing stages in favor of attended stimuli.

Computational advances towards linking BOLD and behavior

Traditionally, fMRI studies have focused on analyzing the mean response amplitude within a cortical area. However, the mean response is blind to many important patterns of cortical modulation, which severely limits the formulation and evaluation of linking hypotheses between neural activity, BOLD responses, and behavior. More recently, multivariate pattern classification analysis (MVPA) has been applied to fMRI data to evaluate the information content of spatially distributed activation patterns. This approach has been remarkably successful at detecting the presence of specific information in targeted brain regions, and provides an extremely flexible means of extracting that information without a precise generative model for the underlying neural activity. However, this flexibility comes at a cost: since MVPA relies on pooling information across voxels that are selective for many different stimulus attributes, it is difficult to infer how specific sub-sets of tuned neurons are modulated by an experimental manipulation. In contrast, recently developed encoding models can produce more precise estimates of feature-selective tuning functions, and can support the creation of explicit linking hypotheses between neural activity and behavior. Although these encoding models depend on strong - and often untested - assumptions about the response properties of underlying neural generators, they also provide a unique opportunity to evaluate population-level computational theories of perception and cognition that have previously been difficult to assess using either single-unit recording or conventional neuroimaging techniques.

Spikes, BOLD, attention, and awareness: a comparison of electrophysiological and fMRI signals in V1

Early fMRI studies comparing results from fMRI and electrophysiological experiments support the notion that the blood oxygen level-dependent (BOLD) signal reliably follows the spiking activity of an underlying neuronal population averaged across a small region in space and a brief period in time. However, more recent studies focusing on higher level cognitive factors such as attention and visual awareness report striking discrepancies between the fMRI response in humans and electrophysiological signals in macaque early visual areas. Four hypotheses are discussed that can explain the discrepancies between the two methods: (1) the BOLD signal follows local field potential (LFP) signals closer than spikes, and only the LFP is modulated by top-down factors, (2) the BOLD signal is reflecting electrophysiological signals that are occurring later in time due to feedback delay, (3) the BOLD signal is more sensitive than traditional electrophysiological methods due to massive pooling by the hemodynamic coupling process, and finally (4) there is no real discrepancy, and instead, weak but reliable effects on firing rates may be obscured by differences in experimental design and interpretation of results across methods.

Expansion of MT neurons excitatory receptive fields during covert attentive tracking

Primates can attentively track moving objects while keeping gaze stationary. The neural mechanisms underlying this ability are poorly understood. We investigated this issue by recording responses of neurons in area MT of two rhesus monkeys while they performed two different tasks. During the Attend-Fixation task, two moving random dot patterns (RDPs) translated across a screen at the same speed and in the same direction while the animals directed gaze to a fixation spot and detected a change in its luminance. During the Tracking task, the animals kept gaze on the fixation spot and attentively tracked the two RDPs to report a change in the local speed of one of the patterns' dots. In both conditions, neuronal responses progressively increased as the RDPs entered the neurons' receptive field (RF), peaked when they reached its center, and decreased as they translated away. This response profile was well described by a Gaussian function with its center of gravity indicating the RF center and its flanks the RF excitatory borders. During Tracking, responses were increased relative to Attend-Fixation, causing the Gaussian profiles to expand. Such increases were proportionally larger in the RF periphery than at its center, and were accompanied by a decrease in the trial-to-trial response variability (Fano factor) relative to Attend-Fixation. These changes resulted in an increase in the neurons' performance at detecting targets at longer distances from the RF center. Our results show that attentive tracking dynamically changes MT neurons' RF profiles, ultimately improving the neurons' ability to encode the tracked stimulus features.

Attentional control of sensory tuning in human visual perception

Attention is known to affect the response properties of sensory neurons in visual cortex. These effects have been traditionally classified into two categories: 1) changes in the gain (overall amplitude) of the response; and 2) changes in the tuning (selectivity) of the response. We performed an extensive series of behavioral measurements using psychophysical reverse correlation to understand whether/how these neuronal changes are reflected at the level of our perceptual experience. This question has been addressed before, but by different laboratories using different attentional manipulations and stimuli/tasks that are not directly comparable, making it difficult to extract a comprehensive and coherent picture from existing literature. Our results demonstrate that the effect of attention on response gain (not necessarily associated with tuning change) is relatively aspecific: it occurred across all the conditions we tested, including attention directed to a feature orthogonal to the primary feature for the assigned task. Sensory tuning, however, was affected primarily by feature-based attention and only to a limited extent by spatially directed attention, in line with existing evidence from the electrophysiological and behavioral literature.

Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets

How does the brain coordinate saccadic and smooth pursuit eye movements to track objects that move in unpredictable directions and speeds? Saccadic eye movements rapidly foveate peripheral visual or auditory targets, and smooth pursuit eye movements keep the fovea pointed toward an attended moving target. Analyses of tracking data in monkeys and humans reveal systematic deviations from predictions of the simplest model of saccade-pursuit interactions, which would use no interactions other than common target selection and recruitment of shared motoneurons. Instead, saccadic and smooth pursuit movements cooperate to cancel errors of gaze position and velocity, and thus to maximize target visibility through time. How are these two systems coordinated to promote visual localization and identification of moving targets? How are saccades calibrated to correctly foveate a target despite its continued motion during the saccade? The neural model proposed here answers these questions. Modeled interactions encompass motion processing areas MT, MST, FPA, DLPN and NRTP; saccade planning and execution areas FEF, LIP, and SC; the saccadic generator in the brain stem; and the cerebellum. Simulations illustrate the model's ability to functionally explain and quantitatively simulate anatomical, neurophysiological and behavioral data about coordinated saccade-pursuit tracking.

Perceptual and neural consequences of rapid motion adaptation

Nervous systems adapt to the prevailing sensory environment, and the consequences of this adaptation can be observed in the responses of single neurons and in perception. Given the variety of timescales underlying events in the natural world, determining the temporal characteristics of adaptation is important to understanding how perception adjusts to its sensory environment. Previous work has shown that neural adaptation can occur on a timescale of milliseconds, but perceptual adaptation has generally been studied over relatively long timescales, typically on the order of seconds. This disparity raises important questions. Can perceptual adaptation be observed at brief, functionally relevant timescales? And if so, how do its properties relate to the rapid adaptation seen in cortical neurons? We address these questions in the context of visual motion processing, a perceptual modality characterized by rapid temporal dynamics. We demonstrate objectively that 25 ms of motion adaptation is sufficient to generate a motion aftereffect, an illusory sensation of movement experienced when a moving stimulus is replaced by a stationary pattern. This rapid adaptation occurs regardless of whether the adapting motion is perceived. In neurophysiological recordings from the middle temporal area of primate visual cortex, we find that brief motion adaptation evokes direction-selective responses to subsequently presented stationary stimuli. A simple model shows that these neural responses can explain the consequences of rapid perceptual adaptation. Overall, we show that the motion aftereffect is not merely an intriguing perceptual illusion, but rather a reflection of rapid neural and perceptual processes that can occur essentially every time we experience motion.

Visual attention: the past 25 years

This review focuses on covert attention and how it alters early vision. I explain why attention is considered a selective process, the constructs of covert attention, spatial endogenous and exogenous attention, and feature-based attention. I explain how in the last 25 years research on attention has characterized the effects of covert attention on spatial filters and how attention influences the selection of stimuli of interest. This review includes the effects of spatial attention on discriminability and appearance in tasks mediated by contrast sensitivity and spatial resolution; the effects of feature-based attention on basic visual processes, and a comparison of the effects of spatial and feature-based attention. The emphasis of this review is on psychophysical studies, but relevant electrophysiological and neuroimaging studies and models regarding how and where neuronal responses are modulated are also discussed.

The neural basis of visual attention

Visual attention is the mechanism the nervous system uses to highlight specific locations, objects or features within the visual field. This can be accomplished by making an eye movement to bring the object onto the fovea (overt attention) or by increased processing of visual information in neurons representing more peripheral regions of the visual field (covert attention). This review will examine two aspects of visual attention: the changes in neural responses within visual cortices due to the allocation of covert attention; and the neural activity in higher cortical areas involved in guiding the allocation of attention. The first section will highlight processes that occur during visual spatial attention and feature-based attention in cortical visual areas and several related models that have recently been proposed to explain this activity. The second section will focus on the parietofrontal network thought to be involved in targeting eye movements and allocating covert attention. It will describe evidence that the lateral intraparietal area, frontal eye field and superior colliculus are involved in the guidance of visual attention, and describe the priority map model, which is thought to operate in at least several of these areas.

Oscillations and filtering networks support flexible routing of information

The mammalian brain exhibits profuse interregional connectivity. How information flow is rapidly and flexibly switched among connected areas remains poorly understood. Task-dependent changes in the power and interregion coherence of network oscillations suggest that such oscillations play a role in signal routing. We show that switching one of several convergent pathways from an asynchronous to an oscillatory state allows accurate selective transmission of population-coded information, which can be extracted even when other convergent pathways fire asynchronously at comparable rates. We further show that the band-pass filtering required to perform this information extraction can be implemented in a simple spiking network model with a single feed-forward interneuron layer. This constitutes a mechanism for flexible signal routing in neural circuits, which exploits sparsely synchronized network oscillations and temporal filtering by feed-forward inhibition.

Frequency-dependent attentional modulation of local field potential signals in macaque area MT

Visual attention modulates neuronal responses in primate motion processing area MT. However, whether it modulates the strength local field potentials (LFP-power) within this area remains unexplored, as well as how this modulation relates to the one of the neurons' response. We investigated these issues by simultaneously recording LFPs and neuronal responses evoked by moving random dot patterns of varying direction and contrast in area MT of two male monkeys (Macaca mulatta) during different behavioral conditions. We found that: (1) LFP-power in the gamma (30-120 Hz), but not in the delta (2-4 Hz), (4-8 Hz), alpha (8-12 Hz), beta(1) (12-20 Hz), and beta(2) (20-30 Hz) frequency bands, was tuned for motion direction and contrast, similarly to the neurons' response, (2) shifting attention into a neuron's receptive field (RF) decreased LFP-power in the bands below 30 Hz (except the band), whereas shifting attention to a stimulus motion direction outside the RF had no effect in these bands, (3) LFP-power in the gamma band, however, exhibited both spatial- and motion direction-dependent attentional modulation (increase or decrease), which was highly correlated with the modulation of the neurons' response. These results demonstrate that in area MT, shifting attention into the RFs of neurons in the vicinity of the recording electrode, or to the direction of a moving stimulus located far away from these RFs, distinctively modulates LFP-power in the various frequency bands. They further suggest differences in the neural mechanisms underlying these types of attentional modulation of visual processing.

Relationships between the threshold and slope of psychometric and neurometric functions during perceptual learning: implications for neuronal pooling

Perceptual learning involves long-lasting improvements in the ability to perceive simple sensory stimuli. Some forms of perceptual learning are thought to involve an increasingly selective readout of sensory neurons that are most sensitive to the trained stimulus. Here we report novel changes in the relationship between the threshold and slope of the psychometric function during learning that are consistent with such changes in readout and can provide insights into the underlying neural mechanisms. In monkeys trained on a direction-discrimination task, perceptual improvements corresponded to lower psychometric thresholds and slightly shallower slopes. However, this relationship between threshold and slope was much weaker in comparable, ideal-observer "neurometric" functions of neurons in the middle temporal (MT) area, which represent sensory information used to perform the task and whose response properties did not change with training. We propose a linear/nonlinear pooling scheme to account for these results. According to this scheme, MT responses are pooled via linear weights that change with training to more selectively read out responses from the most sensitive neurons, thereby reducing predicted thresholds. An additional nonlinear (power-law) transformation does not change with training and causes the predicted psychometric function to become shallower as uninformative neurons are eliminated from the pooled signal. We show that this scheme is consistent with the measured changes in psychometric threshold and slope throughout training. The results suggest that some forms of perceptual learning involve improvements in a process akin to selective attention that pools the most informative neural signals to guide behavior.

The role of the parietal lobe in visual extinction studied with transcranial magnetic stimulation

Interhemispheric competition between homologous areas in the human brain is believed to be involved in a wide variety of human behaviors from motor activity to visual perception and particularly attention. For example, patients with lesions in the posterior parietal cortex are unable to selectively track objects in the contralesional side of visual space when targets are simultaneously present in the ipsilesional visual field, a form of visual extinction. Visual extinction may arise due to an imbalance in the normal interhemispheric competition. To directly assess the issue of reciprocal inhibition, we used fMRI to localize those brain regions active during attention-based visual tracking and then applied low-frequency repetitive transcranial magnetic stimulation over identified areas in the left and right intraparietal sulcus to asses the behavioral effects on visual tracking. We induced a severe impairment in visual tracking that was selective for conditions of simultaneous tracking in both visual fields. Our data show that the parietal lobe is essential for visual tracking and that the two hemispheres compete for attentional resources during tracking. Our results provide a neuronal basis for visual extinction in patients with parietal lobe damage.

Attention directed by expectations enhances receptive fields in cortical area MT

Expectations, especially those formed on the basis of extensive training, can substantially enhance visual performance. However, it is not clear that the physiological mechanisms underlying this enhancement are identical to those examined by experiments in which attention is directed by explicit instructions rather than strong expectations. To study the changes in visual representations associated with strong expectations, we trained animals to detect a brief motion pulse that was embedded in noise. Because the nature of the pulse and the statistics of its appearance were well known to the animals, they formed strong expectations which determined their behavioral performance. We used white-noise methods to infer the receptive field structure of single neurons in area MT while they were performing this task. Incorporating non-linearities, we compared receptive fields during periods of time when the animals were expecting the motion pulse with periods of time when they were not. We found receptive field changes consistent with an increased reliability in signaling pulse occurrence. Moreover, these changes were not consistent with a simple gain modulation. The results suggest that strong expectations can create very specific changes in the visual representations at a cellular level to enhance performance.

Type of featural attention differentially modulates hMT+ responses to illusory motion aftereffects

Activity in the human motion complex (hMT(+)/V5) is related to the perception of motion, be it either real surface motion or an illusion of motion such as apparent motion (AM) or motion aftereffect (MAE). It is a long-lasting debate whether illusory motion-related activations in hMT(+) represent the motion itself or attention to it. We have asked whether hMT(+) responses to MAEs are present when shifts in arousal are suppressed and attention is focused on concurrent motion versus nonmotion features. Significant enhancement of hMT(+) activity was observed during MAEs when attention was focused either on concurrent spatial angle or color features. This observation was confirmed by direct comparison of adapting (MAE inducing) versus nonadapting conditions. In contrast, this effect was diminished when subjects had to report on concomitant speed changes of superimposed AM. The same finding was observed for concomitant orthogonal real motion (RM), suggesting that selective attention to concurrent illusory or real motion was interfering with the saliency of MAE signals in hMT(+). We conclude that MAE-related changes in the global activity of hMT(+) are present provided selective attention is not focused on an interfering feature such as concurrent motion. Accordingly, there is a genuine MAE-related motion signal in hMT(+) that is neither explained by shifts in arousal nor by selective attention.

Target Selection by the Frontal Cortex during Coordinated Saccadic and Smooth Pursuit Eye Movements

Oculomotor tracking of moving objects is an important component of visually based cognition and planning. Such tracking is achieved by a combination of saccades and smooth-pursuit eye movements. In particular, the saccadic and smooth-pursuit systems interact to often choose the same target, and to maximize its visibility through time. How do multiple brain regions interact, including frontal cortical areas, to decide the choice of a target among several competing moving stimuli? How is target selection information that is created by a bias (e.g., electrical stimulation) transferred from one movement system to another? These saccade–pursuit interactions are clarified by a new computational neural model, which describes interactions between motion processing areas: the middle temporal area, the middle superior temporal area, the frontal pursuit area, and the dorsal lateral pontine nucleus; saccade specification, selection, and planning areas: the lateral intraparietal area, the frontal eye fields, the substantia nigra pars reticulata, and the superior colliculus; the saccadic generator in the brain stem; and the cerebellum. Model simulations explain a broad range of neuroanatomical and neurophysiological data. These results are in contrast with the simplest parallel model with no interactions between saccades and pursuit other than common-target selection and recruitment of shared motoneurons. Actual tracking episodes in primates reveal multiple systematic deviations from predictions of the simplest parallel model, which are explained by the current model.

Attentional modulation of visual responses by flexible input gain

Although it is clear that sensory responses in the cortex can be strongly modulated by stimuli outside of classical receptive fields as well as by extraretinal signals such as attention and anticipation, the exact rules governing the neuronal integration of sensory and behavioral signals remain unclear. For example, most experiments studying sensory interactions have not explored attention, while most studies of attention have relied on the responses to relatively limited sets of stimuli. However, a recent study of V4 responses, in which location, orientation, and spatial attention were systematically varied, suggests that attention can both facilitate and suppress specific sensory inputs to a neuron according to behavioral relevance. To explore the implications of such input gain, we modeled the effects of a center-surround organization of attentional modulation using existing receptive field models of sensory integration. The model is consistent with behavioral measurements of a suppressive effect that surrounds the facilitatory locus of spatial attention. When this center-surround modulation is incorporated into realistic models of sensory integration, it is able to explain seemingly disparate observations of attentional effects in the neurophysiological literature, including spatial shifts in receptive field position and the preferential modulation of low contrast stimuli. The model is also consistent with recent formulations of attention to features in which gain is variably applied among cells with different receptive field properties. Consistent with functional imaging results, the model predicts that spatial attention effects will vary between different visual areas and suggests that attention may act through a common mechanism of selective and flexible gain throughout the visual system.

Context-dependent changes in functional circuitry in visual area MT

Animals can flexibly change their behavior in response to a particular sensory stimulus; the mapping between sensory and motor representations in the brain must therefore be flexible as well. Changes in the correlated firing of pairs of neurons may provide a metric of changes in functional circuitry during behavior. We studied dynamic changes in functional circuitry by analyzing the noise correlations of simultaneously recorded MT neurons in two behavioral contexts: one that promotes cooperative interactions between the two neurons and another that promotes competitive interactions. We found that identical visual stimuli give rise to differences in noise correlation in the two contexts, suggesting that MT neurons receive inputs of central origin whose strength changes with the task structure. The data are consistent with a mixed feature-based attentional strategy model in which the animal sometimes alternates attention between opposite directions of motion and sometimes attends to the two directions simultaneously.

Attention and active vision

Visual perception results from the interaction of incoming sensory signals and top down cognitive and motor signals. Here we focus on the representation of attended locations in parietal cortex and in earlier visual cortical areas. We review evidence that these spatial representations are modulated not only by selective attention but also by the intention to move the eyes. We describe recent experiments in monkey and human that elucidate the mechanisms and circuitry involved in updating, or remapping, the representations of salient stimuli. Two central ideas emerge. First, selective attention and remapping are closely intertwined, and together contribute to the percept of spatial stability. Second, remapping is accomplished not by a single area but by the participation of parietal, frontal and extrastriate cortex as well as subcortical structures. This neural circuitry is distinguished by significant redundancy and plasticity, suggesting that the updating of salient stimuli is fundamental for spatial stability and visuospatial behavior. We conclude that multiple processes and pathways contribute to active vision in the primate brain.

Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area

This study aimed to identify neural mechanisms that underlie perceptual learning in a visual-discrimination task. We trained two monkeys (Macaca mulatta) to determine the direction of visual motion while we recorded from their middle temporal area (MT), which in trained monkeys represents motion information that is used to solve the task, and lateral intraparietal area (LIP), which represents the transformation of motion information into a saccadic choice. During training, improved behavioral sensitivity to weak motion signals was accompanied by changes in motion-driven responses of neurons in LIP, but not in MT. The time course and magnitude of the changes in LIP correlated with the changes in behavioral sensitivity throughout training. Thus, for this task, perceptual learning does not appear to involve improvements in how sensory information is represented in the brain, but rather how the sensory representation is interpreted to form the decision that guides behavior.