Figure-Ground Modulation in the Human Lateral Geniculate Nucleus Is Distinguishable from Top-Down Attention

How does orientation-tuned normalization spread across the visual field?

Visuocortical responses are regulated by gain control mechanisms, giving rise to fundamental neural and perceptual phenomena such as surround suppression. Suppression strength, determined by the composition and relative properties of stimuli, controls the strength of neural responses in early visual cortex, and in turn, the subjective salience of the visual stimulus. Notably, suppression strength is modulated by feature similarity; for instance, responses to a center-surround stimulus in which the components are collinear to each other are weaker than when they are orthogonal. However, this feature-tuned aspect of normalization, and how it may affect the gain of responses, has been understudied. Here, we examine the contribution of the tuned component of suppression to contrast response modulations across the visual field. To do so, we used functional magnetic resonance imaging (fMRI) to measure contrast response functions (CRFs) in early visual cortex (areas V1-V3) in 10 observers while they viewed full-field center-surround gratings. The center stimulus varied in contrast between 2.67% and 96% and was surrounded by a collinear or orthogonal surround at full contrast. We found substantially stronger suppression of responses when the surround was parallel to the center, manifesting as shifts in the population CRF. The magnitude of the CRF shift was strongly dependent on voxel spatial preference and seen primarily in voxels whose receptive field spatial preference corresponds to the area straddling the center-surround boundary in our display, with little-to-no modulation elsewhere.NEW & NOTEWORTHY Visuocortical responses are underpinned by gain control mechanisms. In surround suppression, it has been shown that suppression strength is affected by the orientation similarity between the center and surround stimuli. In this study, we examine the impact of orientation-tuned suppression on population contrast responses in early visual cortex and its spread across the visual field. Results show stronger suppression in parallel stimulus configurations, with suppression largely isolated to voxels near the center-surround boundary.

Population coding for figure-ground texture segregation in macaque V1 and V4

Object recognition often involves the brain segregating objects from their surroundings. Neurophysiological studies of figure-ground texture segregation have yielded inconsistent results, particularly on whether V1 neurons can perform figure-ground texture segregation or just detect texture borders. To address this issue from a population perspective, we utilized two-photon calcium imaging to simultaneously record the responses of large samples of V1 and V4 neurons to figure-ground texture stimuli in awake, fixating macaques. The average response changes indicate that V1 neurons mainly detect texture borders, while V4 neurons are involved in figure-ground segregation. However, population analysis (SVM decoding of PCA-transformed neuronal responses) reveal that V1 neurons not only detect figure-ground borders, but also contribute to figure-ground texture segregation, although requiring substantially more principal components than V4 neurons to reach a 75 % decoding accuracy. Individually, V1/V4 neurons showing larger (negative/positive) figure-ground response differences contribute more to figure-ground segregation. But for V1 neurons, the contribution becomes significant only when many principal components are considered. We conclude that V1 neurons participate in figure-ground segregation primarily by defining the figure borders, and the poorly structured figure-ground information V1 neurons carry could be further utilized by V4 neurons to accomplish figure-ground segregation.

Convolutional neural network models applied to neuronal responses in macaque V1 reveal limited nonlinear processing

Computational models of the primary visual cortex (V1) have suggested that V1 neurons behave like Gabor filters followed by simple nonlinearities. However, recent work employing convolutional neural network (CNN) models has suggested that V1 relies on far more nonlinear computations than previously thought. Specifically, unit responses in an intermediate layer of VGG-19 were found to best predict macaque V1 responses to thousands of natural and synthetic images. Here, we evaluated the hypothesis that the poor performance of lower layer units in VGG-19 might be attributable to their small receptive field size rather than to their lack of complexity per se. We compared VGG-19 with AlexNet, which has much larger receptive fields in its lower layers. Whereas the best-performing layer of VGG-19 occurred after seven nonlinear steps, the first convolutional layer of AlexNet best predicted V1 responses. Although the predictive accuracy of VGG-19 was somewhat better than that of standard AlexNet, we found that a modified version of AlexNet could match the performance of VGG-19 after only a few nonlinear computations. Control analyses revealed that decreasing the size of the input images caused the best-performing layer of VGG-19 to shift to a lower layer, consistent with the hypothesis that the relationship between image size and receptive field size can strongly affect model performance. We conducted additional analyses using a Gabor pyramid model to test for nonlinear contributions of normalization and contrast saturation. Overall, our findings suggest that the feedforward responses of V1 neurons can be well explained by assuming only a few nonlinear processing stages.

Training-induced changes in population receptive field properties in visual cortex: Impact of eccentric vision training on population receptive field properties and the crowding effect

This study aimed to investigate the impact of eccentric-vision training on population receptive field (pRF) estimates to provide insights into brain plasticity processes driven by practice. Fifteen participants underwent functional magnetic resonance imaging (fMRI) measurements before and after behavioral training on a visual crowding task, where the relative orientation of the opening (gap position: up/down, left/right) in a Landolt C optotype had to be discriminated in the presence of flanking ring stimuli. Drifting checkerboard bar stimuli were used for pRF size estimation in multiple regions of interest (ROIs): dorsal-V1 (dV1), dorsal-V2 (dV2), ventral-V1 (vV1), and ventral-V2 (vV2), including the visual cortex region corresponding to the trained retinal location. pRF estimates in V1 and V2 were obtained along eccentricities from 0.5° to 9°. Statistical analyses revealed a significant decrease of the crowding anisotropy index (p = 0.009) after training, indicating improvement on crowding task performance following training. Notably, pRF sizes at and near the trained location decreased significantly (p = 0.005). Dorsal and ventral V2 exhibited significant pRF size reductions, especially at eccentricities where the training stimuli were presented (p < 0.001). In contrast, no significant changes in pRF estimates were found in either vV1 (p = 0.181) or dV1 (p = 0.055) voxels. These findings suggest that practice on a crowding task can lead to a reduction of pRF sizes in trained visual cortex, particularly in V2, highlighting the plasticity and adaptability of the adult visual system induced by prolonged training.

Involvement of superior colliculus in complex figure detection of mice

Object detection is an essential function of the visual system. Although the visual cortex plays an important role in object detection, the superior colliculus can support detection when the visual cortex is ablated or silenced. Moreover, it has been shown that superficial layers of mouse SC (sSC) encode visual features of complex objects, and that this code is not inherited from the primary visual cortex. This suggests that mouse sSC may provide a significant contribution to complex object vision. Here, we use optogenetics to show that mouse sSC is involved in figure detection based on differences in figure contrast, orientation, and phase. Additionally, our neural recordings show that in mouse sSC, image elements that belong to a figure elicit stronger activity than those same elements when they are part of the background. The discriminability of this neural code is higher for correct trials than for incorrect trials. Our results provide new insight into the behavioral relevance of the visual processing that takes place in sSC.

A selection and targeting framework of cortical locations for line-scanning fMRI

Depth-resolved functional magnetic resonance imaging (fMRI) is an emerging field growing in popularity given the potential of separating signals from different computational processes in cerebral cortex. Conventional acquisition schemes suffer from low spatial and temporal resolutions. Line-scanning methods allow depth-resolved fMRI by sacrificing spatial coverage to sample blood oxygenated level-dependent (BOLD) responses at ultra-high temporal and spatial resolution. For neuroscience applications, it is critical to be able to place the line accurately to (1) sample the right neural population and (2) target that neural population with tailored stimuli or tasks. To this end, we devised a multi-session framework where a target cortical location is selected based on anatomical and functional properties. The line is then positioned according to this information in a separate second session, and we tailor the experiment to focus on the target location. Anatomically, the precision of the line placement was confirmed by projecting a nominal representation of the acquired line back onto the surface. Functional estimates of neural selectivities in the line, as quantified by a visual population-receptive field model, resembled the target selectivities well for most subjects. This functional precision was quantified in detail by estimating the distance between the visual field location of the targeted vertex and the location in visual cortex (V1) that most closely resembled the line-scanning estimates; this distance was on average ~5.5 mm. Given the dimensions of the line, differences in acquisition, session, and stimulus design, this validates that line-scanning can be used to probe local neural sensitivities across sessions. In summary, we present an accurate framework for line-scanning MRI; we believe such a framework is required to harness the full potential of line-scanning and maximize its utility. Furthermore, this approach bridges canonical fMRI experiments with electrophysiological experiments, which in turn allows novel avenues for studying human physiology non-invasively.

Different patterns of foreground and background processing contribute to texture segregation in humans: an electrophysiological study

Background

Figure-ground segregation is a necessary process for accurate visual recognition. Previous neurophysiological and human brain imaging studies have suggested that foreground-background segregation relies on both enhanced foreground representation and suppressed background representation. However, in humans, it is not known when and how foreground and background processing play a role in texture segregation.

Methods

To answer this question, it is crucial to extract and dissociate the neural signals elicited by the foreground and background of a figure texture with high temporal resolution. Here, we combined an electroencephalogram (EEG) recording and a temporal response function (TRF) approach to specifically track the neural responses to the foreground and background of a figure texture from the overall EEG recordings in the luminance-tracking TRF. A uniform texture was included as a neutral condition. The texture segregation visual evoked potential (tsVEP) was calculated by subtracting the uniform TRF from the foreground and background TRFs, respectively, to index the specific segregation activity.

Results

We found that the foreground and background of a figure texture were processed differently during texture segregation. In the posterior region of the brain, we found a negative component for the foreground tsVEP in the early stage of foreground-background segregation, and two negative components for the background tsVEP in the early and late stages. In the anterior region, we found a positive component for the foreground tsVEP in the late stage, and two positive components for the background tsVEP in the early and late stages of texture processing.

Discussion

In this study we investigated the temporal profile of foreground and background processing during texture segregation in human participants at a high time resolution. The results demonstrated that the foreground and background jointly contribute to figure-ground segregation in both the early and late phases of texture processing. Our findings provide novel evidence for the neural correlates of foreground-background modulation during figure-ground segregation in humans.

Backward masking implicates cortico-cortical recurrent processes in convex figure context effects and cortico-thalamic recurrent processes in resolving figure-ground ambiguity

Introduction

Previous experiments purportedly showed that image-based factors like convexity were sufficient for figure assignment. Recently, however, we found that the probability of perceiving a figure on the convex side of a central border was only slightly higher than chance for two-region displays and increased with the number of display regions; this increase was observed only when the concave regions were homogeneously colored. These convex figure context effects (CEs) revealed that figure assignment in these classic displays entails more than a response to local convexity. A Bayesian observer replicated the convex figure CEs using both a convexity object prior and a new, homogeneous background prior and made the novel prediction that the classic displays in which both the convex and concave regions were homogeneous were ambiguous during perceptual organization.

Methods

Here, we report three experiments investigating the proposed ambiguity and examining how the convex figure CEs unfold over time with an emphasis on whether they entail recurrent processing. Displays were shown for 100 ms followed by pattern masks after ISIs of 0, 50, or 100 ms. The masking conditions were designed to add noise to recurrent processing and therefore to delay the outcome of processes in which they play a role. In Exp. 1, participants viewed two- and eight-region displays with homogeneous convex regions (homo-convex displays; the putatively ambiguous displays). In Exp. 2, participants viewed putatively unambiguous hetero-convex displays. In Exp. 3, displays and masks were presented to different eyes, thereby delaying mask interference in the thalamus for up to 100 ms.

Results and discussion

The results of Exps. 1 and 2 are consistent with the interpretation that recurrent processing is involved in generating the convex figure CEs and resolving the ambiguity of homo-convex displays. The results of Exp. 3 suggested that corticofugal recurrent processing is involved in resolving the ambiguity of homo-convex displays and that cortico-cortical recurrent processes play a role in generating convex figure CEs and these two types of recurrent processes operate in parallel. Our results add to evidence that perceptual organization evolves dynamically and reveal that stimuli that seem unambiguous can be ambiguous during perceptual organization.

Ultra-High Field Imaging of Human Visual Cognition

Functional magnetic resonance imaging (fMRI), the key methodology for mapping the functions of the human brain in a noninvasive manner, is limited by low temporal and spatial resolution. Recent advances in ultra-high field (UHF) fMRI provide a mesoscopic (i.e., submillimeter resolution) tool that allows us to probe laminar and columnar circuits, distinguish bottom-up versus top-down pathways, and map small subcortical areas. We review recent work demonstrating that UHF fMRI provides a robust methodology for imaging the brain across cortical depths and columns that provides insights into the brain's organization and functions at unprecedented spatial resolution, advancing our understanding of the fine-scale computations and interareal communication that support visual cognition.

Rapid adaptation of primate LGN neurons to drifting grating stimulation

The visual system needs to dynamically adapt to changing environments. Much is known about the adaptive effects of constant stimulation over prolonged periods. However, there are open questions regarding adaptation to stimuli that are changing over time, interrupted, or repeated. Feature-specific adaptation to repeating stimuli has been shown to occur as early as primary visual cortex (V1), but there is also evidence for more generalized, fatigue-like adaptation that might occur at an earlier stage of processing. Here, we show adaptation in the lateral geniculate nucleus (LGN) of awake, fixating monkeys following brief (1 s) exposure to repeated cycles of a 4-Hz drifting grating. We examined the relative change of each neuron's response across successive (repeated) grating cycles. We found that neurons from all cell classes (parvocellular, magnocellular, and koniocellular) showed significant adaptation. However, only magnocellular neurons showed adaptation when responses were averaged to a population response. In contrast to firing rates, response variability was largely unaffected. Finally, adaptation was comparable between monocular and binocular stimulation, suggesting that rapid LGN adaptation is monocular in nature.NEW & NOTEWORTHY Neural adaptation can be defined as reduction of spiking responses following repeated or prolonged stimulation. Adaptation helps adjust neural responsiveness to avoid saturation and has been suggested to improve perceptual selectivity, information transmission, and predictive coding. Here, we report rapid adaptation to repeated cycles of gratings drifting over the receptive field of neurons at the earliest site of postretinal processing, the lateral geniculate nucleus of the thalamus.

Neural representations of competing stimuli along the dorsal and ventral visual pathways during binocular rivalry

Binocular rivalry arises when two discrepant stimuli are simultaneously presented to different eyes, during which observers consciously experience vivid perceptual alternations without physical changes in visual inputs. Neural dynamics tracking such perceptual alternations have been identified at both early and late visual areas, leading to the fundamental debate concerning the primary neural substrate underlying binocular rivalry. One promising hypothesis that might reconcile these seemingly paradoxical findings is a gradual shift from interocular competition between monocular neurons to pattern competition among binocular neurons. Here, we examined this hypothesis by investigating how neural representations of rivalrous stimuli evolved along the visual pathway. We found that representations of the dominant and the suppressed stimuli initially co-existed in V1, which were enhanced and attenuated respectively in extrastriate visual areas. Notably, neural activity in V4 was dictated by the representation of the dominant stimulus, while the representation of the suppressed stimulus was only partially inhibited in dorsal areas V3A and MT+. Our findings revealed a progressive transition from the co-existing representations of the rivalrous inputs to the dictatorial representation of the dominant stimulus in the ventral pathway, and advocated different cortical evolutionary patterns of visual representations between the dorsal and the ventral pathways.

Attention differentially modulates multiunit activity in the lateral geniculate nucleus and V1 of macaque monkeys

Attention promotes the selection of behaviorally relevant sensory signals from the barrage of sensory information available. Visual attention modulates the gain of neuronal activity in all visual brain areas examined, although magnitudes of gain modulations vary across areas. For example, attention gain magnitudes in the dorsal lateral geniculate nucleus (LGN) and primary visual cortex (V1) vary tremendously across fMRI measurements in humans and electrophysiological recordings in behaving monkeys. We sought to determine whether these discrepancies are due simply to differences in species or measurement, or more nuanced properties unique to each visual brain area. We also explored whether robust and consistent attention effects, comparable to those measured in humans with fMRI, are observable in the LGN or V1 of monkeys. We measured attentional modulation of multiunit activity in the LGN and V1 of macaque monkeys engaged in a contrast change detection task requiring shifts in covert visual spatial attention. Rigorous analyses of LGN and V1 multiunit activity revealed robust and consistent attentional facilitation throughout V1, with magnitudes comparable to those observed with fMRI. Interestingly, attentional modulation in the LGN was consistently negligible. These findings demonstrate that discrepancies in attention effects are not simply due to species or measurement differences. We also examined whether attention effects correlated with the feature selectivity of recorded multiunits. Distinct relationships suggest that attentional modulation of multiunit activity depends upon the unique structure and function of visual brain areas.

Robust effects of corticothalamic feedback and behavioral state on movie responses in mouse dLGN

Neurons in the dorsolateral geniculate nucleus (dLGN) of the thalamus receive a substantial proportion of modulatory inputs from corticothalamic (CT) feedback and brain stem nuclei. Hypothesizing that these modulatory influences might be differentially engaged depending on the visual stimulus and behavioral state, we performed in vivo extracellular recordings from mouse dLGN while optogenetically suppressing CT feedback and monitoring behavioral state by locomotion and pupil dilation. For naturalistic movie clips, we found CT feedback to consistently increase dLGN response gain and promote tonic firing. In contrast, for gratings, CT feedback effects on firing rates were mixed. For both stimulus types, the neural signatures of CT feedback closely resembled those of behavioral state, yet effects of behavioral state on responses to movies persisted even when CT feedback was suppressed. We conclude that CT feedback modulates visual information on its way to cortex in a stimulus-dependent manner, but largely independently of behavioral state.

Early-stage visual perception impairment in schizophrenia, bottom-up and back again

Visual perception is one of the basic tools for exploring the world. However, in schizophrenia, this modality is disrupted. So far, there has been no clear answer as to whether the disruption occurs primarily within the brain or in the precortical areas of visual perception (the retina, visual pathways, and lateral geniculate nucleus [LGN]). A web-based comprehensive search of peer-reviewed journals was conducted based on various keyword combinations including schizophrenia, saliency, visual cognition, visual pathways, retina, and LGN. Articles were chosen with respect to topic relevance. Searched databases included Google Scholar, PubMed, and Web of Science. This review describes the precortical circuit and the key changes in biochemistry and pathophysiology that affect the creation and characteristics of the retinal signal as well as its subsequent modulation and processing in other parts of this circuit. Changes in the characteristics of the signal and the misinterpretation of visual stimuli associated with them may, as a result, contribute to the development of schizophrenic disease.

Population receptive fields in non-human primates from whole-brain fMRI and large-scale neurophysiology in visual cortex

Population receptive field (pRF) modeling is a popular fMRI method to map the retinotopic organization of the human brain. While fMRI-based pRF maps are qualitatively similar to invasively recorded single-cell receptive fields in animals, it remains unclear what neuronal signal they represent. We addressed this question in awake nonhuman primates comparing whole-brain fMRI and large-scale neurophysiological recordings in areas V1 and V4 of the visual cortex. We examined the fits of several pRF models based on the fMRI blood-oxygen-level-dependent (BOLD) signal, multi-unit spiking activity (MUA), and local field potential (LFP) power in different frequency bands. We found that pRFs derived from BOLD-fMRI were most similar to MUA-pRFs in V1 and V4, while pRFs based on LFP gamma power also gave a good approximation. fMRI-based pRFs thus reliably reflect neuronal receptive field properties in the primate brain. In addition to our results in V1 and V4, the whole-brain fMRI measurements revealed retinotopic tuning in many other cortical and subcortical areas with a consistent increase in pRF size with increasing eccentricity, as well as a retinotopically specific deactivation of default mode network nodes similar to previous observations in humans.

The Missing Link in Early Emotional Processing

Initial evaluation structures (IESs) currently proposed as the earliest detectors of affective stimuli (e.g., amygdala, orbitofrontal cortex, or insula) are high-order structures (a) whose response latency cannot account for the first visual cortex emotion-related response (~80 ms), and (b) lack the necessary infrastructure to locally analyze the visual features that define emotional stimuli. Several thalamic structures accomplish both criteria. The lateral geniculate nucleus (LGN), a first-order thalamic nucleus that actively processes visual information, with the complement of the thalamic reticular nucleus (TRN) are proposed as core IESs. This LGN–TRN tandem could be supported by the pulvinar, a second-order thalamic structure, and by other extrathalamic nuclei. The visual thalamus, scarcely explored in affective neurosciences, seems crucial in early emotional evaluation.

Zooming-in on higher-level vision: High-resolution fMRI for understanding visual perception and awareness

One of the central questions in visual neuroscience is how the sparse retinal signals leaving our eyes are transformed into a rich subjective visual experience of the world. Invasive physiology studies, which offers the highest spatial resolution, have revealed many facts about the processing of simple visual features like contrast, color, and orientation, focusing on the early visual areas. At the same time, standard human fMRI studies with comparably coarser spatial resolution have revealed more complex, functionally specialized, and category-selective responses in higher visual areas. Although the visual system is the best understood among the sensory modalities, these two areas of research remain largely segregated. High-resolution fMRI opens up a possibility for linking them. On the one hand, it allows studying how the higher-level visual functions affect the fine-scale activity in early visual areas. On the other hand, it allows discovering the fine-scale functional organization of higher visual areas and exploring their functional connectivity with visual areas lower in the hierarchy. In this review, I will discuss examples of successful work undertaken in these directions using high-resolution fMRI and discuss where this method could be applied in the future to advance our understanding of the complexity of higher-level visual processing.

A source for awareness-dependent figure-ground segregation in human prefrontal cortex

Figure-ground modulation, i.e., the enhancement of neuronal responses evoked by the figure relative to the background, has three complementary components: edge modulation (boundary detection), center modulation (region filling), and background modulation (background suppression). However, the neuronal mechanisms mediating these three modulations and how they depend on awareness remain unclear. For each modulation, we compared both the cueing effect produced in a Posner paradigm and fMRI blood oxygen-level dependent (BOLD) signal in primary visual cortex (V1) evoked by visible relative to invisible orientation-defined figures. We found that edge modulation was independent of awareness, whereas both center and background modulations were strongly modulated by awareness, with greater modulations in the visible than the invisible condition. Effective-connectivity analysis further showed that the awareness-dependent region-filling and background-suppression processes in V1 were not derived through intracortical interactions within V1, but rather by feedback from the frontal eye field (FEF) and dorsolateral prefrontal cortex (DLPFC), respectively. These results indicate a source for an awareness-dependent figure-ground segregation in human prefrontal cortex.

Role of Feedback Connections in Central Visual Processing

The physiological response properties of neurons in the visual system are inherited mainly from feedforward inputs. Interestingly, feedback inputs often outnumber feedforward inputs. Although they are numerous, feedback connections are weaker, slower, and considered to be modulatory, in contrast to fast, high-efficacy feedforward connections. Accordingly, the functional role of feedback in visual processing has remained a fundamental mystery in vision science. At the core of this mystery are questions about whether feedback circuits regulate spatial receptive field properties versus temporal responses among target neurons, or whether feedback serves a more global role in arousal or attention. These proposed functions are not mutually exclusive, and there is compelling evidence to support multiple functional roles for feedback. In this review, the role of feedback in vision will be explored mainly from the perspective of corticothalamic feedback. Further generalized principles of feedback applicable to corticocortical connections will also be considered.

Resolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling

The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N = 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENT A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.

Neuroscience: Figured Out by Feedback to the Thalamus

The lateral geniculate nucleus of the thalamus (LGN) is a relay nucleus between the retina and the visual cortex. A new brain imaging study shows that LGN activity is modulated by figure-ground organization, even when the figure and ground are presented to different eyes: a hallmark of a cortical feedback effect.