Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex

Signal switching may enhance processing power of the brain

Our ability to perceive multiple objects is mysterious. Sensory neurons are broadly tuned, producing potential overlap in the populations of neurons activated by each object in a scene. This overlap raises questions about how distinct information is retained about each item. We present a novel signal switching theory of neural representation, which posits that neural signals may interleave representations of individual items across time. Evidence for this theory comes from new statistical tools that overcome the limitations inherent to standard time-and-trial-pooled assessments of neural signals. Our theory has implications for diverse domains of neuroscience, including attention, figure binding/scene segregation, oscillations, and divisive normalization. The general concept of switching between functions could also lend explanatory power to theories of grounded cognition.

Training-induced changes in population receptive field properties in visual cortex: Impact of eccentric vision training on population receptive field properties and the crowding effect

This study aimed to investigate the impact of eccentric-vision training on population receptive field (pRF) estimates to provide insights into brain plasticity processes driven by practice. Fifteen participants underwent functional magnetic resonance imaging (fMRI) measurements before and after behavioral training on a visual crowding task, where the relative orientation of the opening (gap position: up/down, left/right) in a Landolt C optotype had to be discriminated in the presence of flanking ring stimuli. Drifting checkerboard bar stimuli were used for pRF size estimation in multiple regions of interest (ROIs): dorsal-V1 (dV1), dorsal-V2 (dV2), ventral-V1 (vV1), and ventral-V2 (vV2), including the visual cortex region corresponding to the trained retinal location. pRF estimates in V1 and V2 were obtained along eccentricities from 0.5° to 9°. Statistical analyses revealed a significant decrease of the crowding anisotropy index (p = 0.009) after training, indicating improvement on crowding task performance following training. Notably, pRF sizes at and near the trained location decreased significantly (p = 0.005). Dorsal and ventral V2 exhibited significant pRF size reductions, especially at eccentricities where the training stimuli were presented (p < 0.001). In contrast, no significant changes in pRF estimates were found in either vV1 (p = 0.181) or dV1 (p = 0.055) voxels. These findings suggest that practice on a crowding task can lead to a reduction of pRF sizes in trained visual cortex, particularly in V2, highlighting the plasticity and adaptability of the adult visual system induced by prolonged training.

Convolutional neural networks develop major organizational principles of early visual cortex when enhanced with retinal sampling

Primate visual cortex exhibits key organizational principles: cortical magnification, eccentricity-dependent receptive field size and spatial frequency tuning as well as radial bias. We provide compelling evidence that these principles arise from the interplay of the non-uniform distribution of retinal ganglion cells, and a quasi-uniform convergence rate from the retina to the cortex. We show that convolutional neural networks outfitted with a retinal sampling layer, which resamples images according to retinal ganglion cell density, develop these organizational principles. Surprisingly, our results indicate that radial bias is spatial-frequency dependent and only manifests for high spatial frequencies. For low spatial frequencies, the bias shifts towards orthogonal orientations. These findings introduce a novel hypothesis about the origin of radial bias. Quasi-uniform convergence limits the range of spatial frequencies (in retinal space) that can be resolved, while retinal sampling determines the spatial frequency content throughout the retina.

Feedback scales the spatial tuning of cortical responses during visual memory

Perception, working memory, and long-term memory each evoke neural responses in visual cortex, suggesting that memory uses encoding mechanisms shared with perception. While previous research has largely focused on how perception and memory are similar, we hypothesized that responses in visual cortex would differ depending on the origins of the inputs. Using fMRI, we quantified spatial tuning in visual cortex while participants (both sexes) viewed, maintained in working memory, or retrieved from long-term memory a peripheral target. In each of these conditions, BOLD responses were spatially tuned and were aligned with the target's polar angle in all measured visual field maps including V1. As expected given the increasing sizes of receptive fields, polar angle tuning during perception increased in width systematically up the visual hierarchy from V1 to V2, V3, hV4, and beyond. In stark contrast, the widths of tuned responses were broad across the visual hierarchy during working memory and long-term memory, matched to the widths in perception in later visual field maps but much broader in V1. This pattern is consistent with the idea that mnemonic responses in V1 stem from top-down sources. Moreover, these tuned responses when biased (clockwise or counterclockwise of target) predicted matched biases in memory, suggesting that the readout of maintained and reinstated mnemonic responses influences memory guided behavior. We conclude that feedback constrains spatial tuning during memory, where earlier visual maps inherit broader tuning from later maps thereby impacting the precision of memory.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Primate V2 Receptive Fields Derived from Anatomically Identified Large-Scale V1 Inputs

In the primate visual system, visual object recognition involves a series of cortical areas arranged hierarchically along the ventral visual pathway. As information flows through this hierarchy, neurons become progressively tuned to more complex image features. The circuit mechanisms and computations underlying the increasing complexity of these receptive fields (RFs) remain unidentified. To understand how this complexity emerges in the secondary visual area (V2), we investigated the functional organization of inputs from the primary visual cortex (V1) to V2 by combining retrograde anatomical tracing of these inputs with functional imaging of feature maps in macaque monkey V1 and V2. We found that V1 neurons sending inputs to single V2 orientation columns have a broad range of preferred orientations, but are strongly biased towards the orientation represented at the injected V2 site. For each V2 site, we then constructed a feedforward model based on the linear combination of its anatomically-identified large-scale V1 inputs, and studied the response proprieties of the generated V2 RFs. We found that V2 RFs derived from the linear feedforward model were either elongated versions of V1 filters or had spatially complex structures. These modeled RFs predicted V2 neuron responses to oriented grating stimuli with high accuracy. Remarkably, this simple model also explained the greater selectivity to naturalistic textures of V2 cells compared to their V1 input cells. Our results demonstrate that simple linear combinations of feedforward inputs can account for the orientation selectivity and texture sensitivity of V2 RFs.

Distributed representations of behaviorally-relevant object dimensions in the human visual system

Object vision is commonly thought to involve a hierarchy of brain regions processing increasingly complex image features, with high-level visual cortex supporting object recognition and categorization. However, object vision supports diverse behavioral goals, suggesting basic limitations of this category-centric framework. To address these limitations, we mapped a series of behaviorally-relevant dimensions derived from a large-scale analysis of human similarity judgments directly onto the brain. Our results reveal broadly distributed representations of behaviorally-relevant information, demonstrating selectivity to a wide variety of novel dimensions while capturing known selectivities for visual features and categories. Behaviorally-relevant dimensions were superior to categories at predicting brain responses, yielding mixed selectivity in much of visual cortex and sparse selectivity in category-selective clusters. This framework reconciles seemingly disparate findings regarding regional specialization, explaining category selectivity as a special case of sparse response profiles among representational dimensions, suggesting a more expansive view on visual processing in the human brain.

Multiple objects evoke fluctuating responses in several regions of the visual pathway

How neural representations preserve information about multiple stimuli is mysterious. Because tuning of individual neurons is coarse (e.g., visual receptive field diameters can exceed perceptual resolution), the populations of neurons potentially responsive to each individual stimulus can overlap, raising the question of how information about each item might be segregated and preserved in the population. We recently reported evidence for a potential solution to this problem: when two stimuli were present, some neurons in the macaque visual cortical areas V1 and V4 exhibited fluctuating firing patterns, as if they responded to only one individual stimulus at a time (Jun et al., 2022). However, whether such an information encoding strategy is ubiquitous in the visual pathway and thus could constitute a general phenomenon remains unknown. Here, we provide new evidence that such fluctuating activity is also evoked by multiple stimuli in visual areas responsible for processing visual motion (middle temporal visual area, MT), and faces (middle fundus and anterolateral face patches in inferotemporal cortex - areas MF and AL), thus extending the scope of circumstances in which fluctuating activity is observed. Furthermore, consistent with our previous results in the early visual area V1, MT exhibits fluctuations between the representations of two stimuli when these form distinguishable objects but not when they fuse into one perceived object, suggesting that fluctuating activity patterns may underlie visual object formation. Taken together, these findings point toward an updated model of how the brain preserves sensory information about multiple stimuli for subsequent processing and behavioral action.

Crowding expands and is less sensitive to target-flanker differences during a shift of visual attention

Target-flanker similarity and critical spacing control visual crowding when attention is pre-allocated, but these have not been studied when attention shifts. Flanked target Gabors appeared 8° left and right of central fixation throughout each 1.5 s trial. Subjects reported target Gabor tilt. In Expt. 1, target blinks increased accuracy, and flanker blinks decreased it, but only when attention shifted left or right from a central RSVP cue, hardly before it, indicating an exogenous/endogenous synergy. Whether parallel or orthogonal, flankers of the same wavelength as the target crowded substantially. Parallel half-wavelength flankers also crowded, but orthogonal half-wavelength ones did not. In Expt. 2, crowding when attention shifts was the same for targets and flankers within Bouma's bound (2.5° apart) as outside it (5.0° apart.) In Expt. 3, Bouma's bound was restored when attention was focused continuously on the target. We conclude that crowding temporarily expands and becomes less discriminative when attention shifts.

Brain-inspired neural circuit evolution for spiking neural networks

In biological neural systems, different neurons are capable of self-organizing to form different neural circuits for achieving a variety of cognitive functions. However, the current design paradigm of spiking neural networks is based on structures derived from deep learning. Such structures are dominated by feedforward connections without taking into account different types of neurons, which significantly prevent spiking neural networks from realizing their potential on complex tasks. It remains an open challenge to apply the rich dynamical properties of biological neural circuits to model the structure of current spiking neural networks. This paper provides a more biologically plausible evolutionary space by combining feedforward and feedback connections with excitatory and inhibitory neurons. We exploit the local spiking behavior of neurons to adaptively evolve neural circuits such as forward excitation, forward inhibition, feedback inhibition, and lateral inhibition by the local law of spike-timing-dependent plasticity and update the synaptic weights in combination with the global error signals. By using the evolved neural circuits, we construct spiking neural networks for image classification and reinforcement learning tasks. Using the brain-inspired Neural circuit Evolution strategy (NeuEvo) with rich neural circuit types, the evolved spiking neural network greatly enhances capability on perception and reinforcement learning tasks. NeuEvo achieves state-of-the-art performance on CIFAR10, DVS-CIFAR10, DVS-Gesture, and N-Caltech101 datasets and achieves advanced performance on ImageNet. Combined with on-policy and off-policy deep reinforcement learning algorithms, it achieves comparable performance with artificial neural networks. The evolved spiking neural circuits lay the foundation for the evolution of complex networks with functions.

Cytoarchitectonic, receptor distribution and functional connectivity analyses of the macaque frontal lobe

Based on quantitative cyto- and receptor architectonic analyses, we identified 35 prefrontal areas, including novel subdivisions of Walker's areas 10, 9, 8B, and 46. Statistical analysis of receptor densities revealed regional differences in lateral and ventrolateral prefrontal cortex. Indeed, structural and functional organization of subdivisions encompassing areas 46 and 12 demonstrated significant differences in the interareal levels of α2 receptors. Furthermore, multivariate analysis included receptor fingerprints of previously identified 16 motor areas in the same macaque brains and revealed 5 clusters encompassing frontal lobe areas. We used the MRI datasets from the non-human primate data sharing consortium PRIME-DE to perform functional connectivity analyses using the resulting frontal maps as seed regions. In general, rostrally located frontal areas were characterized by bigger fingerprints, that is, higher receptor densities, and stronger regional interconnections. Whereas more caudal areas had smaller fingerprints, but showed a widespread connectivity pattern with distant cortical regions. Taken together, this study provides a comprehensive insight into the molecular structure underlying the functional organization of the cortex and, thus, reconcile the discrepancies between the structural and functional hierarchical organization of the primate frontal lobe. Finally, our data are publicly available via the EBRAINS and BALSA repositories for the entire scientific community.

Multiple objects evoke fluctuating responses in several regions of the visual pathway

How neural representations preserve information about multiple stimuli is mysterious. Because tuning of individual neurons is coarse (for example, visual receptive field diameters can exceed perceptual resolution), the populations of neurons potentially responsive to each individual stimulus can overlap, raising the question of how information about each item might be segregated and preserved in the population. We recently reported evidence for a potential solution to this problem: when two stimuli were present, some neurons in the macaque visual cortical areas V1 and V4 exhibited fluctuating firing patterns, as if they responded to only one individual stimulus at a time. However, whether such an information encoding strategy is ubiquitous in the visual pathway and thus could constitute a general phenomenon remains unknown. Here we provide new evidence that such fluctuating activity is also evoked by multiple stimuli in visual areas responsible for processing visual motion (middle temporal visual area, MT), and faces (middle fundus and anterolateral face patches in inferotemporal cortex - areas MF and AL), thus extending the scope of circumstances in which fluctuating activity is observed. Furthermore, consistent with our previous results in the early visual area V1, MT exhibits fluctuations between the representations of two stimuli when these form distinguishable objects but not when they fuse into one perceived object, suggesting that fluctuating activity patterns may underlie visual object formation. Taken together, these findings point toward an updated model of how the brain preserves sensory information about multiple stimuli for subsequent processing and behavioral action.

Impact Statement

We find neural fluctuations in multiple areas along the visual cortical hierarchy that could allow the brain to represent distinct co-occurring visual stimuli.

Effects of cortical distance on the Ebbinghaus and Delboeuf illusions

The Ebbinghaus and Delboeuf illusions affect the perceived size of a target circle depending on the size and proximity of circular inducers or a ring. Converging evidence suggests that these illusions are driven by interactions between contours mediated by their cortical distance in primary visual cortex. We tested the effect of cortical distance on these illusions using two methods: First, we manipulated retinal distance between target and inducers in a two-interval forced choice design, finding that targets appeared larger with a closer surround. Next, we predicted that targets presented peripherally should appear larger due to cortical magnification. Hence, we tested the illusion strength when positioning the stimuli at various eccentricities, with results supporting this hypothesis. We calculated estimated cortical distances between illusion elements in each experiment and used these estimates to compare the relationship between cortical distance and illusion strength across our experiments. In a final experiment, we modified the Delboeuf illusion to test whether the influence of the inducers/annuli in this illusion is influenced by an inhibitory surround. We found evidence that an additional outer ring makes targets appear smaller compared to a single-ring condition, suggesting that near and distal contours have antagonistic effects on perceived target size.

Could theropod dinosaurs have evolved to a human level of intelligence?

Noting that some theropod dinosaurs had large brains, large grasping hands, and likely binocular vision, paleontologist Dale Russell suggested that a branch of these dinosaurs might have evolved to a human intelligence level, had dinosaurs not become extinct. I offer reasons why the likely pallial organization in dinosaurs would have made this improbable, based on four assumptions. First, it is assumed that achieving human intelligence requires evolving an equivalent of the about 200 functionally specialized cortical areas characteristic of humans. Second, it is assumed that dinosaurs had an avian nuclear type of pallial organization, in contrast to the mammalian cortical organization. Third, it is assumed that the interactions between the different neuron types making up an information processing unit within pallium are critical to its role in analyzing information. Finally, it is assumed that increasing axonal length between the neuron sets carrying out this operation impairs its efficacy. Based on these assumptions, I present two main reasons why dinosaur pallium might have been unable to add the equivalent of 200 efficiently functioning cortical areas. First, a nuclear pattern of pallial organization would require increasing distances between the neuron groups corresponding to the separate layers of any given mammalian cortical area, as more sets of nuclei equivalent to a cortical area are interposed between the existing sets, increasing axon length and thereby impairing processing efficiency. Second, because of its nuclear organization, dinosaur pallium could not reduce axon length by folding to bring adjacent areas closer together, as occurs in cerebral cortex.

Decoding self-motion from visual image sequence predicts distinctive features of reflexive motor responses to visual motion

Visual motion analysis is crucial for humans to detect external moving objects and self-motion which are informative for planning and executing actions for various interactions with environments. Here we show that the image motion analysis trained to decode the self-motion during human natural movements by a convolutional neural network exhibits similar specificities with the reflexive ocular and manual responses induced by a large-field visual motion, in terms of stimulus spatiotemporal frequency tuning. The spatiotemporal frequency tuning of the decoder peaked at high-temporal and low-spatial frequencies, as observed in the reflexive ocular and manual responses, but differed significantly from the frequency power of the visual image itself and the density distribution of self-motion. Further, artificial manipulations of the learning data sets predicted great changes in the specificity of the spatiotemporal tuning. Interestingly, despite similar spatiotemporal frequency tunings in the vertical-axis rotational direction and in the transversal direction to full-field visual stimuli, the tunings for center-masked stimuli were different between those directions, and the specificity difference is qualitatively similar to the discrepancy between ocular and manual responses, respectively. In addition, the representational analysis demonstrated that head-axis rotation was decoded by relatively simple spatial accumulation over the visual field, while the transversal motion was decoded by more complex spatial interaction of visual information. These synthetic model examinations support the idea that visual motion analyses eliciting the reflexive motor responses, which are critical in interacting with the external world, are acquired for decoding self-motion.

Fast discrimination of fragmentary images: the role of local optimal information

In naturalistic conditions, objects in the scene may be partly occluded and the visual system has to recognize the whole image based on the little information contained in some visible fragments. Previous studies demonstrated that humans can successfully recognize severely occluded images, but the underlying mechanisms occurring in the early stages of visual processing are still poorly understood. The main objective of this work is to investigate the contribution of local information contained in a few visible fragments to image discrimination in fast vision. It has been already shown that a specific set of features, predicted by a constrained maximum-entropy model to be optimal carriers of information (optimal features), are used to build simplified early visual representations (primal sketch) that are sufficient for fast image discrimination. These features are also considered salient by the visual system and can guide visual attention when presented isolated in artificial stimuli. Here, we explore whether these local features also play a significant role in more natural settings, where all existing features are kept, but the overall available information is drastically reduced. Indeed, the task requires discrimination of naturalistic images based on a very brief presentation (25 ms) of a few small visible image fragments. In the main experiment, we reduced the possibility to perform the task based on global-luminance positional cues by presenting randomly inverted-contrast images, and we measured how much observers' performance relies on the local features contained in the fragments or on global information. The size and the number of fragments were determined in two preliminary experiments. Results show that observers are very skilled in fast image discrimination, even when a drastic occlusion is applied. When observers cannot rely on the position of global-luminance information, the probability of correct discrimination increases when the visible fragments contain a high number of optimal features. These results suggest that such optimal local information contributes to the successful reconstruction of naturalistic images even in challenging conditions.

Underestimation of the number of hidden objects

The perceptual representation of our environment does not only involve what we actually can see, but also inferences about what is hidden from our sight. For example, in amodal completion, simple contours or surfaces are filled-in behind occluding objects allowing for a complete representation. This is important for many everyday tasks, such as visual search, foraging, and object handling. Although there is support for completion of simple patterns from behavioral and neurophysiological studies, it is unclear if these mechanisms extend to complex, irregular patterns. Here, we show that the number of hidden objects on partially occluded surfaces is underestimated. Observers did not consider accurately the number of visible objects and the proportion of occlusion to infer the number of hidden objects, although these quantities were perceived accurately and reliably. However, visible objects were not simply ignored: estimations of hidden objects increased when the visible objects formed a line across the occluder and decreased when the visible objects formed a line outside of the occluder. Confidence ratings for numerosity estimation were similar for fully visible and partially occluded surfaces. These results suggest that perceptual inferences about what is hidden in our environment can be very inaccurate und underestimate the complexity of the environment.

Signal denoising through topographic modularity of neural circuits

Information from the sensory periphery is conveyed to the cortex via structured projection pathways that spatially segregate stimulus features, providing a robust and efficient encoding strategy. Beyond sensory encoding, this prominent anatomical feature extends throughout the neocortex. However, the extent to which it influences cortical processing is unclear. In this study, we combine cortical circuit modeling with network theory to demonstrate that the sharpness of topographic projections acts as a bifurcation parameter, controlling the macroscopic dynamics and representational precision across a modular network. By shifting the balance of excitation and inhibition, topographic modularity gradually increases task performance and improves the signal-to-noise ratio across the system. We demonstrate that in biologically constrained networks, such a denoising behavior is contingent on recurrent inhibition. We show that this is a robust and generic structural feature that enables a broad range of behaviorally relevant operating regimes, and provide an in-depth theoretical analysis unraveling the dynamical principles underlying the mechanism.

Attention along the cortical hierarchy: Development matters

We build on the existing biased competition view to argue that attention is an emergent property of neural computations within and across hierarchically embedded and structurally connected cortical pathways. Critically then, one must ask, what is attention emergent from? Within this framework, developmental changes in the quality of sensory input and feedforward-feedback information flow shape the emergence and efficiency of attention. Several gradients of developing structural and functional cortical architecture across the caudal-to-rostral axis provide the substrate for attention to emerge. Neural activity within visual areas depends on neuronal density, receptive field size, tuning properties of neurons, and the location of and competition between features and objects in the visual field. These visual cortical properties highlight the information processing bottleneck attention needs to resolve. Recurrent feedforward and feedback connections convey sensory information through a series of steps at each level of the cortical hierarchy, integrating sensory information across the entire extent of the cortical hierarchy and linking sensory processing to higher-order brain regions. Higher-order regions concurrently provide input conveying behavioral context and goals. Thus, attention reflects the output of a series of complex biased competition neural computations that occur within and across hierarchically embedded cortical regions. Cortical development proceeds along the caudal-to-rostral axis, mirroring the flow in sensory information from caudal to rostral regions, and visual processing continues to develop into childhood. Examining both typical and atypical development will offer critical mechanistic insight not otherwise available in the adult stable state. This article is categorized under: Psychology > Attention.

Emergence of radial orientation selectivity: Effect of cell density changes and eccentricity in a layered network

We establish a simple mechanism by which radially oriented simple cells can emerge in the primary visual cortex. In 1986, R. Linsker. proposed a means by which radially symmetric, spatial opponent cells can evolve, driven entirely by noise, from structure in the initial synaptic connectivity distribution. We provide an analytical derivation of Linsker's results, and further show that radial eigenfunctions can be expressed as a weighted sum of degenerate Cartesian eigenfunctions, and vice-versa. These results are extended to allow for radially dependent cell density, from which we show that, despite a circularly symmetric synaptic connectivity distribution, radially biased orientation selectivity emerges in the third layer when cell density in the first layer, or equivalently, synaptic radius, changes with eccentricity; i.e., distance to the center of the lamina. This provides a potential mechanism for the emergence of radial orientation in the primary visual cortex before eye opening and the onset of structured visual input after birth.

Motion adaptation improves acuity (but perceived size doesn't matter)

Recognition acuity—the minimum size of a high-contrast object that allows us to recognize it—is limited by optical and neural elements of the eye and by processing within the visual cortex. The perceived size of objects can be changed by motion-adaptation. Viewing receding or looming motion makes subsequently viewed stimuli appear to grow or shrink, respectively. It has been reported that resulting changes in perceived size impact recognition acuity. We set out to determine if such acuity changes are reliable and what drives this phenomenon. We measured the effect of adaptation to receding and looming motion on acuity for crowded tumbling-T stimuli (). We quantified the role of crowding, individuals’ susceptibility to motion-adaptation, and potentially confounding effects of pupil size and eye movements. Adaptation to receding motion made targets appear larger and improved acuity (–0.037 logMAR). Although adaptation to looming motion made targets appear smaller, it induced not the expected decrease in acuity but a modest acuity improvement (–0.018 logMAR). Further, each observer's magnitude of acuity change was not correlated with their individual perceived-size change following adaptation. Finally, we found no evidence that adaptation-induced acuity gains were related to crowding, fixation stability, or pupil size. Adaptation to motion modestly enhances visual acuity, but unintuitively, this is dissociated from perceived size. Ruling out fixation and pupillary behavior, we suggest that motion adaptation may improve acuity via incidental effects on sensitivity—akin to those arising from blur adaptation—which shift sensitivity to higher spatial frequency-tuned channels.

Neural correlates of lateral modulation and perceptual filling-in in center-surround radial sinusoidal gratings: an fMRI study

We investigated lateral modulation effects with functional magnetic resonance imaging. We presented radial sinusoidal gratings in random sequence: a scotoma grating with two arc-shaped blank regions (scotomata) in the periphery, one in the left and one in the right visual field, a center grating containing pattern only in the scotoma regions, and a full-field grating where the pattern occupied the whole screen. On each trial, one of the three gratings flickered in counterphase for 10 s, followed by a blank period. Observers were instructed to perform a fixation task and report whether filling-in was experienced during the scotoma condition. The results showed that the blood-oxygen-level-dependent signal was reduced in areas corresponding to the scotoma regions in the full-field compared to the center condition in V1 to V3 areas, indicating a lateral inhibition effect when the surround was added to the center pattern. The univariate analysis results showed no difference between the filling-in and no-filling-in trials. However, multivariate pattern analysis results showed that classifiers trained on activation pattern in V1 to V3 could differentiate between filling-in and no-filling-in trials, suggesting that the neural activation pattern in visual cortex correlated with the subjective percept.

Psychophysical Reverse Correlation Revealed Broader Orientation Tuning and Prolonged Reaction Time in Amblyopia

Purpose

Neural selectivity of orientation is a fundamental property of visual system. We aim to investigate whether and how the orientation selectivity changes in amblyopia.

Methods

Seventeen patients with amblyopia (27.1 ± 7.1 years) and 18 healthy participants (25.1 ± 2.7 years) took part in this study. They were asked to continuously detect vertical gratings embedded in a stream of randomly oriented gratings. Using a technique of subspace reverse correlation, the orientation-time perceptive field (PF) for the atypical grating detection task was derived for each participant. Detailed comparisons were made between the PFs measured with the amblyopic and healthy eyes.

Results

The PF of the amblyopic eyes showed significant differences in orientation and time domain compared with that of the normal eyes (cluster-based permutation test, ps < 0.05), with broader bandwidth of orientation tuning (31.41 ± 10.59 degrees [mean ± SD] vs. 24.76 ± 6.85 degrees, P = 0.039) and delayed temporal dynamics (483 ± 68 ms vs. 425 ± 58 ms, P = 0.015). None of the altered PF properties correlated with the contrast sensitivity at 1 cycle per degree (c/deg) in amblyopia. No difference in PFs between the dominant and non-dominant eyes in the healthy group was found.

Conclusions

The altered orientation-time PF to the low spatial frequency and high contrast stimuli suggests amblyopes had coarser orientation selectivity and prolonged reaction time. The broader orientation tuning probably reflects the abnormal lateral interaction in the primary visual cortex, whereas the temporal delay might indicate a high level deficit.

Saturating Nonlinearities of Contrast Response in Human Visual Cortex

Response nonlinearities are ubiquitous throughout the brain, especially within sensory cortices where changes in stimulus intensity typically produce compressed responses. Although this relationship is well established in electrophysiological measurements, it remains controversial whether the same nonlinearities hold for population-based measurements obtained with human fMRI. We propose that these purported disparities are not contingent on measurement type and are instead largely dependent on the visual system state at the time of interrogation. We show that deploying a contrast adaptation paradigm permits reliable measurements of saturating sigmoidal contrast response functions (10 participants, 7 female). When not controlling the adaptation state, our results coincide with previous fMRI studies, yielding nonsaturating, largely linear contrast responses. These findings highlight the important role of adaptation in manifesting measurable nonlinear responses within human visual cortex, reconciling discrepancies reported in vision neuroscience, re-establishing the qualitative relationship between stimulus intensity and response across different neural measures and the concerted study of cortical gain control.SIGNIFICANCE STATEMENT Nonlinear stimulus-response relationships govern many essential brain functions, ranging from the sensory to cognitive level. Certain core response properties previously shown to be nonlinear with nonhuman electrophysiology recordings have yet to be reliably measured with human neuroimaging, prompting uncertainty and reconsideration. The results of this study stand to reconcile these incongruencies in the vision neurosciences, demonstrating the profound impact adaptation can have on brain activation throughout the early visual cortex. Moving forward, these findings facilitate the study of modulatory influences on sensory processing (i.e., arousal and attention) and help establish a closer link between neural recordings in animals and hemodynamic measurements from human fMRI, resuming a concerted effort to understand operations in the mammalian cortex.

Analytic Model for Feature Maps in the Primary Visual Cortex

A compact analytic model is proposed to describe the combined orientation preference (OP) and ocular dominance (OD) features of simple cells and their mutual constraints on the spatial layout of the combined OP-OD map in the primary visual cortex (V1). This model consists of three parts: (i) an anisotropic Laplacian (AL) operator that represents the local neural sensitivity to the orientation of visual inputs; and (ii) obtain a receptive field (RF) operator that models the anisotropic spatial projection from nearby neurons to a given V1 cell over scales of a few tenths of a millimeter and combines with the AL operator to give an overall OP operator; and (iii) a map that describes how the parameters of these operators vary approximately periodically across V1. The parameters of the proposed model maximize the neural response at a given OP with an OP tuning curve fitted to experimental results. It is found that the anisotropy of the AL operator does not significantly affect OP selectivity, which is dominated by the RF anisotropy, consistent with Hubel and Wiesel's original conclusions that orientation tuning width of V1 simple cell is inversely related to the elongation of its RF. A simplified and idealized OP-OD map is then constructed to describe the approximately periodic local OP-OD structure of V1 in a compact form. It is shown explicitly that the OP map can be approximated by retaining its dominant spatial Fourier coefficients, which are shown to suffice to reconstruct its basic spatial structure. Moreover, this representation is a suitable form to analyze observed OP maps compactly and to be used in neural field theory (NFT) for analyzing activity modulated by the OP-OD structure of V1. Application to independently simulated V1 OP structure shows that observed irregularities in the map correspond to a spread of dominant coefficients in a circle in Fourier space. In addition, there is a strong bias toward two perpendicular directions when only a small patch of local map is included. The bias is decreased as the amount of V1 included in the Fourier transform is increased.

The Global Configuration of Visual Stimuli Alters Co-Fluctuations of Cross-Hemispheric Human Brain Activity

We tested how a stimulus gestalt, defined by the neuronal interaction between local and global features of a stimulus, is represented within human primary visual cortex (V1). We used high-resolution fMRI, which serves as a surrogate of neuronal activation, to measure co-fluctuations within subregions of V1 as (male and female) subjects were presented with peripheral stimuli, each with different global configurations. We found stronger cross-hemisphere correlations when fine-scale V1 cortical subregions represented parts of the same object compared with different objects. This result was consistent with the vertical bias in global processing and, critically, was independent of the task and local discontinuities within objects. Thus, despite the relatively small receptive fields of neurons within V1, global stimulus configuration affects neuronal processing via correlated fluctuations between regions that represent different sectors of the visual field.SIGNIFICANCE STATEMENT We provide the first evidence for the impact of global stimulus configuration on cross-hemispheric fMRI fluctuations, measured in human primary visual cortex. Our results are consistent with changes in the level of γ-band synchrony, which has been shown to be affected by global stimulus configuration, being reflected in the level fMRI co-fluctuations. These data help narrow the gap between knowledge of global stimulus configuration encoding at the single-neuron level versus at the behavioral level.

Retinotopic organization of visual cortex in human infants

Vision develops rapidly during infancy, yet how visual cortex is organized during this period is unclear. In particular, it is unknown whether functional maps that organize the mature adult visual cortex are present in the infant striate and extrastriate cortex. Here, we test the functional maturity of infant visual cortex by performing retinotopic mapping with functional magnetic resonance imaging (fMRI). Infants aged 5-23 months had retinotopic maps, with alternating preferences for vertical and horizontal meridians indicating the boundaries of visual areas V1 to V4 and an orthogonal gradient of preferences from high to low spatial frequencies. The presence of multiple visual maps throughout visual cortex in infants indicates a greater maturity of extrastriate cortex than previously appreciated. The areas showed subtle age-related fine-tuning, suggesting that early maturation undergoes continued refinement. This early maturation of area boundaries and tuning may scaffold subsequent developmental changes.

A touch of hierarchy: population receptive fields reveal fingertip integration in Brodmann areas in human primary somatosensory cortex

Several neuroimaging studies have shown the somatotopy of body part representations in primary somatosensory cortex (S1), but the functional hierarchy of distinct subregions in human S1 has not been adequately addressed. The current study investigates the functional hierarchy of cyto-architectonically distinct regions, Brodmann areas BA3, BA1, and BA2, in human S1. During functional MRI experiments, we presented participants with vibrotactile stimulation of the fingertips at three different vibration frequencies. Using population Receptive Field (pRF) modeling of the fMRI BOLD activity, we identified the hand region in S1 and the somatotopy of the fingertips. For each voxel, the pRF center indicates the finger that most effectively drives the BOLD signal, and the pRF size measures the spatial somatic pooling of fingertips. We find a systematic relationship of pRF sizes from lower-order areas to higher-order areas. Specifically, we found that pRF sizes are smallest in BA3, increase slightly towards BA1, and are largest in BA2, paralleling the increase in visual receptive field size as one ascends the visual hierarchy. Additionally, we find that the time-to-peak of the hemodynamic response in BA3 is roughly 0.5 s earlier compared to BA1 and BA2, further supporting the notion of a functional hierarchy of subregions in S1. These results were obtained during stimulation of different mechanoreceptors, suggesting that different afferent fibers leading up to S1 feed into the same cortical hierarchy.

Behavioural and electrophysiological evidence for the effect of target-distractor separation in a tactile search task

Evidence suggests that the N140cc component of event-related potentials (ERP) observed in tactile search tasks reflects the attentional selection of the target. Here, we investigated whether the target selection processes are affected by the separation between the target and an ipsilateral singleton distractor (singletons delivered to contiguous or non-contiguous fingers of the same hand). In addition, the external distance between search items was varied through posture (splayed or touching fingers). Accuracy improved when target and distractor were delivered to contiguous fingers that were also touching. Regardless of target-distractor separation, the N140cc was larger when the external distance between search-array stimuli decreased (touching fingers). Importantly, a smaller N140cc was observed at reduced target-distractor separations, suggesting a narrower attentional focus for contiguous singletons. These findings reveal that the mechanisms responsible for tactile target selection in the presence of an ipsilateral singleton distractor are fundamentally different from those emerged in vision.

Active sensing and overt avoidance: Gaze shifts as a mechanism of predictive avoidance in vision

Sensory organs are not only involved in passively transmitting sensory input, but are also involved in actively seeking it. Some sensory organs move dynamically to allow highly prioritized input to be detected by their most sensitive parts. Such 'active sensing' systems engage in pursuing relevant input, relying on attentional prioritizations. However, pursuing input may not always be advantageous. Task-irrelevant input may be distracting and interfere with task performance. We hypothesize that an efficient 'active sensing' mechanism should be able to not only pursue relevant input but also to predict irrelevant input and avoid it. Moreover, we hypothesize that this mechanism should be evident even when the task is non-visual and all visual information acts as a distractor. In this study, we demonstrate the existence of a predictive 'overt avoidance' mechanism in vision. In two experiments, participants were asked to perform a continuous mental-arithmetic task while occasionally being presented with task-irrelevant crowded displays limited to one quadrant of a screen. The locations of these visual stimuli were constant within a block but varied between blocks. Results show that gaze was consistently shifted away from the predicted location of distraction, even prior to its appearance, confirming the existence of a predictive 'overt avoidance' mechanism in vision. Based on these findings, we propose a conceptual model to explain how an 'active sensing' system, hardwired to explore, can overcome this drive when presented with distracting information. According to the model, distraction is handled through a dual mechanism of suppression and avoidance processes that are causally linked. This framework demonstrates how perception and motion work together to approach relevant information while avoiding irrelevant distraction.

The time-course of prediction formation and revision in human visual motion processing

Establishing the real-time position of a moving object poses a challenge to the visual system due to neural processing delays. While sensory information is travelling through the visual hierarchy, the object continues moving and information about its position becomes outdated. By extrapolating the position of a moving object along its trajectory, predictive mechanisms might effectively decrease the processing time associated with these objects. Here, we use time-resolved decoding of electroencephalographic (EEG) data from an apparent motion paradigm to demonstrate the interaction of two separate predictive mechanisms. First, we reveal predictive latency advantages for position representations as soon as the second object in an apparent motion sequence - even before the stimulus contains any physical motion energy. This is consistent with the existence of omni-directional, within-layer waves of sub-threshold activity that bring neurons coding for adjacent positions closer to their firing threshold, thereby reducing the processing time of the second stimulus in one of those positions. Second, we show that an additional direction-specific latency advantage emerges from the third sequence position onward, once the direction of the apparent motion stimulus is uniquely determined. Because the receptive fields of early visual areas are too small to encompass sequential apparent motion positions (as evidenced by the lack of latency modulation for the second stimulus position), this latency advantage most likely arises from descending predictions from higher to lower visual areas through feedback connections. Finally, we reveal that the same predictive activation that facilitates the processing of the object in its expected position needs to be overcome when the object's trajectory unexpectedly reverses, causing an additional latency disadvantage for stimuli that violate predictions. Altogether, our results suggest that two complementary mechanisms interact to form and revise predictions in visual motion processing, modulating the latencies of neural position representations at different levels of visual processing.

Scotopic Vision Is Selectively Processed in Thick-Type Columns in Human Extrastriate Cortex

In humans, visual stimuli can be perceived across an enormous range of light levels. Evidence suggests that different neural mechanisms process different subdivisions of this range. For instance, in the retina, stimuli presented at very low (scotopic) light levels activate rod photoreceptors, whereas cone photoreceptors are activated relatively more at higher (photopic) light levels. Similarly, different retinal ganglion cells are activated by scotopic versus photopic stimuli. However, in the brain, it remains unknown whether scotopic versus photopic information is: 1) processed in distinct channels, or 2) neurally merged. Using high-resolution functional magnetic resonance imaging at 7 T, we confirmed the first hypothesis. We first localized thick versus thin-type columns within areas V2, V3, and V4, based on photopic selectivity to motion versus color, respectively. Next, we found that scotopic stimuli selectively activated thick- (compared to thin-) type columns in V2 and V3 (in measurements of both overlap and amplitude) and V4 (based on overlap). Finally, we found stronger resting-state functional connections between scotopically dominated area MT with thick- (compared to thin-) type columns in areas V2, V3, and V4. We conclude that scotopic stimuli are processed in partially segregated parallel streams, emphasizing magnocellular influence, from retina through middle stages of visual cortex.

Effect of Visual Field Location on Global Motion Perception: A Developmental Study

Previous work has shown that motion perception in school-age children is similar to that of adults for fast speeds but is immature at slow speeds for stimuli presented in the central visual field. This study examined whether visual field location affects this developmental pattern. We measured left/right and up/down global motion direction discrimination for fast and slow speeds in 7- to 10-year-old children and in adults with stimuli presented to upper, central, or lower visual fields. For left/right direction discrimination, children showed significantly higher (worse) coherence thresholds than adults for slow, but not fast, speeds in the central visual field. In the upper and lower visual fields, children showed significantly higher coherence thresholds than adults for both speeds. For up/down direction discrimination, children showed similar performance to adults for the central visual field. In the upper and lower visual fields, children performed significantly worse than adults; this finding was speed-tuned only for the lower visual field. Thus, children show immature global motion perception in the periphery even when performance in central vision is adult-like. These results enrich our understanding of motion perception development in children with typical vision.

Feature-specific neural reactivation during episodic memory

We present a multi-voxel analytical approach, feature-specific informational connectivity (FSIC), that leverages hierarchical representations from a neural network to decode neural reactivation in fMRI data collected while participants performed an episodic visual recall task. We show that neural reactivation associated with low-level (e.g. edges), high-level (e.g. facial features), and semantic (e.g. “terrier”) features occur throughout the dorsal and ventral visual streams and extend into the frontal cortex. Moreover, we show that reactivation of both low- and high-level features correlate with the vividness of the memory, whereas only reactivation of low-level features correlates with recognition accuracy when the lure and target images are semantically similar. In addition to demonstrating the utility of FSIC for mapping feature-specific reactivation, these findings resolve the contributions of low- and high-level features to the vividness of visual memories and challenge a strict interpretation the posterior-to-anterior visual hierarchy.

Memory recollection involves reactivation of neural activity that occurred during the recalled experience. Here, the authors show that neural reactivation can be decomposed into visual-semantic features, is widely synchronized throughout the brain, and predicts memory vividness and accuracy.

In vivo functional localization of the temporal monocular crescent representation in human primary visual cortex

The temporal monocular crescent (TMC) is the most peripheral portion of the visual field whose perception relies solely on input from the ipsilateral eye. According to a handful of post-mortem histological studies in humans and non-human primates, the TMC is represented visuotopically within the most anterior portion of the primary visual cortical area (V1). However, functional evidence of the TMC visuotopic representation in human visual cortex is rare, mostly due to the small size of the TMC representation (~6% of V1) and due to the technical challenges of stimulating the most peripheral portion of the visual field inside the MRI scanner. In this study, by taking advantage of custom-built MRI-compatible visual stimulation goggles with curved displays, we successfully stimulated the TMC region of the visual field in eight human subjects, half of them right-eye dominant, inside a 3 ​T MRI scanner. This enabled us to localize the representation of TMC, along with the blind spot representation (another visuotopic landmark in V1), in all volunteers, which match the expected spatial pattern based on prior anatomical studies. In all hemispheres, the TMC visuotopic representation was localized along the peripheral border of V1, within the most anterior portion of the calcarine sulcus, without any apparent extension into the second visual area (V2). We further demonstrate the reliability of this localization within/across experimental sessions, and consistency in the spatial location of TMC across individuals after accounting for inter-subject structural differences.

Tracking the completion of parts into whole objects: Retinotopic activation in response to illusory figures in the lateral occipital complex

Illusory figures demonstrate the visual system's ability to integrate separate parts into coherent, whole objects. The present study was performed to track the neuronal object construction process in human observers, by incrementally manipulating the grouping strength within a given configuration until the emergence of a whole-object representation. Two tasks were employed: First, in the spatial localization task, object completion could facilitate performance and was task-relevant, whereas it was irrelevant in the second, luminance discrimination task. Concurrent functional magnetic resonance imaging (fMRI) used spatial localizers to locate brain regions representing task-critical illusory-figure parts to investigate whether the step-wise object construction process would modulate neural activity in these localized brain regions. The results revealed that both V1 and the lateral occipital complex (LOC, with sub-regions LO1 and LO2) were involved in Kanizsa figure processing. However, completion-specific activations were found predominantly in LOC, where neural activity exhibited a modulation in accord with the configuration's grouping strength, whether or not the configuration was relevant to performing the task at hand. Moreover, right LOC activations were confined to LO2 and responded primarily to surface and shape completions, whereas left LOC exhibited activations in both LO1 and LO2 and was related to encoding shape structures with more detail. Together, these results demonstrate that various grouping properties within a visual scene are integrated automatically in LOC, with sub-regions located in different hemispheres specializing in the component sub-processes that render completed objects.

The perceptual continuity field is retinotopic

Visual perception is systematically biased towards input from the recent past: perceived orientation, numerosity, and face identity are pulled towards previously seen stimuli. To better understand the brain level at which serial dependence occurs, the present study examined its spatial tuning. In three experiments, serial dependence occurred between stimuli occupying the same retinal position. Serial dependence between stimuli at distant retinal locations was smaller, even when the stimuli occupied the same location in external space. The spatial window over which serial dependence occurs is thus retinotopic, but wide, suggesting that serial dependence occurs at late stages of visual processing.

Asymmetries in Global Perception Are Represented in Near- versus Far-Preferring Clusters in Human Visual Cortex

Human perception is more “global” when stimuli are viewed within the lower (rather than the upper) visual field. This phenomenon is typically considered as a 2-D phenomenon, likely due to differential neural processing within dorsal versus ventral cortical areas that represent lower versus upper visual fields, respectively. Here we test a novel hypothesis that this vertical asymmetry in global processing is a 3-D phenomenon associated with (1) higher ecological relevance of low-spatial frequency (SF) components in encoding near (compared with far) visual objects and (2) the fact that near objects are more frequently found in lower rather than upper visual fields. Using high-resolution fMRI, collected within an ultra-high-field (7 T) scanner, we found that the extent of vertical asymmetry in global visual processing in human subjects (n = 10) was correlated with the fMRI response evoked by disparity-varying stimuli in human cortical area V3A. We also found that near-preferring clusters in V3A, located within stereoselective cortical columns, responded more selectively than far-preferring clusters, to low-SF features. These findings support the hypothesis that vertical asymmetry in global processing is a 3-D (not a 2-D) phenomenon, associated with the function of the stereoselective columns within visual cortex, especially those located within visual area V3A.

SIGNIFICANCE STATEMENT Here we test and confirm a new hypothesis: fine-scale neural mechanisms underlying the vertical asymmetry in global visual processing. According to this hypothesis, the asymmetry in global visual processing is a 3-D (rather than a 2-D) phenomenon, reflected in the function of fine-scale cortical structures (clusters and columns) underlying depth perception. Our findings highlight the importance of considering these structures, as regions of interest, in clarifying the neural mechanisms underlying visual perception. The results also highlight the importance of statistics of natural scenes in shaping human visual perception.

Size Perception Biases Are Temporally Stable and Vary Consistently Between Visual Field Meridians

The apparent size of visual stimuli depends on where in the visual field they appear. We recently presented a model of how size perception could be biased by stimulus encoding in retinotopic cortex. However, it remains unclear if such perceptual biases are instead trivially related to discrimination ability and if they are temporally stable. An independent test of the model is also still outstanding. Here, I show that perceptual biases are stable across stimulus durations between 50 and 1,000 milliseconds, even though discrimination ability unsurprisingly improves with duration. Furthermore, perceptual biases are stronger along the vertical than the horizontal meridian, which mirrors reported differences in spatial vision and the positional selectivity of early visual cortex. Taken together, these findings support our model of how size is inferred from cortical responses.

Size Increase in Panum’s Fusional Range Is Driven by Eccentricity and Not Separation

Previous research has demonstrated that Panum’s fusional range increases in the periphery, and this increase is usually attributed to eccentricity. However, it is unclear whether the increase in the periphery is driven by eccentricity or separation between the stimulus and the central fixation marker. In Experiment 1, we independently measured the effects of eccentricity and stimulus separation on Panum’s fusional range for horizontal disparities. We observed significant increases in Panum’s range as eccentricity increased independently of stimulus separation. Experiment 2 revealed that the extent of Panum’s fusional range for vertical disparities increased with eccentricity independently of stimulus separation. Taken together, these results strongly support previously held conclusions that Panum’s fusional range for both horizontal and vertical disparities increases with increasing eccentricity and is little impacted by stimulus separation.

Attentional Modulation of Visual Spatial Integration: Psychophysical Evidence Supported by Population Coding Modeling

Two prominent strategies that the human visual system uses to reduce incoming information are spatial integration and selective attention. Whereas spatial integration summarizes and combines information over the visual field, selective attention can single it out for scrutiny. The way in which these well-known mechanisms-with rather opposing effects-interact remains largely unknown. To address this, we had observers perform a gaze-contingent search task that nudged them to deploy either spatial or feature-based attention to maximize performance. We found that, depending on the type of attention employed, visual spatial integration strength changed either in a strong and localized or a more modest and global manner compared with a baseline condition. Population code modeling revealed that a single mechanism can account for both observations: Attention acts beyond the neuronal encoding stage to tune the spatial integration weights of neural populations. Our study shows how attention and integration interact to optimize the information flow through the brain.

The Human Connectome Project 7 Tesla retinotopy dataset: Description and population receptive field analysis

About a quarter of human cerebral cortex is dedicated mainly to visual processing. The large-scale spatial organization of visual cortex can be measured with functional magnetic resonance imaging (fMRI) while subjects view spatially modulated visual stimuli, also known as "retinotopic mapping." One of the datasets collected by the Human Connectome Project involved ultrahigh-field (7 Tesla) fMRI retinotopic mapping in 181 healthy young adults (1.6-mm resolution), yielding the largest freely available collection of retinotopy data. Here, we describe the experimental paradigm and the results of model-based analysis of the fMRI data. These results provide estimates of population receptive field position and size. Our analyses include both results from individual subjects as well as results obtained by averaging fMRI time series across subjects at each cortical and subcortical location and then fitting models. Both the group-average and individual-subject results reveal robust signals across much of the brain, including occipital, temporal, parietal, and frontal cortex as well as subcortical areas. The group-average results agree well with previously published parcellations of visual areas. In addition, split-half analyses show strong within-subject reliability, further demonstrating the high quality of the data. We make publicly available the analysis results for individual subjects and the group average, as well as associated stimuli and analysis code. These resources provide an opportunity for studying fine-scale individual variability in cortical and subcortical organization and the properties of high-resolution fMRI. In addition, they provide a set of observations that can be compared with other Human Connectome Project measures acquired in these same participants.

Deep convolutional networks do not classify based on global object shape

Deep convolutional networks (DCNNs) are achieving previously unseen performance in object classification, raising questions about whether DCNNs operate similarly to human vision. In biological vision, shape is arguably the most important cue for recognition. We tested the role of shape information in DCNNs trained to recognize objects. In Experiment 1, we presented a trained DCNN with object silhouettes that preserved overall shape but were filled with surface texture taken from other objects. Shape cues appeared to play some role in the classification of artifacts, but little or none for animals. In Experiments 2–4, DCNNs showed no ability to classify glass figurines or outlines but correctly classified some silhouettes. Aspects of these results led us to hypothesize that DCNNs do not distinguish object’s bounding contours from other edges, and that DCNNs access some local shape features, but not global shape. In Experiment 5, we tested this hypothesis with displays that preserved local features but disrupted global shape, and vice versa. With disrupted global shape, which reduced human accuracy to 28%, DCNNs gave the same classification labels as with ordinary shapes. Conversely, local contour changes eliminated accurate DCNN classification but caused no difficulty for human observers. These results provide evidence that DCNNs have access to some local shape information in the form of local edge relations, but they have no access to global object shapes.

“Deep learning” systems–specifically, deep convolutional neural networks (DCNNs)–have recently achieved near human levels of performance in object recognition tasks. It has been suggested that the processing in these systems may model or explain object perception abilities in biological vision. For humans, shape is the most important cue for recognizing objects. We tested whether deep convolutional neural networks trained to recognize objects make use of object shape. Our findings indicate that other cues, such as surface texture, play a larger role in deep network classification than in human recognition. Most crucially, we show that deep learning systems have no sensitivity to the overall shape of an object. Whereas deep learning systems can access some local shape features, such as local orientation relations, they are not sensitive to the arrangement of these edge features or global shape in general, and they do not appear to distinguish bounding contours of objects from other edge information. These findings show a crucial divergence between artificial visual systems and biological visual processes.

The attentional repulsion effect and relative size judgments

Rapid shifts of involuntary attention have been shown to induce mislocalizations of nearby objects. One pattern of mislocalization, termed the Attentional Repulsion Effect (ARE), occurs when the onset of peripheral pre-cues lead to perceived shifts of subsequently presented stimuli away from the cued location. While the standard ARE configuration utilizes vernier lines, to date, all previous ARE studies have only assessed distortions along one direction and tested one spatial dimension (i.e., position or shape). The present study assessed the magnitude of the ARE using a novel stimulus configuration. Across three experiments participants judged which of two rectangles on the left or right side of the display appeared wider or taller. Pre-cues were used in Experiments 1 and 2. Results show equivalent perceived expansions in the width and height of the pre-cued rectangle in addition to baseline asymmetries in left/right relative size under no-cue conditions. Altering cue locations led to shifts in the perceived location of the same rectangles, demonstrating distortions in perceived shape and location using the same stimuli and cues. Experiment 3 demonstrates that rectangles are perceived as larger in the periphery compared to fixation, suggesting that eye movements cannot account for results from Experiments 1 and 2. The results support the hypothesis that the ARE reflects a localized, symmetrical warping of visual space that impacts multiple aspects of spatial and object perception.

Inversion produces opposite size illusions for faces and bodies

Faces are complex, multidimensional, and meaningful visual stimuli. Recently, Araragi, Aotani, & Kitaoka (2012) demonstrated an intriguing face size illusion whereby an inverted face is perceived as larger than a physically identical upright face. Like the face, the human body is a highly familiar and important stimulus in our lives. Here, we investigated the specificity of the size underestimation of upright faces illusion, testing whether similar effects also hold for bodies, hands, and everyday objects. Experiments 1a and 1b replicated the face-size illusion. No size illusion was observed for hands or objects. Unexpectedly, a reverse size illusion was observed for bodies, so that upright bodies were perceived as larger than their inverted counterparts. Experiment 2 showed that the face illusion was maintained even when the photographic contrast polarity of the stimuli was reversed, indicating that the visual system driving the illusion relies on geometric featural information rather than image contrast. In Experiment 2, the reverse size illusion for bodies failed to reach significance. Our findings show that size illusions caused by inversion show a high level of category specificity, with opposite illusions for faces and bodies.

Motion adaptation and attention: A critical review and meta-analysis

The motion aftereffect (MAE) provides a behavioural probe into the mechanisms underlying motion perception, and has been used to study the effects of attention on motion processing. Visual attention can enhance detection and discrimination of selected visual signals. However, the relationship between attention and motion processing remains contentious: not all studies find that attention increases MAEs. Our meta-analysis reveals several factors that explain superficially discrepant findings. Across studies (37 independent samples, 76 effects) motion adaptation was significantly and substantially enhanced by attention (Cohen's d = 1.12, p < .0001). The effect more than doubled when adapting to translating (vs. expanding or rotating) motion. Other factors affecting the attention-MAE relationship included stimulus size, eccentricity and speed. By considering these behavioural analyses alongside neurophysiological work, we conclude that feature-based (rather than spatial, or object-based) attention is the biggest driver of sensory adaptation. Comparisons between naïve and non-naïve observers, different response paradigms, and assessment of 'file-drawer effects' indicate that neither response bias nor publication bias are likely to have significantly inflated the estimated effect of attention.

The Attentional Suppressive Surround: Eccentricity, Location-Based and Feature-Based Effects and Interactions

The Selective Tuning model of visual attention (Tsotsos, 1990) has proposed that the focus of attention is surrounded by an inhibitory zone, eliciting a center-surround attentional distribution. This attentional suppressive surround inhibits irrelevant information which is located close to attended information in physical space (e.g., Cutzu and Tsotsos, 2003; Hopf et al., 2010) or in feature space (e.g., Tombu and Tsotsos, 2008; Störmer and Alvarez, 2014; Bartsch et al., 2017). In Experiment 1, we investigate the interaction between location-based and feature-based surround suppression and hypothesize that the attentional surround suppression would be maximized when spatially adjacent stimuli are also represented closely within a feature map. Our results demonstrate that perceptual discrimination is worst when two similar orientations are presented in proximity to each other, suggesting the interplay of the two surround suppression mechanisms. The Selective Tuning model also predicts that the size of the attentional suppressive surround is determined by the receptive field size of the neuron which optimally processes the attended information. The receptive field size of the processing neurons is tightly associated with stimulus size and eccentricity. Therefore, Experiment 2 tested the hypothesis that the size of the attentional suppressive surround would become larger as stimulus size and eccentricity increase, corresponding to an increase in the neuron's receptive field size. We show that stimulus eccentricity but not stimulus size modulates the size of the attentional suppressive surround. These results are consistent for both low- and high-level features (e.g., orientation and human faces). Overall, the present study supports the existence of the attentional suppressive surround and reveals new properties of this selection mechanism.

Feature integration is unaffected by saccade landing point, even when saccades land outside of the range of regular oculomotor variance

The experience of our visual surroundings appears continuous, contradicting the erratic nature of visual processing due to saccades. A possible way the visual system can construct a continuous experience is by integrating presaccadic and postsaccadic visual input. However, saccades rarely land exactly at the intended location. Feature integration would therefore need to be robust against variations in saccade execution to facilitate visual continuity. In the current study, observers reported a feature (color) of the saccade target, which occasionally changed slightly during the saccade. In transsaccadic change-trials, observers reported a mixture of the pre- and postsaccadic color, indicating transsaccadic feature integration. Saccade landing distance was not a significant predictor of the reported color. Next, to investigate the influence of more extreme deviations of saccade landing point on color reports, we used a global effect paradigm in a second experiment. In global effect trials, a distractor appeared together with the saccade target, causing most saccades to land in between the saccade target and the distractor. Strikingly, even when saccades land further away (up to 4°) from the saccade target than one would expect under single target conditions, there was no effect of saccade landing point on the reported color. We reason that saccade landing point does not affect feature integration, due to dissociation between the intended saccade target and the actual saccade landing point. Transsaccadic feature integration seems to be a mechanism that is dependent on visual spatial attention, and, as a result, is robust against variance in saccade landing point.