Peripheral and central sensation: multisensory orienting and recognition across species

Noise correlations and neuronal diversity may limit the utility of winner-take-all readout in a pop out visual search task

Visual search involves active scanning of the environment to locate objects of interest against a background of irrelevant distractors. One widely accepted theory posits that pop out visual search is computed by a winner-take-all (WTA) competition between contextually modulated cells that form a saliency map. However, previous studies have shown that the ability of WTA mechanisms to accumulate information from large populations of neurons is limited, thus raising the question of whether WTA can underlie pop out visual search. To address this question, we conducted a modeling study to investigate how accurately the WTA mechanism can detect the deviant stimulus in a pop out task. We analyzed two types of WTA readout mechanisms: single-best-cell WTA, where the decision is made based on a single winning cell, and a generalized population-based WTA, where the decision is based on the winning population of similarly tuned cells. Our results show that neither WTA mechanism can account for the high accuracy found in behavioral experiments. The inherent neuronal heterogeneity prevents the single-best-cell WTA from accumulating information even from large populations, whereas the accuracy of the generalized population-based WTA algorithm is negatively affected by the widely reported noise correlations. These findings underscore the need to revisit the key assumptions explored in our theoretical analysis, particularly concerning the decoding mechanism and the statistical properties of neuronal population responses to pop out stimuli. The analysis identifies specific response statistics that require further empirical characterization to accurately predict WTA performance in biologically plausible models of visual pop out detection.

Testing the top-down feedback in the central visual field using the reversed depth illusion

In a new framework to understand vision, an information bottleneck impoverishes visual input information downstream of the primary visual cortex along the visual pathway; to aid ongoing visual recognition given the bottleneck, feedback from downstream to upstream visual stages queries for additional information. According to the central-peripheral dichotomy theory, this feedback is primarily directed to the central, rather than the peripheral, visual field. Counterintuitively, this theory predicts illusions visible only in the peripheral visual field, which lacks the feedback query to veto the illusions arising from misleading and impoverished feedforward signals. A paradigmatic example is the predicted and confirmed reversed depth illusion in random-dot stereograms. This theory further predicts that disrupting the feedback renders this illusion visible in the central visual field. We test and confirm this prediction using visual backward masking to disrupt the feedback. This feedback privilege for the central visual field underpins visual understanding through analysis-by-synthesis.

Spatial properties of scintillating grid illusion through visual experiments and numerical simulations

This study investigated the spatial properties of the scintillating grid illusion through three visual experiments and numerical simulations using differential equations. Experiment 1 was conducted to confirm that the scintillating grid illusion occurred in the peripheral vision under binocular viewing. The results showed that illusory blackness was perceived on the white disk at the horizontal viewing angles of ±6.0, ±9.0, and ±12.0 degrees stronger than ±0.6 and ±3.0 degrees. Experiment 2 investigated the area where the scintillating grid illusion occurred not only in the horizontal orientation but also in the vertical orientation. The results showed that the area of the scintillating grid illusion was farther from the fixation point in the horizontal orientation than in the vertical orientation under binocular viewing. Experiment 3 examined the spatial properties of the scintillating grid illusion under monocular viewing, revealing that the area of the scintillating grid illusion was wider in the horizontal orientation than in the vertical orientation. These results suggest that the scintillating grid illusion has spatial anisotropy, regardless of binocular or monocular viewing. Based on the findings in the visual experiments and electrophysiology, this study improved a mathematical model using differential equations for retinal information processing. The improved model demonstrated the results of numerical simulations similar to the spatial properties of the scintillating grid illusion under experimental results. The numerical simulations suggested that the blurring and inhibitory effects could be involved in the spatial properties of the scintillating grid illusion.

Spatial comparison of disappearance and scintillation phenomena using a single-unit scintillating grid illusion

The scintillating grid illusion induces the phenomena of disappearance and scintillation. However, it is unclear in which peripheral region these phenomena occur. This study aimed to investigate the spatial properties of disappearance and scintillation phenomena in the scintillating grid illusion. In Experiment 1, participants binocularly observed a single-unit scintillating grid illusion and responded whether a white disk and illusory blackness were perceived. As a result, the perceptual region of the white disk was larger in the horizontal direction than in the vertical direction, as well as the perceptual region of the illusory blackness. This result indicates that both perceptual regions have spatial anisotropy. In Experiment 2, the same task as in Experiment 1 was performed with monocular viewing. The results did not exactly reject spatial anisotropy in monocular vision, regardless of the perceptual regions. This study may contribute to understanding how disappearance and scintillation phenomena coexist in the scintillating grid illusion.

The unbearable slowness of being: Why do we live at 10 bits/s?

This article is about the neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ∼109 bits/s. The stark contrast between these numbers remains unexplained and touches on fundamental aspects of brain function: what neural substrate sets this speed limit on the pace of our existence? Why does the brain need billions of neurons to process 10 bits/s? Why can we only think about one thing at a time? The brain seems to operate in two distinct modes: the "outer" brain handles fast high-dimensional sensory and motor signals, whereas the "inner" brain processes the reduced few bits needed to control behavior. Plausible explanations exist for the large neuron numbers in the outer brain, but not for the inner brain, and we propose new research directions to remedy this.

Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region

High-dimensional brain activity is often organized into lower-dimensional neural manifolds. However, the neural manifolds of the visual cortex remain understudied. Here, we study large-scale multi-electrode electrophysiological recordings of macaque (Macaca mulatta) areas V1, V4, and DP with a high spatiotemporal resolution. We find that the population activity of V1 contains two separate neural manifolds, which correlate strongly with eye closure (eyes open/closed) and have distinct dimensionalities. Moreover, we find strong top-down signals from V4 to V1, particularly to the foveal region of V1, which are significantly stronger during the eyes-open periods. Finally, in silico simulations of a balanced spiking neuron network qualitatively reproduce the experimental findings. Taken together, our analyses and simulations suggest that top-down signals modulate the population activity of V1. We postulate that the top-down modulation during the eyes-open periods prepares V1 for fast and efficient visual responses, resulting in a type of visual stand-by state.

Both stimulus-specific and configurational features of multiple visual stimuli shape the spatial ventriloquism effect

Studies on multisensory perception often focus on simplistic conditions in which one single stimulus is presented per modality. Yet, in everyday life, we usually encounter multiple signals per modality. To understand how multiple signals within and across the senses are combined, we extended the classical audio-visual spatial ventriloquism paradigm to combine two visual stimuli with one sound. The individual visual stimuli presented in the same trial differed in their relative timing and spatial offsets to the sound, allowing us to contrast their individual and combined influence on sound localization judgements. We find that the ventriloquism bias is not dominated by a single visual stimulus but rather is shaped by the collective multisensory evidence. In particular, the contribution of an individual visual stimulus to the ventriloquism bias depends not only on its own relative spatio-temporal alignment to the sound but also the spatio-temporal alignment of the other visual stimulus. We propose that this pattern of multi-stimulus multisensory integration reflects the evolution of evidence for sensory causal relations during individual trials, calling for the need to extend established models of multisensory causal inference to more naturalistic conditions. Our data also suggest that this pattern of multisensory interactions extends to the ventriloquism aftereffect, a bias in sound localization observed in unisensory judgements following a multisensory stimulus.

Peripheral vision is mainly for looking rather than seeing

Vision includes looking and seeing. Looking, mainly via gaze shifts, selects a fraction of visual input information for passage through the brain's information bottleneck. The selected input is placed within the attentional spotlight, typically in the central visual field. Seeing decodes, i.e., recognizes and discriminates, the selected inputs. Hence, peripheral vision should be mainly devoted to looking, in particular, deciding where to shift the gaze. Looking is often guided exogenously by a saliency map created by the primary visual cortex (V1), and can be effective with no seeing and limited awareness. In seeing, peripheral vision not only suffers from poor spatial resolution, but is also subject to crowding and is more vulnerable to illusions by misleading, ambiguous, and impoverished visual inputs. Central vision, mainly for seeing, enjoys the top-down feedback that aids seeing in light of the bottleneck which is hypothesized to starts from V1 to higher areas. This feedback queries for additional information from lower visual cortical areas such as V1 for ongoing recognition. Peripheral vision is deficient in this feedback according to the Central-peripheral Dichotomy (CPD) theory. The saccades engendered by peripheral vision allows looking to combine with seeing to give human observers the impression of seeing the whole scene clearly despite inattentional blindness.

Eye movement evidence for the V1 Saliency Hypothesis and the Central-peripheral Dichotomy theory in an anomalous visual search task

Typically, searching for a target among uniformly tilted non-targets is easier when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH) - that V1 creates a saliency map to guide attention exogenously - predicts exactly the opposite in a special case: each target or non-target is a pair of equally-sized disks, a homo-pair of two disks of the same color, black or white, or a hetero-pair of two disks of the opposite color; the inter-disk displacement defines its orientation. This prediction - parallel advantage - was supported by the finding that parallel targets require shorter reaction times (RTs) to report targets' locations. Furthermore, it is stronger for targets further from the center of search images, as predicted by the Central-peripheral Dichotomy (CPD) theory entailing that saliency effects are stronger in peripheral than in central vision. However, the parallel advantage could arise from a shorter time required to recognize - rather than to shift attention to - the parallel target. By gaze tracking, the present study confirms that the parallel advantage is solely due to the RTs for the gaze to reach the target. Furthermore, when the gaze is sufficiently far from the target during search, saccade to a parallel, rather than perpendicular, target is more likely, demonstrating the Central-peripheral Dichotomy more directly. Parallel advantage is stronger among observers encouraged to let their search be guided by spontaneous gaze shifts, which are presumably guided by bottom-up saliency rather than top-down factors.