Disparity processing in primary visual cortex

Contextual Binocular Imbalance Impairs Local Stereopsis

Purpose

Binocular imbalance is known to inhibit stereopsis. This study investigates whether an imbalanced context around stereo stimuli also affects local stereopsis and explores the underlying mechanisms.

Methods

Three experiments were conducted with normally sighted participants. Experiment 1 measured local stereo detection thresholds under three context conditions: binocular balance (0.5 vs. 0.5 contrast), left-eye dominance (0.8 vs. 0.2 contrast), and right-eye dominance (0.2 vs. 0.8 contrast). Experiment 2 assessed the modulation of the imbalance effect by context-target collinearity. Experiment 3 examined the imbalance effect with binocular fusion and rivalry context stimuli.

Results

In experiment 1, the average stereo threshold was 62.4 arcsec in the binocular balance condition, elevated to 111.4 arcsec in the left-eye dominance (P = 0.003), and 114.7 arcsec in the right-eye dominance (P < 0.001), with no significant difference between the two imbalance conditions (P = 0.650). Experiment 2 showed that context-target collinearity modulated the imbalance effect, resulting in a smaller threshold elevation in the non-collinear condition (P = 0.011). Experiment 3 revealed significant main effects of imbalance (P = 0.031) and rivalry (P = 0.004), with no significant interaction (P = 0.966).

Conclusions

Contextual binocular imbalance inhibits local stereopsis, an effect modulated by collinearity and similarly observed in both binocular integrative and suppressive contexts. These findings suggest that lateral connectivity in the primary visual cortex (V1) plays a fundamental role in stereopsis generation, offering novel approaches for clinical interventions aimed at restoring binocular balance and stereopsis.

Contributions of Brain Microstructures and Metabolism to Visual Field Loss Patterns in Glaucoma Using Archetypal and Information Gain Analyses

Purpose

To investigate the contributions of the microstructural and metabolic brain environment to glaucoma and their association with visual field (VF) loss patterns by using advanced diffusion magnetic resonance imaging (dMRI), proton magnetic resonance spectroscopy (MRS), and clinical ophthalmic measures.

Methods

Sixty-nine glaucoma and healthy subjects underwent dMRI and/or MRS at 3 Tesla. Ophthalmic data were collected from VF perimetry and optical coherence tomography. dMRI parameters of microstructural integrity in the optic radiation and MRS-derived neurochemical levels in the visual cortex were compared among early glaucoma, advanced glaucoma, and healthy controls. Multivariate regression was used to correlate neuroimaging metrics with 16 archetypal VF loss patterns. We also ranked neuroimaging, ophthalmic, and demographic attributes in terms of their information gain to determine their importance to glaucoma.

Results

In dMRI, decreasing fractional anisotropy, radial kurtosis, and tortuosity and increasing radial diffusivity correlated with greater overall VF loss bilaterally. Regionally, decreasing intra-axonal space and extra-axonal space diffusivities correlated with greater VF loss in the superior-altitudinal area of the right eye and the inferior-altitudinal area of the left eye. In MRS, both early and advanced glaucoma patients had lower gamma-aminobutyric acid (GABA), glutamate, and choline levels than healthy controls. GABA appeared to associate more with superonasal VF loss, and glutamate and choline more with inferior VF loss. Choline ranked third for importance to early glaucoma, whereas radial kurtosis and GABA ranked fourth and fifth for advanced glaucoma.

Conclusions

Our findings highlight the importance of non-invasive neuroimaging biomarkers and analytical modeling for unveiling glaucomatous neurodegeneration and how they reflect complementary VF loss patterns.

Ocular dominance-dependent binocular combination of monocular neuronal responses in macaque V1

Primates rely on two eyes to perceive depth, while maintaining stable vision when either one eye or both eyes are open. Although psychophysical and modeling studies have investigated how monocular signals are combined to form binocular vision, the underlying neuronal mechanisms, particularly in V1 where most neurons exhibit binocularity with varying eye preferences, remain poorly understood. Here, we used two-photon calcium imaging to compare the monocular and binocular responses of thousands of simultaneously recorded V1 superficial-layer neurons in three awake macaques. During monocular stimulation, neurons preferring the stimulated eye exhibited significantly stronger responses compared to those preferring both eyes. However, during binocular stimulation, the responses of neurons preferring either eye were suppressed on the average, while those preferring both eyes were enhanced, resulting in similar neuronal responses irrespective of their eye preferences, and an overall response level similar to that with monocular viewing. A neuronally realistic model of binocular combination, which incorporates ocular dominance-dependent divisive interocular inhibition and binocular summation, is proposed to account for these findings.

Stereopsis without correspondence

Stereopsis has traditionally been considered a complex visual ability, restricted to large-brained animals. The discovery in the 1980s that insects, too, have stereopsis, therefore, challenged theories of stereopsis. How can such simple brains see in three dimensions? A likely answer is that insect stereopsis has evolved to produce simple behaviour, such as orienting towards the closer of two objects or triggering a strike when prey comes within range. Scientific thinking about stereopsis has been unduly anthropomorphic, for example assuming that stereopsis must require binocular fusion or a solution of the stereo correspondence problem. In fact, useful behaviour can be produced with very basic stereoscopic algorithms which make no attempt to achieve fusion or correspondence, or to produce even a coarse map of depth across the visual field. This may explain why some aspects of insect stereopsis seem poorly designed from an engineering point of view: for example, paying no attention to whether interocular contrast or velocities match. Such algorithms demonstrably work well enough in practice for their species, and may prove useful in particular autonomous applications. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Low frequency repetitive transcranial magnetic stimulation promotes plasticity of the visual cortex in adult amblyopic rats

The decline of visual plasticity restricts the recovery of visual functions in adult amblyopia. Repetitive transcranial magnetic stimulation (rTMS) has been shown to be effective in treating adult amblyopia. However, the underlying mechanisms of rTMS on visual cortex plasticity remain unclear. In this study, we found that low-frequency rTMS reinstated the amplitude of visual evoked potentials, but did not influence the impaired depth perception of amblyopic rats. Furthermore, the expression of synaptic plasticity genes and the number of dendritic spines were significantly higher in amblyopic rats which received rTMS when compared with amblyopic rats which received sham stimulation, with reduced level of inhibition and perineuronal nets in visual cortex, as observed via molecular and histological investigations. The results provide further evidence that rTMS enhances functional recovery and visual plasticity in an adult amblyopic animal model.

Stimulating both eyes with matching stimuli enhances V1 responses

Neurons in the primary visual cortex (V1) of primates play a key role in combining monocular inputs to form a binocular response. Although much has been gleaned from studying how V1 responds to discrepant (dichoptic) images, equally important is to understand how V1 responds to concordant (dioptic) images in the two eyes. Here, we investigated the extent to which concordant, balanced, zero-disparity binocular stimulation modifies V1 responses to varying stimulus contrast using intracranial multielectrode arrays. On average, binocular stimuli evoked stronger V1 activity than their monocular counterparts. This binocular facilitation scaled most proportionately with contrast during the initial transient. As V1 responses evolved, additional contrast-mediated dynamics emerged. Specifically, responses exhibited longer maintenance of facilitation for lower contrast and binocular suppression at high contrast. These results suggest that V1 processes concordant stimulation of both eyes in at least two sequential steps: initial response enhancement followed by contrast-dependent control of excitation.

A unified model for binocular fusion and depth perception

We describe a new unified model to explain both binocular fusion and depth perception, over a broad range of depths. At each location, the model consists of an array of paired spatial frequency filters, with different relative horizontal shifts (position disparity) and interocular phase disparities of 0, 90, ±180, or -90°. The paired filters with different spatial profiles (non-zero phase disparity) compute interocular misalignment and provide phase-disparity energy (binocular fusion energy) to drive selection of the appropriate filters along the position disparity space until the misalignment is eliminated and sensory fusion is achieved locally. The paired filters with identical spatial profiles (0 phase disparity) compute the position-disparity energy. After sensory fusion, the combination of position and possible residual phase disparity energies is calculated for binocular depth perception. Binocular fusion occurs at multiple scales following a coarse-to-fine process. At a given location, the apparent depth is the weighted sum of fusion shifts combined with residual phase disparity in all spatial-frequency channels, and the weights depend on stimulus spatial frequency and stimulus contrast. To test the theory, we measured disparity minimum and maximum thresholds (Dmin and Dmax) at three spatial frequencies and with different intraocular contrast levels. The stimuli were Random-Gabor-Patch (RGP) stereograms consisting of Gabor patches with random positions and phases, but with a fixed spatial frequency. The two eyes viewed identical arrays of patches except that one eye's array could be shifted horizontally and could differ in contrast. Our experiments and modeling reveal two contrast normalization mechanisms: (1) Energy Normalization (EN): Binocular energy is normalized with monocular energy after the site of binocular combination. This predicts constant Dmin thresholds when varying stimulus contrast in the two eyes; (2) DSKL model Interocular interactions: Monocular contrasts are normalized before the binocular combination site through interocular contrast gain-control and gain-enhancement mechanisms. This predicts contrast dependent Dmax thresholds. We tested a range of models and found that a model consisting of a second-order pathway with DSKL interocular interactions and a first-order pathway with EN at each spatial-frequency band can account for both the Dmin and Dmax data very well. Simulations show that the model makes reasonable predictions of suprathreshold depth perception.

Evaluating the Acute Effect of Stereoscopic Recovery by Dichoptic Stimulation Using Electroencephalogram

Amblyopia is a common developmental disorder in adolescents and children. Stereoscopic loss is a symptom of amblyopia that can seriously affect the quality of patient's life. Recent studies have shown that the push-pull perceptual learning protocol had a positive effect on stereoscopic recovery. In this study, we developed a stereoscopic training method using a polarized visualization system according to the push-pull protocol. Dichoptic stimulation for 36 anisometropic and amblyopic subjects and 33 children with normal visual acuity (VA) has been conducted. Electroencephalogram (EEG) was used to evaluate the neurophysiological changes before, during, and after stimulation. For the anisometropic and amblyopic subjects, the statistical analysis demonstrated significant differences (p < 0.01) in the beta rhythm at the middle temporal and occipital lobes, while the EEG from the normal VA subjects indicated no significant changes when comparing the results before and after training. We concluded that the dichoptic training in our study can activate the middle temporal visual area and visual cortex. The EEG changes can be used to evaluate the training effects. This study also found that the beta band EEG acquired during visual stimulation at the dorsal visual stream can be potentially used for predicting acute training effect. The results facilitated the optimization of the individual training plan.

Data-Driven Approaches to Understanding Visual Neuron Activity

With modern neurophysiological methods able to record neural activity throughout the visual pathway in the context of arbitrarily complex visual stimulation, our understanding of visual system function is becoming limited by the available models of visual neurons that can be directly related to such data. Different forms of statistical models are now being used to probe the cellular and circuit mechanisms shaping neural activity, understand how neural selectivity to complex visual features is computed, and derive the ways in which neurons contribute to systems-level visual processing. However, models that are able to more accurately reproduce observed neural activity often defy simple interpretations. As a result, rather than being used solely to connect with existing theories of visual processing, statistical modeling will increasingly drive the evolution of more sophisticated theories.

Visual Perception: Neural Networks for Stereopsis

How does our brain use differences between the images in our two eyes, binocular disparities, to generate depth perception? New work shows that a type of neural network trained on natural binocular images can learn parameters that match key properties of visual cortex. Most information is conveyed by cells which sense differences between the two eyes' images.

The psychophysics of stereopsis can be explained without invoking independent ON and OFF channels

Early vision proceeds through distinct ON and OFF channels, which encode luminance increments and decrements respectively. It has been argued that these channels also contribute separately to stereoscopic vision. This is based on the fact that observers perform better on a noisy disparity discrimination task when the stimulus is a random-dot pattern consisting of equal numbers of black and white dots (a “mixed-polarity stimulus,” argued to activate both ON and OFF stereo channels), than when it consists of all-white or all-black dots (“same-polarity,” argued to activate only one). However, it is not clear how this theory can be reconciled with our current understanding of disparity encoding. Recently, a binocular convolutional neural network was able to replicate the mixed-polarity advantage shown by human observers, even though it was based on linear filters and contained no mechanisms which would respond separately to black or white dots. Here, we show that a subtle feature of the way the stimuli were constructed in all these experiments can explain the results. The interocular correlation between left and right images is actually lower for the same-polarity stimuli than for mixed-polarity stimuli with the same amount of disparity noise applied to the dots. Because our current theories suggest stereopsis is based on a correlation-like computation in primary visual cortex, this postulate can explain why performance was better for the mixed-polarity stimuli. We conclude that there is currently no evidence supporting separate ON and OFF channels in stereopsis.

Stereopsis in animals: evolution, function and mechanisms

Stereopsis is the computation of depth information from views acquired simultaneously from different points in space. For many years, stereopsis was thought to be confined to primates and other mammals with front-facing eyes. However, stereopsis has now been demonstrated in many other animals, including lateral-eyed prey mammals, birds, amphibians and invertebrates. The diversity of animals known to have stereo vision allows us to begin to investigate ideas about its evolution and the underlying selective pressures in different animals. It also further prompts the question of whether all animals have evolved essentially the same algorithms to implement stereopsis. If so, this must be the best way to do stereo vision, and should be implemented by engineers in machine stereopsis. Conversely, if animals have evolved a range of stereo algorithms in response to different pressures, that could inspire novel forms of machine stereopsis appropriate for distinct environments, tasks or constraints. As a first step towards addressing these ideas, we here review our current knowledge of stereo vision in animals, with a view towards outlining common principles about the evolution, function and mechanisms of stereo vision across the animal kingdom. We conclude by outlining avenues for future work, including research into possible new mechanisms of stereo vision, with implications for machine vision and the role of stereopsis in the evolution of camouflage.

Summary: Stereopsis has evolved independently in different animals. We review the various functions it serves and the variety of mechanisms that could underlie stereopsis in different species.

Binocular response modulation in the lateral geniculate nucleus

The dorsal lateral geniculate nucleus of the thalamus (LGN) receives the main outputs of both eyes and relays those signals to the visual cortex. Each retina projects to separate layers of the LGN so that each LGN neuron is innervated by a single eye. In line with this anatomical separation, visual responses of almost all of LGN neurons are driven by one eye only. Nonetheless, many LGN neurons are sensitive to what is shown to the other eye as their visual responses differ when both eyes are stimulated compared to when the driving eye is stimulated in isolation. This, predominantly suppressive, binocular modulation of LGN responses might suggest that the LGN is the first location in the primary visual pathway where the outputs from the two eyes interact. Indeed, the LGN features several anatomical structures that would allow for LGN neurons responding to one eye to modulate neurons that respond to the other eye. However, it is also possible that binocular response modulation in the LGN arises indirectly as the LGN also receives input from binocular visual structures. Here we review the extant literature on the effects of binocular stimulation on LGN spiking responses, highlighting findings from cats and primates, and evaluate the neural circuits that might mediate binocular response modulation in the LGN.

Vision in our three-dimensional world

Many aspects of our perceptual experience are dominated by the fact that our two eyes point forward. Whilst the location of our eyes leaves the environment behind our head inaccessible to vision, co-ordinated use of our two eyes gives us direct access to the three-dimensional structure of the scene in front of us, through the mechanism of stereoscopic vision. Scientific understanding of the different brain regions involved in stereoscopic vision and three-dimensional spatial cognition is changing rapidly, with consequent influences on fields as diverse as clinical practice in ophthalmology and the technology of virtual reality devices.This article is part of the themed issue 'Vision in our three-dimensional world'.

Gain Modulation as a Mechanism for Coding Depth from Motion Parallax in Macaque Area MT

Observer translation produces differential image motion between objects that are located at different distances from the observer's point of fixation [motion parallax (MP)]. However, MP can be ambiguous with respect to depth sign (near vs far), and this ambiguity can be resolved by combining retinal image motion with signals regarding eye movement relative to the scene. We have previously demonstrated that both extra-retinal and visual signals related to smooth eye movements can modulate the responses of neurons in area MT of macaque monkeys, and that these modulations generate neural selectivity for depth sign. However, the neural mechanisms that govern this selectivity have remained unclear. In this study, we analyze responses of MT neurons as a function of both retinal velocity and direction of eye movement, and we show that smooth eye movements modulate MT responses in a systematic, temporally precise, and directionally specific manner to generate depth-sign selectivity. We demonstrate that depth-sign selectivity is primarily generated by multiplicative modulations of the response gain of MT neurons. Through simulations, we further demonstrate that depth can be estimated reasonably well by a linear decoding of a population of MT neurons with response gains that depend on eye velocity. Together, our findings provide the first mechanistic description of how visual cortical neurons signal depth from MP.SIGNIFICANCE STATEMENT Motion parallax is a monocular cue to depth that commonly arises during observer translation. To compute from motion parallax whether an object appears nearer or farther than the point of fixation requires combining retinal image motion with signals related to eye rotation, but the neurobiological mechanisms have remained unclear. This study provides the first mechanistic account of how this interaction takes place in the responses of cortical neurons. Specifically, we show that smooth eye movements modulate the gain of responses of neurons in area MT in a directionally specific manner to generate selectivity for depth sign from motion parallax. We also show, through simulations, that depth could be estimated from a population of such gain-modulated neurons.

Neurons in Striate Cortex Signal Disparity in Half-Matched Random-Dot Stereograms

Human stereopsis can operate in dense "cyclopean" images containing no monocular objects. This is believed to depend on the computation of binocular correlation by neurons in primary visual cortex (V1). The observation that humans perceive depth in half-matched random-dot stereograms, although these stimuli have no net correlation, has led to the proposition that human depth perception in these stimuli depends on a distinct "matching" computation possibly performed in extrastriate cortex. However, recording from disparity-selective neurons in V1 of fixating monkeys, we found that they are in fact able to signal disparity in half-matched stimuli. We present a simple model that explains these results. This reinstates the view that disparity-selective neurons in V1 provide the initial substrate for perception in dense cyclopean stimuli, and strongly suggests that separate correlation and matching computations are not necessary to explain existing data on mixed correlation stereograms.

SIGNIFICANCE STATEMENT

The initial step in stereoscopic 3D vision is generally thought to be a correlation-based computation that takes place in striate cortex. Recent research has argued that there must be an additional matching computation involved in extracting stereoscopic depth in random-dot stereograms. This is based on the observation that humans can perceive depth in stimuli with a mean binocular correlation of zero (where a correlation-based mechanism should not signal depth). We show that correlation-based cells in striate cortex do in fact signal depth here because they convert fluctuations in the correlation level into a mean change in the firing rate. Our results reinstate the view that these cells provide a sufficient substrate for the perception of stereoscopic depth.

Spontaneous Rapid Odor Source Localization Behavior Requires Interhemispheric Communication

Navigation, finding food sources, and avoiding danger critically depend on the identification and spatial localization of airborne chemicals. When monitoring the olfactory environment, rodents spontaneously engage in active olfactory sampling behavior, also referred to as exploratory sniffing [1]. Exploratory sniffing is characterized by stereotypical high-frequency respiration, which is also reliably evoked by novel odorant stimuli [2, 3]. To study novelty-induced exploratory sniffing, we developed a novel, non-contact method for measuring respiration by infrared (IR) thermography in a behavioral paradigm in which novel and familiar stimuli are presented to head-restrained mice. We validated the method by simultaneously performing nasal pressure measurements, a commonly used invasive approach [2, 4], and confirmed highly reliable detection of inhalation onsets. We further discovered that mice actively orient their nostrils toward novel, previously unexperienced, smells. In line with the remarkable speed of olfactory processing reported previously [3, 5, 6], we find that mice initiate their response already within the first sniff after odor onset. Moreover, transecting the anterior commissure (AC) disrupted orienting, indicating that the orienting response requires interhemispheric transfer of information. This suggests that mice compare odorant information obtained from the two bilaterally symmetric nostrils to locate the source of the novel odorant. We further demonstrate that asymmetric activation of the anterior olfactory nucleus (AON) is both necessary and sufficient for eliciting orienting responses. These findings support the view that the AON plays an important role in the internostril difference comparison underlying rapid odor source localization.

Three-dimensional ocular kinematics underlying binocular single vision

We have analyzed the binocular coordination of the eyes during far-to-near refixation saccades based on the evaluation of distance ratios and angular directions of the projected target images relative to the eyes' rotation centers. By defining the geometric point of binocular single vision, called Helmholtz point, we found that disparities during fixations of targets at near distances were limited in the subject's three-dimensional visual field to the vertical and forward directions. These disparities collapsed to simple vertical disparities in the projective binocular image plane. Subjects were able to perfectly fuse the vertically disparate target images with respect to the projected Helmholtz point of single binocular vision, independent of the particular location relative to the horizontal plane of regard. Target image fusion was achieved by binocular torsion combined with corrective modulations of the differential half-vergence angles of the eyes in the horizontal plane. Our findings support the notion that oculomotor control combines vergence in the horizontal plane of regard with active torsion in the frontal plane to achieve fusion of the dichoptic binocular target images.