A unified model for binocular fusion and depth perception

Development of human binocular vision: An electrophysiological perspective

Vision with two eyes confers evolutionary advantages in terms of field of view, binocular summation and the sense of depth from disparity and motion. This review summarizes our current knowledge of the development of binocular vision through the lens of Visual Evoked Potentials (VEPs). The review begins with early studies of binocular summation that compared monocular to binocular VEP amplitude ratios. This is followed by a description of more definitive indices of binocular interaction afforded by studies of dichoptic masking and intermodulation. We then describe a striking immaturity of binocular motion processing - the developmental motion asymmetry - a monocular nasalward/temporalward asymmetry of motion processing that reflects the child's developmental history of normal binocular interaction. We end with a review of the development of sensitivity to the primary cue for depth - horizontal retinal disparity. Together, the available results paint a picture of early competency in some respects, combined with both quantitative and important qualitative differences from the adult that suggest the presence of distinct processing mechanisms with different developmental sequences.

Contextual Binocular Imbalance Impairs Local Stereopsis

Purpose

Binocular imbalance is known to inhibit stereopsis. This study investigates whether an imbalanced context around stereo stimuli also affects local stereopsis and explores the underlying mechanisms.

Methods

Three experiments were conducted with normally sighted participants. Experiment 1 measured local stereo detection thresholds under three context conditions: binocular balance (0.5 vs. 0.5 contrast), left-eye dominance (0.8 vs. 0.2 contrast), and right-eye dominance (0.2 vs. 0.8 contrast). Experiment 2 assessed the modulation of the imbalance effect by context-target collinearity. Experiment 3 examined the imbalance effect with binocular fusion and rivalry context stimuli.

Results

In experiment 1, the average stereo threshold was 62.4 arcsec in the binocular balance condition, elevated to 111.4 arcsec in the left-eye dominance (P = 0.003), and 114.7 arcsec in the right-eye dominance (P < 0.001), with no significant difference between the two imbalance conditions (P = 0.650). Experiment 2 showed that context-target collinearity modulated the imbalance effect, resulting in a smaller threshold elevation in the non-collinear condition (P = 0.011). Experiment 3 revealed significant main effects of imbalance (P = 0.031) and rivalry (P = 0.004), with no significant interaction (P = 0.966).

Conclusions

Contextual binocular imbalance inhibits local stereopsis, an effect modulated by collinearity and similarly observed in both binocular integrative and suppressive contexts. These findings suggest that lateral connectivity in the primary visual cortex (V1) plays a fundamental role in stereopsis generation, offering novel approaches for clinical interventions aimed at restoring binocular balance and stereopsis.

What Factors Affect Binocular Summation?

Binocular vision may serve as a good model for research on awareness. Binocular summation (BS) can be defined as the superiority of binocular over monocular visual performance. Early studies of BS found an improvement of a factor of about 1.4 (empirically), leading to models suggesting a quadratic summation of the two monocular inputs (√2). Neural interaction modulates a target's visibility within the same eye or between eyes (facilitation or suppression). Recent results indicated that at a closely flanked stimulus, BS is characterized by instability; it relies on the specific order in which the stimulus condition is displayed. Otherwise, BS is stable. These results were revealed in experiments where the tested eye was open, whereas the other eye was occluded (mono-optic glasses, blocked presentation); thus, the participants were aware of the tested eye. Therefore, in this study, we repeated the same experiments but utilized stereoscopic glasses (intermixed at random presentation) to control the monocular and binocular vision, thus potentially eliminating awareness of the tested condition. The stimuli consisted of a central vertically oriented Gabor target and high-contrast Gabor flankers positioned in two configurations (orthogonal or collinear) with target-flanker separations of either two or three wavelengths (λ), presented at four different presentation times (40, 80, 120, and 200 ms). The results indicate that when utilizing stereoscopic glasses and mixing the testing conditions, the BS is normal, raising the possibility that awareness may be involved.

Evaluation of monocular and binocular contrast perception on virtual reality head-mounted displays

Purpose

Visualization of medical images on a virtual reality (VR) head-mounted display (HMD) requires binocular fusion of a stereoscopic pair of graphical views. However, current image quality assessment on VR HMDs for medical applications has been primarily limited to time-consuming monocular optical bench measurement on a single eyepiece.

Approach

As an alternative to optical bench measurement to quantify the image quality on VR HMDs, we developed a WebXR test platform to perform contrast perceptual experiments that can be used for binocular image quality assessment. We obtained monocular and binocular contrast sensitivity responses (CSRs) from participants on a Meta Quest 2 VR HMD using varied interpupillary distance (IPD) configurations.

Results

The perceptual result shows that contrast perception on VR HMDs is primarily affected by optical aberration of the VR HMD. As a result, monocular CSR degrades at a high spatial frequency greater than 4 cycles per degree when gazing at the periphery of the display field of view, especially for mismatched IPD settings consistent with optical bench measurements. On the contrary, binocular contrast perception is dominated by the monocular view with superior image quality measured by the contrast.

Conclusions

We developed a test platform to investigate monocular and binocular contrast perception by performing perceptual experiments. The test method can be used to evaluate monocular and/or binocular image quality on VR HMDs for potential medical applications without extensive optical bench measurements.

Wearable apparatus for correction of visual alignment under torsional strabismus

SIGNIFICANCE

A wearable optical apparatus that compensates for eye misalignment (strabismus) to correct for double vision (diplopia) is proposed. In contrast to prism lenses, commonly used to compensate for horizontal and/or vertical misalignment, the proposed approach is able to compensate for any combination of horizontal, vertical, and torsional misalignment.

PURPOSE

If the action of the extraocular muscles is compromised (e.g., by nerve damage), a patient may lose their ability to maintain visual alignment, negatively affecting their binocular fusion and stereo depth perception capability. Torsional misalignment cannot be mitigated by standard Fresnel prism lenses. Surgical procedures intended to correct torsional misalignment may be unpredictable. A wearable device able to rectify visual alignment and restore stereo depth perception without surgical intervention could potentially be of great value to people with strabismus.

METHODS

We propose a novel lightweight wearable optical device for visual alignment correction. The device comprises two mirrors and a Fresnel prism, arranged in such a way that together they rotationally shift the view seen by the affected eye horizontally, vertically, and torsionally. The extent of the alignment correction on each axis can be arbitrarily adjusted according to the patient's particular misalignment characteristics.

RESULTS

The proposed approach was tested by computer simulation, and a prototype device was manufactured. The prototype device was tested by a strabismus patient exhibiting horizontal and torsional misalignment. In these tests, the device was found to function as intended, allowing the patient to enjoy binocular fusion and stereo depth perception while wearing the device for daily activities over a period of several months.

CONCLUSIONS

The proposed device is effective in correcting arbitrary horizontal, vertical, and torsional misalignment of the eyes. The results of the initial testing performed are highly encouraging. Future study is warranted to formally assess the effectiveness of the device on multiple test patients.

Absolute and relative disparity mechanisms revealed by an equivalent noise analysis

The precision of stereopsis and vergence are ultimately limited by internal binocular disparity noise. Here we propose an equivalent noise model with both global and local internal disparity noises to provide a unified explanation of both absolute and relative disparity thresholds. To test this model, we developed a psychophysical procedure to measure the equivalent internal disparity noise by adding external disparity noise to random-Gabor-patch stereograms. We used the method of constant stimuli to measure the minimum and maximum disparity thresholds (Dmin and Dmax) for both absolute and relative disparity. Consistent with previous studies, we found that Dmin thresholds are substantially worse for absolute disparity than for relative disparity. We tested three relative disparity mechanisms: (1) the difference between the monocular separations of targets projecting to the two eyes; (2) the direct measurement of relative disparity; and (3) the difference of absolute disparities of targets. Computing the difference of absolute disparities when detecting relative disparity, Mechanism 3 cancels global noise, resulting in a much lower relative Dmin threshold, and provides a reasonable fit to the experimental data. We also found that the presence of as much as 2400 arcsec of external disparity noise does not appear to affect the Dmax threshold. This observation suggests that Dmax is implicated in a mechanism that disregards the disparity variance of individual items, relying instead on the average disparity across all items, supporting the depth model proposed in our previous study (Ding & Levi, 2021), which posits distinct mechanisms governing Dmin and Dmax thresholds.

Effects of binocular disparity on binocular luminance combination

This study aimed to examine the effects of binocular disparity on binocular combination of brightness information coming from luminance increments and decrements. The point of subjective equality was determined by asking the observers to judge which stimulus appeared brighter-a bar stimulus with variable disparity or another stimulus with zero disparity. For the bar stimulus, the interocular luminance ratio was varied to trace an equal brightness curve. Binocular disparity had no effect on luminance increments presented on a gray or black background. In contrast, when luminance decrements were presented on a gray background, non-zero disparities elevated points of subjective equality for stimuli with interocular luminance differences. This means that the binocular brightness combination of the two monocular signals shifted from winner-take-all summation toward linear averaging. It has been argued that this effect may be caused by non-zero binocular disparities attenuating interocular suppression, which is deemed to operate normally with zero disparity.

The effect of different depth planes during a manual tracking task in three-dimensional virtual reality space

Unlike ballistic arm movements such as reaching, the contribution of depth information to the performance of manual tracking movements is unclear. Thus, to understand how the brain handles information, we investigated how a required movement along the depth axis would affect behavioral tracking performance, postulating that it would be affected by the amount of depth movement. We designed a visually guided planar tracking task that requires movement on three planes with different depths: a fronto-parallel plane called ROT (0), a sagittal plane called ROT (90), and a plane rotated by 45° with respect to the sagittal plane called ROT (45). Fifteen participants performed a circular manual tracking task under binocular and monocular visions in a three-dimensional (3D) virtual reality space. As a result, under binocular vision, ROT (90), which required the largest depth movement among the tasks, showed the greatest error in 3D. Similarly, the errors (deviation from the target path) on the depth axis revealed significant differences among the tasks. Under monocular vision, significant differences in errors were observed only on the lateral axis. Moreover, we observed that the errors in the lateral and depth axes were proportional to the required movement on these axes under binocular vision and confirmed that the required depth movement under binocular vision determined depth error independent of the other axes. This finding implies that the brain may independently process binocular vision information on each axis. Meanwhile, the required depth movement under monocular vision was independent of performance along the depth axis, indicating an intractable behavior. Our findings highlight the importance of handling depth movement, especially when a virtual reality situation, involving tracking tasks, is generated.

Space, time, and dynamics of binocular interactions

Binocular summation (BS), defined as the superiority of binocular over monocular visual performance, shows that thresholds are about 40% (a factor of 1.4) better in binocular than in monocular viewing. However, it was reported that different amounts of BS exist in a range from 1.4 to 2 values because BS is affected by the spatiotemporal parameters of the stimulus. Lateral interactions can be defined as the neuron's ability to affect the neighboring neurons by either inhibiting or exciting their activity. We investigated the effect of the spatial and temporal domains on binocular interactions and BS under the lateral masking paradigm and how BS would be affected by lateral interactions via a lateral masking experiment. The two temporal alternative forced-choice (2TAFC) method was used. The stimuli consisted of a central vertically oriented Gabor target and high-contrast Gabor flankers positioned in two configurations (orthogonal or collinear) with target-flanker separations of either 2 or 3 wavelengths (λ), presented at 4 different presentation times (40, 80, 120, and 200 ms) using a different order of measurements across the different experiments. Opaque lenses were used to control the monocular and binocular vision. BS is absent at close distances (2λ), depending on the presentation time's order, for the collinear but not for the orthogonal configuration. However, BS exists at more distant flankers (collinear and orthogonal, 3λ). BS is not uniform (1.4); it depends on the stimulus condition, the presentation times, the order, and the method that was used to control the monocular and binocular vision.

Structural and spontaneous functional brain changes in visual and oculomotor areas identified by functional localization task in intermittent exotropia children

Intermittent exotropia (IXT) is characterizedby an intermittent outward deviation of the eyes. Yet, the neural substrates associated with IXT are not fully understood. This study investigated brain structure and spontaneous functional activity changes in children with IXT. All participants underwent detailed ophthalmological examinations and multimodal magnetic resonance imaging (MRI) scanning. During functional scanning, binocular visual stimuli were presented to subjects to determine brain areas involved in visual and oculomotor processing. Regions of interest(ROI) were subsequently selected based on functional activation to investigate brain structural and spontaneous functional differences between IXT children and healthy controls (HCs) using small volume correction (SVC). Reduced gray matter density (GMD) was found in the right frontal eye field (FEF) and bilateral inferior parietal lobe (IPL) in IXT children compared with HCs. Besides, reduced fractional amplitude of low-frequency fluctuations (fALFF) values were observed in the left lingual gyrus, right inferior occipital gyrus (IOG), bilateral IPL, and bilateral cerebellum in the IXT children compared to the HCs. IXT children with worse eye position control ability exhibited lower GMD and fALFF values in these areas. Finally, resting state functional connectivity (RSFC) was reduced in frontoparietal oculomotor processing areas in IXT children compared to HCs. In addition, increased cortical thickness was found in the right visual areas and bilateral IPL. These results showed that IXT-related structural and functional brain abnormalities occurred in childhood and may be related to underlying neuropathological mechanisms.

Impact of monocular vs. binocular contrast and blur on the range of functional stereopsis

Stereopsis depends on the smallest stereo threshold (lower limit) and the upper fusion limit. While studies have shown that the lower limit worsens with reduced contrast and blur, more strongly in monocular than in binocular conditions, the effect on the upper limit remains uncertain. Here, we assess the impact of contrast and blur on the range of the disparity sensitivity function (DSF) in a stereo letter recognition task. Subjects had to identify the stereo letters embedded in a random dot stereogram, and adaptive staircases were used to estimate the two limits. Five subjects performed the experiment at baseline contrast (100%), with different contrast (32% and 10%) and blur (+0.75DS and +1.25DS) in monocular and binocular degradation. We proposed three possible outcomes: 1) the range collapses in both directions 2) the lower limit threshold reduces, but the upper limit is not affected 3) the threshold for both limits increases and the range remains the same. We found that the curve for both limits was lowpass in shape, resulting in a smaller range at higher SFs. The results were similar to the first prediction, where the threshold for the lower limit increased while the upper limit was reduced at lower contrast and higher blur. The shrinkage of DSF is significant in monocular conditions. However, with blur, there was inter-subject variability. A simple cross-correlation stereo-matching algorithm was used to quantify the effect of contrast and blur. The results were consistent with the behavioral result that the range of DSF decreases with image degradation.

Functional changes in fusional vergence-related brain areas and correlation with clinical features in intermittent exotropia using functional magnetic resonance imaging

To explore the functional changes of the frontal eye field (FEF) and relevant brain regions and its role in the pathogenesis of intermittent exotropia (IXT) children via functional magnetic resonance imaging (fMRI). Twenty-four IXT children (mean age, 11.83 ± 1.93 years) and 28 normal control (NC) subjects (mean age, 11.11 ± 1.50 years) were recruited. During fMRI scans, the IXT children and NCs were provided with static visual stimuli (to evoke sensory fusion) and dynamic visual stimuli (to evoke motor fusion and vergence eye movements) with binocular disparity. Brain activation in the relevant brain regions and clinical characteristics were evaluated. Group differences of brain activation and brain-behavior correlations were investigated. For dynamic and static visual disparity relative to no visual disparity, reduced brain activation in the right FEF and right inferior occipital gyrus (IOG), and increased brain activation in the left middle temporal gyrus complex (MT+) were found in the IXT children compared with NCs. Significant positive correlations between the fusional vergence amplitude and the brain activation values were found in the right FEF, right IPL, and left cerebellum in the NC group. Positive correlations between brain activation values and Newcastle Control Scores (NCS) were found in the left MT+ in the IXT group. For dynamic visual disparity relative to static visual disparity, reduced brain activation in the right middle occipital gyrus, left cerebellum, and bilateral IPL was found in the IXT children compared with NCs. Significant positive correlations between brain activation values and the fusional vergence amplitude were found in the right FEF and right cerebellum in the NC group. Negative correlations between brain activation values and NCS were found in the right middle occipital gyrus, right cerebellum, left IPL, and right FEF in the IXT group. These results suggest that the reduced brain activation in the right FEF, left IPL, and cerebellum may play an important role in the pathogenesis of IXT by influencing fusional vergence function. While the increased brain activation in the left MT+ may compensate for this dysfunction in IXT children.

Learning to see in depth

Stereopsis provides us with a vivid impression of the depth and distance of objects in our 3- dimensional world. Stereopsis is important for a number of everyday visual tasks, including (but not limited to) reaching and grasping, fine visuo-motor control, and navigating in our world. This review briefly discusses the neural substrate for normal binocular vision and stereopsis and its development in primates; outlines some of the issues and limitations of stereopsis tests and examines some of the factors that limit the typical development of stereopsis and the causes and consequences of stereo-deficiency and stereo-blindness. Finally, we review several approaches to improving or recovering stereopsis in both neurotypical individuals and those with stereo-deficiency and stereo-blindness and outline some emerging strategies for improving stereopsis.

Unusual Mathematical Approaches Untangle Nervous Dynamics

The massive amount of available neurodata suggests the existence of a mathematical backbone underlying neuronal oscillatory activities. For example, geometric constraints are powerful enough to define cellular distribution and drive the embryonal development of the central nervous system. We aim to elucidate whether underrated notions from geometry, topology, group theory and category theory can assess neuronal issues and provide experimentally testable hypotheses. The Monge’s theorem might contribute to our visual ability of depth perception and the brain connectome can be tackled in terms of tunnelling nanotubes. The multisynaptic ascending fibers connecting the peripheral receptors to the neocortical areas can be assessed in terms of knot theory/braid groups. Presheaves from category theory permit the tackling of nervous phase spaces in terms of the theory of infinity categories, highlighting an approach based on equivalence rather than equality. Further, the physical concepts of soft-matter polymers and nematic colloids might shed new light on neurulation in mammalian embryos. Hidden, unexpected multidisciplinary relationships can be found when mathematics copes with neural phenomena, leading to novel answers for everlasting neuroscientific questions. For instance, our framework leads to the conjecture that the development of the nervous system might be correlated with the occurrence of local thermal changes in embryo–fetal tissues.

Stimulating both eyes with matching stimuli enhances V1 responses

Neurons in the primary visual cortex (V1) of primates play a key role in combining monocular inputs to form a binocular response. Although much has been gleaned from studying how V1 responds to discrepant (dichoptic) images, equally important is to understand how V1 responds to concordant (dioptic) images in the two eyes. Here, we investigated the extent to which concordant, balanced, zero-disparity binocular stimulation modifies V1 responses to varying stimulus contrast using intracranial multielectrode arrays. On average, binocular stimuli evoked stronger V1 activity than their monocular counterparts. This binocular facilitation scaled most proportionately with contrast during the initial transient. As V1 responses evolved, additional contrast-mediated dynamics emerged. Specifically, responses exhibited longer maintenance of facilitation for lower contrast and binocular suppression at high contrast. These results suggest that V1 processes concordant stimulation of both eyes in at least two sequential steps: initial response enhancement followed by contrast-dependent control of excitation.

Transfer of Perceptual Learning From Local Stereopsis to Global Stereopsis in Adults With Amblyopia: A Preliminary Study

It has long been debated whether the analysis of global and local stereoscopic depth is performed by a single system or by separate systems. Global stereopsis requires the visual system to solve a complex binocular matching problem to obtain a coherent percept of depth. In contrast, local stereopsis requires only a simple matching of similar image features. In this preliminary study, we recruited five adults with amblyopia who lacked global stereopsis and trained them on a computerized local stereopsis depth task for an average of 12 h. Three out of five (60%) participants recovered fine global stereoscopic vision through training. Those who recovered global stereopsis reached a learning plateau more quickly on the local stereopsis task, and they tended to start the training with better initial local stereopsis performance, to improve more on local stereopsis with training, and to have less severe amblyopia. The transfer of learning from local stereopsis to global stereopsis is compatible with an interacting two-stage model.

Recent understanding of binocular vision in the natural environment with clinical implications

Technological advances in recent decades have allowed us to measure both the information available to the visual system in the natural environment and the rich array of behaviors that the visual system supports. This review highlights the tasks undertaken by the binocular visual system in particular and how, for much of human activity, these tasks differ from those considered when an observer fixates a static target on the midline. The everyday motor and perceptual challenges involved in generating a stable, useful binocular percept of the environment are discussed, together with how these challenges are but minimally addressed by much of current clinical interpretation of binocular function. The implications for new technology, such as virtual reality, are also highlighted in terms of clinical and basic research application.

Nearby contours abolish the binocular advantage

That binocular viewing confers an advantage over monocular viewing for detecting isolated low luminance or low contrast objects, has been known for well over a century; however, the processes involved in combining the images from the two eyes are still not fully understood. Importantly, in natural vision, objects are rarely isolated but appear in context. It is well known that nearby contours can either facilitate or suppress detection, depending on their distance from the target and the global configuration. Here we report that at close distances collinear (but not orthogonal) flanking contours suppress detection more under binocular compared to monocular viewing, thus completely abolishing the binocular advantage, both at threshold and suprathreshold levels. In contrast, more distant flankers facilitate both monocular and binocular detection, preserving a binocular advantage up to about four times the detection threshold. Our results for monocular and binocular viewing, for threshold contrast discrimination without nearby flankers, can be explained by a gain control model with uncertainty and internal multiplicative noise adding additional constraints on detection. However, in context with nearby flankers, both contrast detection threshold and suprathreshold contrast appearance matching require the addition of both target-to-target and flank-to-target interactions occurring before the site of binocular combination. To test an alternative model, in which the interactions occur after the site of binocular combination, we performed a dichoptic contrast matching experiment, with the target presented to one eye, and the flanks to the other eye. The two models make very different predictions for abutting flanks under dichoptic conditions. Interactions after the combination site predict that the perceived contrast of the flanked target will be strongly suppressed, while interactions before the site predict the perceived contrast will be more or less veridical. The data are consistent with the latter model, strongly suggesting that the interactions take place before the site of binocular combination.