High-fidelity eye, head, body, and world tracking with a wearable device

We describe the design and performance of a high-fidelity wearable head-, body-, and eye-tracking system that offers significant improvement over previous such devices. This device's sensors include a binocular eye tracker, an RGB-D scene camera, a high-frame-rate scene camera, and two visual odometry sensors, for a total of ten cameras, which we synchronize and record from with a data rate of over 700 MB/s. The sensors are operated by a mini-PC optimized for fast data collection, and powered by a small battery pack. The device records a subject's eye, head, and body positions, simultaneously with RGB and depth data from the subject's visual environment, measured with high spatial and temporal resolution. The headset weighs only 1.4 kg, and the backpack with batteries 3.9 kg. The device can be comfortably worn by the subject, allowing a high degree of mobility. Together, this system overcomes many limitations of previous such systems, allowing high-fidelity characterization of the dynamics of natural vision.

Rendering algorithms for aberrated human vision simulation

Vision-simulated imagery-the process of generating images that mimic the human visual system-is a valuable tool with a wide spectrum of possible applications, including visual acuity measurements, personalized planning of corrective lenses and surgeries, vision-correcting displays, vision-related hardware development, and extended reality discomfort reduction. A critical property of human vision is that it is imperfect because of the highly influential wavefront aberrations that vary from person to person. This study provides an overview of the existing computational image generation techniques that properly simulate human vision in the presence of wavefront aberrations. These algorithms typically apply ray tracing with a detailed description of the simulated eye or utilize the point-spread function of the eye to perform convolution on the input image. Based on the description of the vision simulation techniques, several of their characteristic features have been evaluated and some potential application areas and research directions have been outlined.

Characterising and dissecting human perception of scene complexity

Humans can effortlessly assess the complexity of the visual stimuli they encounter. However, our understanding of how we do this, and the relevant factors that result in our perception of scene complexity remain unclear; especially for the natural scenes in which we are constantly immersed. We introduce several new datasets to further understanding of human perception of scene complexity. Our first dataset (VISC-C) contains 800 scenes and 800 corresponding two-dimensional complexity annotations gathered from human observers, allowing exploration for how complexity perception varies across a scene. Our second dataset, (VISC-CI) consists of inverted scenes (reflection on the horizontal axis) with corresponding complexity maps, collected from human observers. Inverting images in this fashion is associated with destruction of semantic scene characteristics when viewed by humans, and hence allows analysis of the impact of semantics on perceptual complexity. We analysed perceptual complexity from both a single-score and a two-dimensional perspective, by evaluating a set of calculable and observable perceptual features based upon grounded psychological research (clutter, symmetry, entropy and openness). We considered these factors' relationship to complexity via hierarchical regressions analyses, tested the efficacy of various neural models against our datasets, and validated our perceptual features against a large and varied complexity dataset consisting of nearly 5000 images. Our results indicate that both global image properties and semantic features are important for complexity perception. We further verified this by combining identified perceptual features with the output of a neural network predictor capable of extracting semantics, and found that we could increase the amount of explained human variance in complexity beyond that of low-level measures alone. Finally, we dissect our best performing prediction network, determining that artificial neurons learn to extract both global image properties and semantic details from scenes for complexity prediction. Based on our experimental results, we propose the "dual information" framework of complexity perception, hypothesising that humans rely on both low-level image features and high-level semantic content to evaluate the complexity of images.

Color Vision Deficiency Survey in Anatomic Pathology

OBJECTIVES

To learn what color vision-deficient pathologists and cytotechnologists consider their most significant problems and advantages as well as any accommodations.

METHODS

An anonymous online survey developed for practicing pathologists and cytotechnologists regarding their experiences with stains was sent to the members of 4 national societies.

RESULTS

We received 377 responses. Twenty-three people, all men, identified themselves as color vision deficient, with 22 reporting red-green color vision deficiency and 1 reporting uncertain type. Eight pathologists and cytotechnologists indicated that they thought that their color vision deficiency conferred advantages to them, including a greater appreciation of morphology, with less confusion resulting from variations in stain quality or intensity. Nineteen pathologists and cytotechnologists thought that their color vision deficiency conferred disadvantages; the most common disadvantages stated were the identification of eosinophils and acid-fast bacilli. Other difficulties included interpretation of RBCs and nucleoli and sometimes Alcian blue, Brown and Brenn, Congo red, crystal violet, Fite, Giemsa, mucicarmine, periodic acid-Schiff, and fluorescence in situ hybridization stains. Only 2 of the color vision-deficient pathologists and cytotechnologists found digital slides more difficult than glass slides.

CONCLUSIONS

Color vision-deficient pathologists and cytotechnologists report that they have developed approaches to viewing slides that do not compromise their interpretations. Digital pathology may provide several approaches for aiding color vision-deficient pathologists with the interpretation of certain stains.

Effective connectivity in subcortical visual structures in de novo Patients with Parkinson's Disease

BACKGROUND

Parkinson's disease (PD) manifests with the appearance of non-motor symptoms before motor symptoms onset. Among these, dysfunctioning visual structures have recently been reported to occur at early disease stages.

OBJECTIVE

This study addresses effective connectivity in the visual network of PD patients.

METHODS

Using functional MRI and dynamic causal modeling analysis, we evaluated the connectivity between the superior colliculus, the lateral geniculate nucleus and the primary visual area V1 in de novo untreated PD patients (n = 22). A subset of the PD patients (n = 8) was longitudinally assessed two times at two months and at six months after starting dopaminergic treatment. Results were compared to those of age-matched healthy controls (n = 22).

RESULTS

Our results indicate that the superior colliculus drives cerebral activity for luminance contrast processing both in healthy controls and untreated PD patients. The same effective connectivity was observed with neuromodulatory differences in terms of neuronal dynamic interactions. Our main findings were that the modulation induced by luminance contrast changes of the superior colliculus connectivity (self-connectivity and connectivity to the lateral geniculate nucleus) was inhibited in PD patients (effect of contrast: p = 0.79 and p = 0.77 respectively). The introduction of dopaminergic medication in a subset (n = 8) of the PD patients failed to restore the effective connectivity modulation observed in the healthy controls.

INTERPRETATION

The deficits in luminance contrast processing in PD was associated with a deficiency in connectivity adjustment from the superior colliculus to the lateral geniculate nucleus and to V1. No differences in cerebral blood flow were observed between controls and PD patients suggesting that the deficiency was at the neuronal level. Administration of a dopaminergic treatment over six months was not able to normalize the observed alterations in inter-regional coupling. These findings highlight the presence of early dysfunctions in primary visual areas, which might be used as early markers of the disease.

The electrophysiological response to polarization-modulated patterned visual stimuli

Recent reports indicate that the subjective ability of humans to discriminate between polarization E-vector orientations approaches that of many invertebrates. Here, we show that polarization-modulated patterned stimuli generate an objectively recordable electrophysiological response in humans with normal vision. We investigated visual evoked potential (VEP) and electroretinographic (ERG) responses to checkerboard patterns defined solely by their polarization E-vector orientation alternating between ± 45°. Correcting for multiple comparisons, paired-samples t-tests were conducted to assess the significance of post-stimulus deflections from baseline measures of noise. Using standard check pattern sizes for clinical electrophysiology, and a pattern-reversal protocol, participants showed a VEP response to polarization-modulated patterns (PolVEP) with a prominent and consistent positive component near 150 ms (p < 0.01), followed by more variable negative components near 200 ms and 300 ms. The effect was unrecordable with visible wavelengths >550 nm. Further, pseudo-depolarization negated the responses, while control studies provided confirmatory evidence that the PolVEP response was not the product of luminance artefacts. Polarization-modulated patterns did not elicit a recordable ERG response. The possible origins of the PolVEP signals, and the absence of recordable ERG signals, are discussed. We conclude that evoked cortical responses to polarization-modulated patterns provide an objective measure of foveal function, suitable for both humans and non-human primates with equivalent macular anatomy.

Neural circuits for long-range color filling-in

Surface color appearance depends on both local surface chromaticity and global context. How are these inter-dependencies supported by cortical networks? Combining functional imaging and psychophysics, we examined if color from long-range filling-in engages distinct pathways from responses caused by a field of uniform chromaticity. We find that color from filling-in is best classified and best correlated with appearance by two dorsal areas, V3A and V3B/KO. In contrast, a field of uniform chromaticity is best classified by ventral areas hV4 and LO. Dynamic causal modeling revealed feedback modulation from area V3A to areas V1 and LO for filling-in, contrasting with feedback from LO modulating areas V1 and V3A for a matched uniform chromaticity. These results indicate a dorsal stream role in color filling-in via feedback modulation of area V1 coupled with a cross-stream modulation of ventral areas suggesting that local and contextual influences on color appearance engage distinct neural networks.

How lateral inhibition and fast retinogeniculo-cortical oscillations create vision: A new hypothesis

The role of the physiological processes involved in human vision escapes clarification in current literature. Many unanswered questions about vision include: 1) whether there is more to lateral inhibition than previously proposed, 2) the role of the discs in rods and cones, 3) how inverted images on the retina are converted to erect images for visual perception, 4) what portion of the image formed on the retina is actually processed in the brain, 5) the reason we have an after-image with antagonistic colors, and 6) how we remember space. This theoretical article attempts to clarify some of the physiological processes involved with human vision. The global integration of visual information is conceptual; therefore, we include illustrations to present our theory. Universally, the eyeball is 2.4cm and works together with membrane potential, correspondingly representing the retinal layers, photoreceptors, and cortex. Images formed within the photoreceptors must first be converted into chemical signals on the photoreceptors' individual discs and the signals at each disc are transduced from light photons into electrical signals. We contend that the discs code the electrical signals into accurate distances and are shown in our figures. The pre-existing oscillations among the various cortices including the striate and parietal cortex, and the retina work in unison to create an infrastructure of visual space that functionally "places" the objects within this "neural" space. The horizontal layers integrate all discs accurately to create a retina that is pre-coded for distance. Our theory suggests image inversion never takes place on the retina, but rather images fall onto the retina as compressed and coiled, then amplified through lateral inhibition through intensification and amplification on the OFF-center cones. The intensified and amplified images are decompressed and expanded in the brain, which become the images we perceive as external vision.

SUMMARY

This is a theoretical article presenting a novel hypothesis about the physiological processes in vision, and expounds upon the visual aspect of two of our previously published articles, "A unified 3D default space consciousness model combining neurological and physiological processes that underlie conscious experience", and "Functional representation of vision within the mind: A visual consciousness model based in 3D default space." Currently, neuroscience teaches that visual images are initially inverted on the retina, processed in the brain, and then conscious perception of vision happens in the visual cortex. Here, we propose that inversion of visual images never takes place because images enter the retina as coiled and compressed graded potentials that are intensified and amplified in OFF-center photoreceptors. Once they reach the brain, they are decompressed and expanded to the original size of the image, which is perceived by the brain as the external image. We adduce that pre-existing oscillations (alpha, beta, and gamma) among the various cortices in the brain (including the striate and parietal cortex) and the retina, work together in unison to create an infrastructure of visual space thatfunctionally "places" the objects within a "neural" space. These fast oscillations "bring" the faculties of the cortical activity to the retina, creating the infrastructure of the space within the eye where visual information can be immediately recognized by the brain. By this we mean that the visual (striate) cortex synchronizes the information with the photoreceptors in the retina, and the brain instantaneously receives the already processed visual image, thereby relinquishing the eye from being required to send the information to the brain to be interpreted before it can rise to consciousness. The visual system is a heavily studied area of neuroscience yet very little is known about how vision occurs. We believe that our novel hypothesis provides new insights into how vision becomes part of consciousness, helps to reconcile various previously proposed models, and further elucidates current questions in vision based on our unified 3D default space model. Illustrations are provided to aid in explaining our theory.

Brain activation difference evoked by different binocular disparities of stereograms: An fMRI study

The binocular disparity of two retina images is a main cue of stereoscopic vision. However, the global dependency between brain response and binocular disparity still remains unclear. Here, we used functional Magnetic Resonance Imaging (fMRI) to identify stereopsis-related brain regions with a modified Random Dot Stereogram (RDS) and plotted the activation variation curves under different disparity size. In order to eliminate the confounding shape difference between the stereogram and the plane, commonly seen in RDS, we modified the RDS to a checkerboard version. We found that V3A, V7 and MT+/V5 in dorsal visual stream were activated in stereoscopic experiment, while little activation was found in ventral visual regions. According to the activation trends, 13 subjects were divided into three groups: 5 subjects with turning points (a shift from increased to decreased activation), 5 subjects without turning points and 3 subjects with activation unrelated to disparity. We inferred that the dorsal visual stream primarily processes spatial depth information, rather than shape information.