History constrains the evolution of efficient color naming, enabling historical inference

Color naming in natural languages is not arbitrary: It reflects efficient partitions of perceptual color space [T. Regier, P. Kay, N. Khetarpal, Proc. Natl. Acad. Sci. U.S.A. 104, 1436-1441 (2007)] modulated by the relative needs to communicate about different colors [C. Twomey, G. Roberts, D. Brainard, J. Plotkin, Proc. Natl. Acad. Sci. U.S.A. 118, e2109237118 (2021)]. These psychophysical and communicative constraints help explain why languages around the world have remarkably similar, but not identical, mappings of colors to color terms. Languages converge on a small set of efficient representations.But languages also evolve, and the number of terms in a color vocabulary may change over time. Here we show that history, i.e. the existence of an antecedent color vocabulary, acts as a nonadaptive constraint that biases the choice of efficient solution as a language transitions from a vocabulary of size [Formula: see text] to [Formula: see text] terms. Moreover, as efficient vocabularies evolve to include more terms they explore a smaller fraction of all possible efficient vocabularies compared to equally sized vocabularies constructed de novo. This path dependence of the cultural evolution of color naming presents an opportunity. Historical constraints can be used to reconstruct ancestral color vocabularies, allowing us to answer long-standing questions about the evolutionary sequences of color words, and enabling us to draw inferences from phylogenetic patterns of language change.

Orthogonal neural representations support perceptual judgements of natural stimuli

In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations support stable perception of objects and features despite the tremendous richness of natural visual scenes.

Poster Session I: Vernier thresholds of a Poisson-noise-limited computational observer with and without fixational eye movements

Vernier acuity is a fundamental measure of spatial vision. We modeled how stimulus encoding by the cones limits Vernier acuity. We determined Vernier thresholds for a computational observer that had access to the Poisson-distributed cone photopigment excitations. The observer also had access to the cone mosaic layout and the stimulus possibilities on each trial. We varied stimulus contrast (100%, 50%, 22%, 11% Michelson contrast) and duration (2, 4, 9, 18 stimulus frames; frame duration 8.33 ms) while fixing other stimulus properties (foveal viewing; two achromatic vertical bars; length 6.2 arcmin; width 1 arcmin; vertical gap 0.1 arcmin). When the retinal image is stationary, Vernier thresholds depend jointly on contrast and duration through contrast energy: squared contrast times duration. Introducing fixational drift eye movements impairs performance, when the information about eye path is not accounted for by the computational observer. When the path of fixational drift is made available and used ideally, there is no noticeable difference with the stationary case. The lack of improvement when the path of fixational drift is known exactly may reflect the high-density of foveal cones relative to the optical point spread function and the fact that we did not introduce temporal filtering by the visual system. Our results suggest the possibility of a rich interaction between optics, cone sampling, fixational eye movements, post-receptoral filtering and visual performance.

Poster Session II: Computational analysis of the effect of cone temporal filtering on detection threshold with and without retinal motion

Retinal stimulus motion can increase visual acuity, but recent experimental evidence indicates that for briefly presented stimuli this benefit does not always occur (Braun et al, VSS 2023). To understand this effect, we modeled how the temporally low-pass filtering that occurs within each cone is expected to impact visual performance. We simulated a grating (10 cpd Gabor) detection task in which the stimulus was present for two 15 msec frames. We used the ISETBio software to compute cone excitations and cone photocurrent responses to the stimulus, for different contrasts and retinal positions, and used linear SVM classifiers to estimate computational-observer detection thresholds. We examined three retinal-motion conditions: 1) stimulus stabilized on the retina; 2) stimulus shifted 0.5 cycle orthogonal to grating orientation across the two frames; 3) stimulus shifted 1 cycle. Across all three conditions, detection threshold estimated on the basis of cone excitations varied little. Threshold estimated on the basis of photocurrent, however, was highest for the 0.5 cycle shift, and about the same for the no shift and 1 cycle shift conditions. Qualitatively, this result recapitulates the findings of Braun et al. and suggests that those findings may be understood as a consequence of the initial visual encoding. We note, however, that we analyzed detection rather than acuity; planned work will attempt to bring the computations into more direct contact with the experimental results.

Contributed Session II: The relationship between temporal summation at detection threshold and fixational eye movements

We studied the relationship between the threshold temporal summation of increment pulses and fixational eye-movements. Six participants completed a 2AFC increment detection task. Stimuli were 0.16 x 2.2 arcmin increments of 543 nm light presented via an AOSLO with a 60 Hz frame rate. Stimuli for temporal integration were two single frame presentations with a 16 ms (consecutive frames), 33 ms, 100 ms, or 300 ms inter-stimulus interval (ISI). Data were also collected for increments presented on a single frame. Stimuli were presented in either world-fixed coordinates (natural retinal image motion) or were stabilised on the retina. There were large differences in overall sensitivity across individuals, but the time-course of performance change with ISI was similar across participants. Thresholds for ISI=33 ms were close to performance with two consecutive frames, suggesting complete summation of light energy; whereas thresholds for ISI=300 ms were closer to the single-frame case, suggesting limited summation; and thresholds for ISI=100 ms were intermediate, suggesting residual summation. The effect of ISI on threshold was similar for stabilised stimuli and natural viewing, but there was a small trend towards lower thresholds for stabilised stimuli at short ISI and vice-versa at long ISI. We plan to present our results in the context of an ideal observer calculation that may clarify how the initial visual encoding, including temporal summation within cones, shapes performance.

Contributed Session II: Computational modeling of shift in unique yellow for small stimuli

Unique yellow (UY) is largely invariant to L:M cone proportion for spatially-extended stimuli in healthy trichromats. However, a recent adaptive-optics-based study by Boehm et al. reveals that when stimulus size is reduced to a few arcmin, color appearance depends on the local L:M proportion in the patch of the retina on which the stimulus was imaged. We aimed to determine if such findings are consistent with a normative account of visual processing. A series of 3.5 and 10 arcmin stimuli were simulated as isoluminant mixtures of 540 and 680 nm primaries. We modeled sensory encoding under adaptive-optics conditions using the open-source software ISETBio, for simulated retinal cone mosaics with varying local L:M proportions. The resultant cone excitations were decoded using a Bayesian image reconstruction algorithm (Zhang et al., 2022). For the 3.5 arcmin stimuli, as local L:M proportion decreased, the 540 nm component of the reconstructions increased relative to the 680 nm component. This is qualitatively consistent with the experimental observations of Boehm et al. For 10 arcmin stimuli, in contrast, reconstructions were stable across variation in local L:M cone proportion. Notably, reconstructions depend not only on the local L:M cone proportion, but also on the proportion in the immediately surrounding retina, leading to a testable prediction. The computational observations frame the experimental results as a normative consequence of visual processing.

Contributed Session III: In vivo calcium imaging of macaque foveolar retinal ganglion cells reveals spatiochromatic receptive field properties

Here, we optically record responses to spatial and chromatic stimuli using a calcium indicator in the living macaque eye to characterize the receptive field (RF) properties of retinal ganglion cells (RGCs) serving the foveal center. GCaMP6s was expressed in three female macaques. Adaptive optics ophthalmoscopy was used to image fluorescence (488nm ex, 520/35nm em) from RGCs whose RF centers were driven by cones in the central 36 arcmin of the fovea and additional RGCs driven by cones in the central 6 arcmin of the foveola. Using cone isolating and luminance flicker (1.3deg, 0.15Hz, LED 420nm, 530nm, 660nm), we derived cone weights in over 250 RGCs. Using drifting gratings (1.9deg, 6Hz, 4-50c/deg, 561nm), we derived the spatial frequency responses of 15 L vs. M cone opponent RGCs at the foveolar center. Employing computational modeling (ISETbio), we inferred the full spatial difference of gaussians center and surround structure for those 15 cells. Of the 34 foveolar RGCs, 44% exhibited L vs. M cone opponency, 15% were L+M ON, 6% were -L-M OFF, and 35% showed only L or only M responses. The spatial frequency response functions of 12/15 L vs. M opponent cells peaked at high spatial frequencies (25-40c/deg) and had a strong bandpass characteristic. Our model indicates that the responses of all 15 L vs. M opponent cells are consistent with single cone input to their RF centers and that our data are consistent with extrafoveal data when the blurring of the optics is accounted for.

Automated Assessment of Photoreceptor Visibility in Adaptive Optics Split-Detection Images Using Edge Detection

Purpose

Adaptive optics scanning laser ophthalmoscopy (AOSLO) is a high-resolution imaging modality that allows measurements of cellular-level retinal changes in living patients. In retinal diseases, the visibility of photoreceptors in AOSLO images is affected by pathology, patient motion, and optics, which can lead to variability in analyses of the photoreceptor mosaic. Current best practice for AOSLO mosaic quantification requires manual assessment of photoreceptor visibility across overlapping images, a laborious and time-consuming task.

Methods

We propose an automated measure for quantification of photoreceptor visibility in AOSLO. Our method detects salient edge features, which can represent visible photoreceptor boundaries in each image. We evaluate our measure against two human graders and two standard automated image quality assessment algorithms.

Results

We evaluate the accuracy of pairwise ordering (PO) and the correlation of ordinal rankings (ORs) of photoreceptor visibility in 29 retinal regions, taken from five subjects with choroideremia. The proposed measure had high association with manual assessments (Grader 1: PO = 0.71, OR = 0.61; Grader 2: PO = 0.67, OR = 0.62), which is comparable with intergrader reliability (PO = 0.76, OR = 0.75) and outperforms the top standard approach (PO = 0.57; OR = 0.46).

Conclusions

Our edge-based measure can automatically assess photoreceptor visibility and order overlapping images within AOSLO montages. This can significantly reduce the manual labor required to generate high-quality AOSLO montages and enables higher throughput for quantitative studies of photoreceptors.

Translational Relevance

Automated assessment of photoreceptor visibility allows us to more rapidly quantify photoreceptor morphology in the living eye. This has applications to ophthalmic medicine by allowing detailed characterization of retinal degenerations, thus yielding potential biomarkers of treatment safety and efficacy.

Equivalent noise characterization of human lightness constancy

A goal of visual perception is to provide stable representations of task-relevant scene properties (e.g. object reflectance) despite variation in task-irrelevant scene properties (e.g. illumination and reflectance of other nearby objects). To study such stability in the context of the perceptual representation of lightness, we introduce a threshold-based psychophysical paradigm. We measure how thresholds for discriminating the achromatic reflectance of a target object (task-relevant property) in rendered naturalistic scenes are impacted by variation in the reflectance functions of background objects (task-irrelevant property), using a two-alternative forced-choice paradigm in which the reflectance of the background objects is randomized across the two intervals of each trial. We control the amount of background reflectance variation by manipulating a statistical model of naturally occurring surface reflectances. For low background object reflectance variation, discrimination thresholds were nearly constant, indicating that observers’ internal noise determines threshold in this regime. As background object reflectance variation increases, its effects start to dominate performance. A model based on signal detection theory allows us to express the effects of task-irrelevant variation in terms of the equivalent noise, that is relative to the intrinsic precision of the task-relevant perceptual representation. The results indicate that although naturally occurring background object reflectance variation does intrude on the perceptual representation of target object lightness, the effect is modest – within a factor of two of the equivalent noise level set by internal noise.

An image reconstruction framework for characterizing initial visual encoding

We developed an image-computable observer model of the initial visual encoding that operates on natural image input, based on the framework of Bayesian image reconstruction from the excitations of the retinal cone mosaic. Our model extends previous work on ideal observer analysis and evaluation of performance beyond psychophysical discrimination, takes into account the statistical regularities of the visual environment, and provides a unifying framework for answering a wide range of questions regarding the visual front end. Using the error in the reconstructions as a metric, we analyzed variations of the number of different photoreceptor types on human retina as an optimal design problem. In addition, the reconstructions allow both visualization and quantification of information loss due to physiological optics and cone mosaic sampling, and how these vary with eccentricity. Furthermore, in simulations of color deficiencies and interferometric experiments, we found that the reconstructed images provide a reasonable proxy for modeling subjects' percepts. Lastly, we used the reconstruction-based observer for the analysis of psychophysical threshold, and found notable interactions between spatial frequency and chromatic direction in the resulting spatial contrast sensitivity function. Our method is widely applicable to experiments and applications in which the initial visual encoding plays an important role.

Reflexive Eye Closure in Response to Cone and Melanopsin Stimulation: A Study of Implicit Measures of Light Sensitivity in Migraine

BACKGROUND AND OBJECTIVES

To quantify interictal photophobia in migraine with and without aura using reflexive eye closure as an implicit measure of light sensitivity and to assess the contribution of melanopsin and cone signals to these responses.

METHODS

Participants were screened to meet criteria for 1 of 3 groups: headache-free (HF) controls, migraine without aura (MO), and migraine with visual aura (MA). MO and MA participants were included if they endorsed ictal and interictal photophobia. Exclusion criteria included impaired vision, inability to collect usable pupillometry, and history of either head trauma or seizure. Participants viewed light pulses that selectively targeted melanopsin, the cones, or their combination during recording of orbicularis oculi EMG (OO-EMG) and blinking activity.

RESULTS

We studied 20 participants in each group. MA and MO groups reported increased visual discomfort to light stimuli (discomfort rating, 400% contrast, MA: 4.84 [95% confidence interval 0.33, 9.35]; MO: 5.23 [0.96, 9.50]) as compared to HF controls (2.71 [0, 6.47]). Time course analysis of OO-EMG and blinking activity demonstrated that reflexive eye closure was tightly coupled to the light pulses. The MA group had greater OO-EMG and blinking activity in response to these stimuli (EMG activity, 400% contrast: 42.9%Δ [28.4, 57.4]; blink activity, 400% contrast: 11.2% [8.8, 13.6]) as compared to the MO (EMG activity, 400% contrast: 9.9%Δ [5.8, 14.0]; blink activity, 400% contrast: 4.7% [3.5, 5.9]) and HF control (EMG activity, 400% contrast: 13.2%Δ [7.1, 19.3]; blink activity, 400% contrast: 4.5% [3.1, 5.9]) groups.

DISCUSSION

Our findings suggest that the intrinsically photosensitive retinal ganglion cells (ipRGCs), which integrate melanopsin and cone signals, provide the afferent input for light-induced reflexive eye closure in a photophobic state. Moreover, we find a dissociation between implicit and explicit measures of interictal photophobia depending on a history of visual aura in migraine. This implies distinct pathophysiology in forms of migraine, interacting with separate neural pathways by which the amplification of ipRGC signals elicits implicit and explicit signs of visual discomfort.

A computational observer model of spatial contrast sensitivity: Effects of photocurrent encoding, fixational eye movements, and inference engine

We have recently shown that the relative spatial contrast sensitivity function (CSF) of a computational observer operating on the cone mosaic photopigment excitations of a stationary retina has the same shape as human subjects. Absolute human sensitivity, however, is 5- to 10-fold lower than the computational observer. Here we model how additional known features of early vision affect the CSF: fixational eye movements and the conversion of cone photopigment excitations to cone photocurrents (phototransduction). For a computational observer that uses a linear classifier applied to the responses of a stimulus-matched linear filter, fixational eye movements substantially change the shape of the CSF by reducing sensitivity above 10 c/deg. For a translation-invariant computational observer that operates on the squared response of a quadrature-pair of linear filters, the CSF shape is little changed by eye movements, but there is a two fold reduction in sensitivity. Phototransduction dynamics introduce an additional two fold sensitivity decrease. Hence, the combined effects of fixational eye movements and phototransduction bring the absolute CSF of the translation-invariant computational observer to within a factor of 1 to 2 of the human CSF. We note that the human CSF depends on processing of the retinal representation by many thalamo-cortical neurons, which are individually quite noisy. Our modeling suggests that the net effect of post-retinal noise on contrast-detection performance, when considered at the neural population and behavioral level, is quite small: The inference mechanisms that determine the CSF, presumably in cortex, make efficient use of the information carried by the cone photocurrents of the fixating eye.

Optoretinography of individual human cone photoreceptors

Photoreceptors mediate the first step of vision, transducing light and passing signals to retinal neurons that ultimately send signals along the optic nerve to the brain. A functional deficiency in the photoreceptors, due to either congenital or acquired disease, can significantly affect an individual's sight and quality of life. Methods for quantifying the health and function of photoreceptors are essential for understanding both the progression of disease and the efficacy of treatment. Given that emerging treatments such as gene, stem cell, and small molecule therapy are designed to operate at the cellular scale, it is desirable to monitor function at the commensurate resolution of individual photoreceptors. Previously, non-invasive imaging methods for visualizing photoreceptor mosaic structure have been used to infer photoreceptor health, but these methods do not assess function directly. Conversely, most functional techniques, such as ERG and conventional microperimetry, measure function by aggregating the effects of signals from many photoreceptors. We have previously shown that stimulus-evoked intrinsic changes in intensity can be measured reliably in populations of cone photoreceptors in the intact human eye, a measurement we refer to more generally as the cone optoretinogram. Here we report that we can resolve the intensity optoretinogram at the level of individual cones. Moreover, we show that the individual cone optoretinogram exhibits two key signatures expected of a functional measure. First, responses in individual cones increase systematically as a function of stimulus irradiance. Second, we can use the amplitude of the functional response to middle wavelength (545 nm) light to separate the population of short-wavelength-sensitive (S) cones from the population of middle- and long-wavelength-sensitive (L and M) cones. Our results demonstrate the promise of optoretinography as a direct diagnostic measure of individual cone function in the living human eye.

Color Vision 2020: Introduction by the feature editors

Collapse

Automatic longitudinal montaging of adaptive optics retinal images using constellation matching

Adaptive optics (AO) scanning laser ophthalmoscopy offers a non-invasive approach for observing the retina at a cellular level. Its high resolution capabilities have direct application for monitoring and treating retinal diseases by providing quantitative assessment of cone health and density across time. However, accurate longitudinal analysis of AO images requires that AO images from different sessions be aligned, such that cell-to-cell correspondences can be established between timepoints. Such alignment is currently done manually, a time intensive task that is restrictive for large longitudinal AO studies. Automated longitudinal montaging for AO images remains a challenge because the intensity pattern of imaged cone mosaics can vary significantly, even across short timespans. This limitation prevents existing intensity-based montaging approaches from being accurately applied to longitudinal AO images. In the present work, we address this problem by presenting a constellation-based method for performing longitudinal alignment of AO images. Rather than matching intensity similarities between images, our approach finds structural patterns in the cone mosaics and leverages these to calculate the correct alignment. These structural patterns are robust to intensity variations, allowing us to make accurate longitudinal alignments. We validate our algorithm using 8 longitudinal AO datasets, each with two timepoints separated 6-12 months apart. Our results show that the proposed method can produce longitudinal AO montages with cell-to-cell correspondences across the full extent of the montage. Quantitative assessment of the alignment accuracy shows that the algorithm is able to find longitudinal alignments whose accuracy is on par with manual alignments performed by a trained rater.

Color, Pattern, and the Retinal Cone Mosaic

Textbook trichromacy accounts for human color vision in terms of spectral sampling by three classes of cone photoreceptors. This account neglects entangling of color and pattern information created by wavelength-dependent optical blur (chromatic aberrations) and interleaved spatial sampling of the retinal image by the three classes of cones. Recent experimental, computational, and neurophysiological work is now considering color and pattern vision at the elementary scale of daylight vison, that is at the scale of individual cones. The results provide insight about rich interactions between color and pattern vision as well as the role of the statistical structure of natural scenes in shaping visual processing.

Ray tracing 3D spectral scenes through human optics models

Scientists and engineers have created computations and made measurements that characterize the first steps of seeing. ISETBio software integrates such computations and data into an open-source software package. The initial ISETBio implementations modeled image formation (physiological optics) for planar or distant scenes. The ISET3d software described here extends that implementation, simulating image formation for three-dimensional scenes. The software system relies on a quantitative computer graphics program that ray traces the scene radiance through the physiological optics to the retinal irradiance. We describe and validate the implementation for several model eyes. Then, we use the software to quantify the impact of several physiological optics parameters on three-dimensional image formation. ISET3d is integrated with ISETBio, making it straightforward to convert the retinal irradiance into cone excitations. These methods help the user compute the predictions of optics models for a wide range of spatially rich three-dimensional scenes. They can also be used to evaluate the impact of nearby visual occlusion, the information available to binocular vision, or the retinal images expected from near-field and augmented reality displays.

A Conversation with Jacob Nachmias

We are sad to report that Professor Jacob (Jack) Nachmias passed away on March 2, 2019. Nachmias was born in Athens, Greece, on June 9, 1928. To escape the Nazis, he and his family came to the United States in 1939. He received his undergraduate degree from Cornell University and then an MA from Swarthmore College, where he worked with Hans Wallach and Wolfgang Kohler; his PhD in Psychology was from Harvard University. Nachmias spent the majority of his career as a Professor of Psychology at the University of Pennsylvania. He made fundamental contributions to our understanding of vision, most notably through the study of eye movements, the development of signal detection theory and forced-choice psychophysical methods, and the psychophysical characterization of spatial-frequency-selective visual channels. Nachmias' work was recognized by his election to the National Academy of Sciences and receipt of the Optical Society's Tillyer Award.

Computational-observer analysis of illumination discrimination

The spectral properties of the ambient illumination provide useful information about time of day and weather. We study the perceptual representation of illumination by analyzing measurements of how well people discriminate between illuminations across scene configurations. More specifically, we compare human performance to a computational-observer analysis that evaluates the information available in the isomerizations of cone photopigment in a model human photoreceptor mosaic. The performance of such an observer is limited by the Poisson variability of the number of isomerizations in each cone. The overall level of Poisson-limited computational-observer sensitivity exceeded that of human observers. This was modeled by increasing the amount of noise in the number of isomerizations of each cone. The additional noise brought the overall level of performance of the computational observer into the same range as that of human observers, allowing us to compare the pattern of sensitivity across stimulus manipulations. Key patterns of human performance were not accounted for by the computational observer. In particular, neither the elevation of illumination-discrimination thresholds for illuminant changes in a blue color direction (when thresholds are expressed in CIELUV ΔE units), nor the effects of varying the ensemble of surfaces in the scenes being viewed, could be accounted for by variation in the information available in the cone isomerizations.

A computational-observer model of spatial contrast sensitivity: Effects of wave-front-based optics, cone-mosaic structure, and inference engine

We present a computational-observer model of the human spatial contrast-sensitivity function based on the Image Systems Engineering Toolbox for Biology (ISETBio) simulation framework. We demonstrate that ISETBio-derived contrast-sensitivity functions agree well with ones derived using traditional ideal-observer approaches, when the mosaic, optics, and inference engine are matched. Further simulations extend earlier work by considering more realistic cone mosaics, more recent measurements of human physiological optics, and the effect of varying the inference engine used to link visual representations to psychophysical performance. Relative to earlier calculations, our simulations show that the spatial structure of realistic cone mosaics reduces the upper bounds on performance at low spatial frequencies, whereas realistic optics derived from modern wave-front measurements lead to increased upper bounds at high spatial frequencies. Finally, we demonstrate that the type of inference engine used has a substantial effect on the absolute level of predicted performance. Indeed, the performance gap between an ideal observer with exact knowledge of the relevant signals and human observers is greatly reduced when the inference engine has to learn aspects of the visual task. ISETBio-derived estimates of stimulus representations at various stages along the visual pathway provide a powerful tool for computing the limits of human performance.

Visual Function at the Atrophic Border in Choroideremia Assessed with Adaptive Optics Microperimetry

PURPOSE

Recent advances in retinal imaging allow visualization of structural abnormalities in retinal disease at the cellular level. This study used adaptive optics (AO) microperimetry to assess visual sensitivity with high spatial precision and to examine how function varies across 2 phenotypic features observed in choroideremia: atrophic lesion borders and outer retinal tubulations (ORTs).

DESIGN

Cross-sectional study.

PARTICIPANTS

Twelve choroideremia patients.

METHODS

A custom AO scanning light ophthalmoscope (AOSLO) equipped with both confocal and nonconfocal split-detection imaging methods was used to image the photoreceptor inner and outer segment mosaics. For AO microperimetry, circular 550-nm stimuli were presented through the AOSLO system; stimuli were either 9.6 or 38.3 arcmin² (approximately 60 or 15 times smaller than a Goldman III stimulus). Test locations were identified in structural images and stimuli were targeted to these locations using real-time retinal tracking combined with measurements of transverse chromatic aberration. Psychophysical detection thresholds were measured at the atrophic border in 12 patients. Additionally, visual sensitivity was probed along ORTs in 4 patients.

MAIN OUTCOME MEASURE

Visual sensitivity thresholds measured with AO microperimetry at retinal locations corresponding to structural phenotypes observed on AOSLO retinal images.

RESULTS

In choroideremia, sharp borders between intact central islands of the photoreceptor mosaic and complete atrophy of the outer retina and retinal pigment epithelium were observed in both split-detection and confocal structural images. Adaptive optics microperimetry at locations spanning these borders showed a commensurately sharp decrease in function, with readily measurable visual sensitivity on one side and dense scotoma on the other. These functional transitions often occurred over a distance smaller than the diameter of the Goldman III stimulus. Thresholds measured along ORTs showed dense scotoma over the tubule in all 4 participants, despite the visibility of remnant cone inner segments on the AO images.

CONCLUSIONS

Choroideremia patients exhibited sharp functional transitions that collocated with structural transitions from intact to severely degenerated retina. We found no evidence of visual sensitivity over ORTs. Measuring cone function with high resolution offered insight into disease mechanisms and may enable precise assessment of whether experimental therapies, such as gene therapy, provide a functional benefit.

The relative contribution of color and material in object selection

Object perception is inherently multidimensional: information about color, material, texture and shape all guide how we interact with objects. We developed a paradigm that quantifies how two object properties (color and material) combine in object selection. On each experimental trial, observers viewed three blob-shaped objects—the target and two tests—and selected the test that was more similar to the target. Across trials, the target object was fixed, while the tests varied in color (across 7 levels) and material (also 7 levels, yielding 49 possible stimuli). We used an adaptive trial selection procedure (Quest+) to present, on each trial, the stimulus test pair that is most informative of underlying processes that drive selection. We present a novel computational model that allows us to describe observers’ selection data in terms of (1) the underlying perceptual stimulus representation and (2) a color-material weight, which quantifies the relative importance of color vs. material in selection. We document large individual differences in the color-material weight across the 12 observers we tested. Furthermore, our analyses reveal limits on how precisely selection data simultaneously constrain perceptual representations and the color-material weight. These limits should guide future efforts towards understanding the multidimensional nature of object perception.

Much is known about how the visual system extracts information about individual object properties, such as color or material. Considerably less is known about how percepts of these properties interact to form a multidimensional object representation. We report the first quantitative analysis of how perceived color and material combine in object selection, using a task designed to reflect key aspects of how we use vision in real life. We introduce a computational model that describes observers’ selection behavior in terms of (1) how objects are represented in an underlying subjective perceptual color-material space and (2) how differences in perceived object color and material combine to guide selection. We find large individual differences in the degree to which observers select objects based on color relative to material: some base their selections almost entirely on color, some weight color and material nearly equally, and others rely almost entirely on material. A fine-grained analysis clarifies the limits on how precisely selection data may be leveraged to simultaneously understand the underlying perceptual representations on one hand and how the information about perceived color and material combine on the other. Our work provides a foundation for improving our understanding of visual computations in natural viewing.

Probing Computation in the Primate Visual System at Single-Cone Resolution

Daylight vision begins when light activates cone photoreceptors in the retina, creating spatial patterns of neural activity. These cone signals are then combined and processed in downstream neural circuits, ultimately producing visual perception. Recent technical advances have made it possible to deliver visual stimuli to the retina that probe this processing by the visual system at its elementary resolution of individual cones. Physiological recordings from nonhuman primate retinas reveal the spatial organization of cone signals in retinal ganglion cells, including how signals from cones of different types are combined to support both spatial and color vision. Psychophysical experiments with human subjects characterize the visual sensations evoked by stimulating a single cone, including the perception of color. Future combined physiological and psychophysical experiments focusing on probing the elementary visual inputs are likely to clarify how neural processing generates our perception of the visual world.

Illumination discrimination for chromatically biased illuminations: Implications for color constancy

We measured discrimination thresholds for illumination changes along different chromatic directions starting from chromatically biased reference illuminations. Participants viewed a Mondrian-papered scene illuminated by LED lamps. The scene was first illuminated by a reference illumination, followed by two comparisons. One comparison matched the reference (the target); the other (the test) varied from the reference, nominally either bluer, yellower, redder, or greener. The participant's task was to correctly select the target. A staircase procedure found thresholds for discrimination of an illumination change along each axis of chromatic change. Nine participants completed the task for five different reference illumination conditions (neutral, blue, yellow, red, and green). We find that relative discrimination thresholds for different chromatic directions of illumination change vary with the reference illumination. For the neutral reference, there is a trend for thresholds to be highest in the bluer illumination-change direction, replicating our previous reports of a "blue bias" for neutral reference illuminations. For the four chromatic references (blue, yellow, red, and green), the change in illumination toward the neutral reference is less well discriminated than changes in the other directions: a "neutral bias." The results have implications for color constancy: In considering the stability of surface appearance under changes in illumination, both the starting chromaticity of the illumination and direction of change must be considered, as well as the chromatic characteristics of the surface reflectance ensemble. They also suggest it will be worthwhile to explore whether and how the human visual system has internalized the statistics of natural illumination changes.

Computational luminance constancy from naturalistic images

The human visual system supports stable percepts of object color even though the light that reflects from object surfaces varies significantly with the scene illumination. To understand the computations that support stable color perception, we study how estimating a target object's luminous reflectance factor (LRF; a measure of the light reflected from the object under a standard illuminant) depends on variation in key properties of naturalistic scenes. Specifically, we study how variation in target object reflectance, illumination spectra, and the reflectance of background objects in a scene impact estimation of a target object's LRF. To do this, we applied supervised statistical learning methods to the simulated excitations of human cone photoreceptors, obtained from labeled naturalistic images. The naturalistic images were rendered with computer graphics. The illumination spectra of the light sources and the reflectance spectra of the surfaces in the scene were generated using statistical models of natural spectral variation. Optimally decoding target object LRF from the responses of a small learned set of task-specific linear receptive fields that operate on a contrast representation of the cone excitations yields estimates that are within 13% of the correct LRF. Our work provides a framework for evaluating how different sources of scene variability limit performance on luminance constancy.

Simulation of visual perception and learning with a retinal prosthesis

OBJECTIVE

The nature of artificial vision with a retinal prosthesis, and the degree to which the brain can adapt to the unnatural input from such a device, are poorly understood. Therefore, the development of current and future devices may be aided by theory and simulations that help to infer and understand what prosthesis patients see.

APPROACH

A biologically-informed, extensible computational framework is presented here to predict visual perception and the potential effect of learning with a subretinal prosthesis. The framework relies on optimal linear reconstruction of the stimulus from retinal responses to infer the visual information available to the patient. A simulation of the physiological optics of the eye and light responses of the major retinal neurons was used to calculate the optimal linear transformation for reconstructing natural images from retinal activity. The result was then used to reconstruct the visual stimulus during the artificial activation expected from a subretinal prosthesis in a degenerated retina, as a proxy for inferred visual perception.

MAIN RESULTS

Several simple observations reveal the potential utility of such a simulation framework. The inferred perception obtained with prosthesis activation was substantially degraded compared to the inferred perception obtained with normal retinal responses, as expected given the limited resolution and lack of cell type specificity of the prosthesis. Consistent with clinical findings and the importance of cell type specificity, reconstruction using only ON cells, and not OFF cells, was substantially more accurate. Finally, when reconstruction was re-optimized for prosthesis stimulation, simulating the greatest potential for learning by the patient, the accuracy of inferred perception was much closer to that of healthy vision.

SIGNIFICANCE

The reconstruction approach thus provides a more complete method for exploring the potential for treating blindness with retinal prostheses than has been available previously. It may also be useful for interpreting patient data in clinical trials, and for improving prosthesis design.

Neuronal population mechanisms of lightness perception

The way that humans and animals perceive the lightness of an object depends on its physical luminance as well as its surrounding context. While neuronal responses throughout the visual pathway are modulated by context, the relationship between neuronal responses and lightness perception is poorly understood. We searched for a neuronal mechanism of lightness by recording responses of neuronal populations in monkey primary visual cortex (V1) and area V4 to stimuli that produce a lightness illusion in humans, in which the lightness of a disk depends on the context in which it is embedded. We found that the way individual units encode the luminance (or equivalently for our stimuli, contrast) of the disk and its context is extremely heterogeneous. This motivated us to ask whether the population representation in either V1 or V4 satisfies three criteria: 1) disk luminance is represented with high fidelity, 2) the context surrounding the disk is also represented, and 3) the representations of disk luminance and context interact to create a representation of lightness that depends on these factors in a manner consistent with human psychophysical judgments of disk lightness. We found that populations of units in both V1 and V4 fulfill the first two criteria but that we cannot conclude that the two types of information in either area interact in a manner that clearly predicts human psychophysical measurements: the interpretation of our population measurements depends on how subsequent areas read out lightness from the population responses. NEW & NOTEWORTHY A core question in visual neuroscience is how the brain extracts stable representations of object properties from the retinal image. We searched for a neuronal mechanism of lightness perception by determining whether the responses of neuronal populations in primary visual cortex and area V4 could account for a lightness illusion measured using human psychophysics. Our results suggest that comparing psychophysics with population recordings will yield insight into neuronal mechanisms underlying a variety of perceptual phenomena.

Pulses of Melanopsin-Directed Contrast Produce Highly Reproducible Pupil Responses That Are Insensitive to a Change in Background Radiance

Purpose

To measure the pupil response to pulses of melanopsin-directed contrast, and compare this response to those evoked by cone-directed contrast and spectrally narrowband stimuli.

Methods

Three-second unipolar pulses were used to elicit pupil responses in human subjects across three sessions. Thirty subjects were studied in session 1, and most returned for sessions 2 and 3. The stimuli of primary interest were "silent substitution" cone- and melanopsin-directed modulations. Red and blue narrowband pulses delivered using the post-illumination pupil response (PIPR) paradigm were also studied. Sessions 1 and 2 were identical, whereas session 3 involved modulations around higher radiance backgrounds. The pupil responses were fit by a model whose parameters described response amplitude and temporal shape.

Results

Group average pupil responses for all stimuli overlapped extensively across sessions 1 and 2, indicating high reproducibility. Model fits indicate that the response to melanopsin-directed contrast is prolonged relative to that elicited by cone-directed contrast. The group average cone- and melanopsin-directed pupil responses from session 3 were highly similar to those from sessions 1 and 2, suggesting that these responses are insensitive to background radiance over the range studied. The increase in radiance enhanced persistent pupil constriction to blue light.

Conclusions

The group average pupil response to stimuli designed through silent substitution provides a reliable probe of the function of a melanopsin-mediated system in humans. As disruption of the melanopsin system may relate to clinical pathology, the reproducibility of response suggests that silent substitution pupillometry can test if melanopsin signals differ between clinical groups.

Spatial summation in the human fovea: Do normal optical aberrations and fixational eye movements have an effect?

Psychophysical inferences about the neural mechanisms supporting spatial vision can be undermined by uncertainties introduced by optical aberrations and fixational eye movements, particularly in fovea where the neuronal grain of the visual system is fine. We examined the effect of these preneural factors on photopic spatial summation in the human fovea using a custom adaptive optics scanning light ophthalmoscope that provided control over optical aberrations and retinal stimulus motion. Consistent with previous results, Ricco's area of complete summation encompassed multiple photoreceptors when measured with ordinary amounts of ocular aberrations and retinal stimulus motion. When both factors were minimized experimentally, summation areas were essentially unchanged, suggesting that foveal spatial summation is limited by postreceptoral neural pooling. We compared our behavioral data to predictions generated with a physiologically-inspired front-end model of the visual system, and were able to capture the shape of the summation curves obtained with and without pre-retinal factors using a single postreceptoral summing filter of fixed spatial extent. Given our data and modeling, neurons in the magnocellular visual pathway, such as parasol ganglion cells, provide a candidate neural correlate of Ricco's area in the central fovea.

The perception of colour and material in naturalistic tasks

Perceived object colour and material help us to select and interact with objects. Because there is no simple mapping between the pattern of an object's image on the retina and its physical reflectance, our perceptions of colour and material are the result of sophisticated visual computations. A long-standing goal in vision science is to describe how these computations work, particularly as they act to stabilize perceived colour and material against variation in scene factors extrinsic to object surface properties, such as the illumination. If we take seriously the notion that perceived colour and material are useful because they help guide behaviour in natural tasks, then we need experiments that measure and models that describe how they are used in such tasks. To this end, we have developed selection-based methods and accompanying perceptual models for studying perceived object colour and material. This focused review highlights key aspects of our work. It includes a discussion of future directions and challenges, as well as an outline of a computational observer model that incorporates early, known, stages of visual processing and that clarifies how early vision shapes selection performance.

Illumination discrimination in the absence of a fixed surface-reflectance layout

Previous studies have shown that humans can discriminate spectral changes in illumination and that this sensitivity depends both on the chromatic direction of the illumination change and on the ensemble of surfaces in the scene. These studies, however, always used stimulus scenes with a fixed surface-reflectance layout. Here we compared illumination discrimination for scenes in which the surface reflectance layout remains fixed (fixed-surfaces condition) to those in which surface reflectances were shuffled randomly across scenes, but with the mean scene reflectance held approximately constant (shuffled-surfaces condition). Illumination discrimination thresholds in the fixed-surfaces condition were commensurate with previous reports. Thresholds in the shuffled-surfaces condition, however, were considerably elevated. Nonetheless, performance in the shuffled-surfaces condition exceeded that attainable through random guessing. Analysis of eye fixations revealed that in the fixed-surfaces condition, low illumination discrimination thresholds (across observers) were predicted by low overall fixation spread and high consistency of fixation location and fixated surface reflectances across trial intervals. Performance in the shuffled-surfaces condition was not systematically related to any of the eye-fixation characteristics we examined for that condition, but was correlated with performance in the fixed-surfaces condition.

Erratum: Non-invasive assessment of human cone photoreceptor function: erratum

[This corrects the article on p. 5098 in vol. 8, PMID: 29188106.].

Non-invasive assessment of human cone photoreceptor function

Vision begins when light isomerizes the photopigments within photoreceptors. Noninvasive cellular-scale observation of the structure of the human photoreceptor mosaic is made possible through the use of adaptive optics (AO) enhanced ophthalmoscopes, but establishing noninvasive objective measures of photoreceptor function on a similar scale has been more difficult. AO ophthalmoscope images acquired with near-infrared light show that individual cone photoreceptor reflectance can change in response to a visible stimulus. Here we show that the intrinsic response depends on stimulus wavelength and intensity, and that its action spectrum is well-matched to the spectral sensitivity of cone-mediated vision. Our results demonstrate that the cone reflectance response is mediated by photoisomerization, thus making it a direct measure of photoreceptor function.

Use of Crossed Polarizers to Enhance Images of the Eyelids

PURPOSE

To describe imaging of the external eye with Crossed Polarizers to enhance clinically important features in digital photographs of the eyelids.

METHODS

External photographs with and without crossed polarizing filters were taken of patients with blepharitis and controls with no clinical eye pathology.

RESULTS

Photographing eyelid skin through Crossed Polarizers decreased reflections on the skin surface and improved visualization of eyelid telangiectasias and blood vessels in patients with a broad range of skin pigmentation and ethnicities.

CONCLUSIONS

The use of Crossed Polarizers in imaging the external eye reduces reflections and glare from the eyelid skin and margins, thereby allowing for a more detailed evaluation of underlying structures and analysis of images. These findings suggest that including Crossed Polarizers in clinical photography has informative applications for assessing eyelid disease.

Hadza Color Terms Are Sparse, Diverse, and Distributed, and Presage the Universal Color Categories Found in Other World Languages

In our empirical and theoretical study of color naming among the Hadza, a Tanzanian hunter-gatherer group, we show that Hadza color naming is sparse (the color appearance of many stimulus tiles was not named), diverse (there was little consensus in the terms for the color appearance of most tiles), and distributed (the universal color categories of world languages are revealed in nascent form within the Hadza language community, when we analyze the patterns of how individual Hadza deploy color terms). Using our Hadza data set, Witzel shows an association between two measures of color naming performance and the chroma of the stimuli. His prediction of which colored tiles will be named with what level of consensus, while interesting, does not alter the validity of our conclusions.

Multi-modal automatic montaging of adaptive optics retinal images

We present a fully automated adaptive optics (AO) retinal image montaging algorithm using classic scale invariant feature transform with random sample consensus for outlier removal. Our approach is capable of using information from multiple AO modalities (confocal, split detection, and dark field) and can accurately detect discontinuities in the montage. The algorithm output is compared to manual montaging by evaluating the similarity of the overlapping regions after montaging, and calculating the detection rate of discontinuities in the montage. Our results show that the proposed algorithm has high alignment accuracy and a discontinuity detection rate that is comparable (and often superior) to manual montaging. In addition, we analyze and show the benefits of using multiple modalities in the montaging process. We provide the algorithm presented in this paper as open-source and freely available to download.

Illumination discrimination in real and simulated scenes

Characterizing humans' ability to discriminate changes in illumination provides information about the visual system's representation of the distal stimulus. We have previously shown that humans are able to discriminate illumination changes and that sensitivity to such changes depends on their chromatic direction. Probing illumination discrimination further would be facilitated by the use of computer-graphics simulations, which would, in practice, enable a wider range of stimulus manipulations. There is no a priori guarantee, however, that results obtained with simulated scenes generalize to real illuminated scenes. To investigate this question, we measured illumination discrimination in real and simulated scenes that were well-matched in mean chromaticity and scene geometry. Illumination discrimination thresholds were essentially identical for the two stimulus types. As in our previous work, these thresholds varied with illumination change direction. We exploited the flexibility offered by the use of graphics simulations to investigate whether the differences across direction are preserved when the surfaces in the scene are varied. We show that varying the scene's surface ensemble in a manner that also changes mean scene chromaticity modulates the relative sensitivity to illumination changes along different chromatic directions. Thus, any characterization of sensitivity to changes in illumination must be defined relative to the set of surfaces in the scene.

Colour Vision: Understanding #TheDress

A widely-viewed image of a dress elicits striking individual variation in colour perception. Experiments with multiple variants of the image suggest that the individual differences may arise through the action of visual mechanisms that normally stabilise object colour.