Luminance and color inputs to mid-level and high-level vision

Brightness perception under photopic conditions: experiments and modeling with contributions of S-cone and ipRGC

In 1924, the CIE published and standardized the photopic luminous efficiency function. Based on the standardized curve, luminous flux in lumens, luminance in cd/m[Formula: see text], and illuminance in lux are determined by an integral of the curve and the incident light spectra in photometers and are considered physical brightness. However, human brightness perception is not only weighted by this simple determination, but is a more complicated combination of all L-cones, M-cones, S-cones, rods and later ipRGCs, which was partly described by the equivalent brightness of Fotios et al. with the correction factor [Formula: see text]. Recently, new research has demonstrated the role of ipRGCs in human light perception. However, it is still unclear how these signal components of the human visual system are involved in the overall human brightness perception. In this work, human brightness perception under photopic conditions was investigated by visual experiments with 28 subjects under 25 different light spectra. In this way, the contributions of the signal components can be investigated. An optimization process was then performed on the resulting database. The results show that not only the [Formula: see text] component, but also the S-cones and ipRGC play a role, although it is smaller. Thus, the visually scaled brightness model based on the database optimization was constructed using not only illuminance but also S-cones and ipRGC with [Formula: see text] of 0.9554 and RMSE of 4.7802. These results are much better than the brightness model after Fotios et al. using only S-cones ([Formula: see text] = 0.8161, RMSE = 9.7123) and the traditional model without S-cones and ipRGC ([Formula: see text] = 0.8121, RMSE = 9.8171).

Luminance Contrast Drives Interactions between Perception and Working Memory

Visual working memory (WM) enables the use of past sensory experience in guiding behavior. Yet, laboratory tasks commonly evaluate WM in a way that separates it from its sensory bottleneck. To understand how perception interacts with visual memory, we used a delayed shape recognition task to probe how WM may differ for stimuli that bias processing toward different visual pathways. Luminance compared with chromatic signals are more efficient in driving the processing of shapes and may thus also lead to better WM encoding, maintenance, and memory recognition. To evaluate this prediction, we conducted two experiments. In the first psychophysical experiment, we measured contrast thresholds for different WM loads. Luminance contrast was encoded into WM more efficiently than chromatic contrast, even when both sets of stimuli were equated for discriminability. In the second experiment, which also equated stimuli for discriminability, early sensory responses in the EEG that are specific to luminance pathways were modulated by WM load and thus likely reflect the neural substrate of the increased efficiency. Our results cannot be accounted for by simple saliency differences between luminance and color. Rather, they provide evidence for a direct connection between low-level perceptual mechanisms and WM by showing a crucial role of luminance for forming WM representations of shape.

Spatial frequency impacts perceptual and attentional ERP components across cultures

Culture impacts visual perception in several ways.To identify stages of perceptual processing that differ between cultures, we usedelectroencephalography measures of perceptual and attentional responses to simple visual stimuli.Gabor patches of higher or lower spatialfrequencywere presented at high contrast to 25 American and 31 East Asian participants while they were watching for the onset of aninfrequent, oddball stimulus. Region of interest and mass univariate analyses assessed how cultural background and stimuli spatial frequency affected the visual evoked response potentials. Across both groups, the Gabor of lower spatial frequency produced stronger evoked response potentials in the anterior N1 and P3 than did the higher frequency Gabor. The mass univariate analyses also revealed effects of spatial frequency, including a frontal negativity around 150 ms and a widespread posterior positivity around 300 ms. The effects of spatial frequency generally differed little across cultures; although there was some evidence for cultural differences in the P3 response to different frequencies at the Pz electrode, this effect did not emerge in the mass univariate analyses. We discuss these results in relation to those from previous studies, and explore the potential advantages of mass univariate analyses for cultural neuroscience.

Emergence of crowding: The role of contrast and orientation salience

Crowding causes difficulties in judging attributes of an object surrounded by other objects. We investigated crowding for stimuli that isolated either S-cone or luminance mechanisms or combined them. By targeting different retinogeniculate mechanisms with contrast-matched stimuli, we aim to determine the earliest site at which crowding emerges. Discrimination was measured in an orientation judgment task where Gabor targets were presented parafoveally among flankers. In the first experiment, we assessed flanked and unflanked orientation discrimination thresholds for pure S-cone and achromatic stimuli and their combinations. In the second experiment, to capture individual differences, we measured unflanked detection and orientation sensitivity, along with performance under flanker interference for stimuli containing luminance only or combined with S-cone contrast. We confirmed that orientation sensitivity was lower for unflanked S-cone stimuli. When flanked, the pattern of results for S-cone stimuli was the same as for achromatic stimuli with comparable (i.e. low) contrast levels. We also found that flanker interference exhibited a genuine signature of crowding only when orientation discrimination threshold was reliably surpassed. Crowding, therefore, emerges at a stage that operates on signals representing task-relevant featural (here, orientation) information. Because luminance and S-cone mechanisms have very different spatial tuning properties, it is most parsimonious to conclude that crowding takes place at a neural processing stage after they have been combined.

PupilEXT: Flexible Open-Source Platform for High-Resolution Pupillometry in Vision Research

The human pupil behavior has gained increased attention due to the discovery of the intrinsically photosensitive retinal ganglion cells and the afferent pupil control path's role as a biomarker for cognitive processes. Diameter changes in the range of 10-2 mm are of interest, requiring reliable and characterized measurement equipment to accurately detect neurocognitive effects on the pupil. Mostly commercial solutions are used as measurement devices in pupillometry which is associated with high investments. Moreover, commercial systems rely on closed software, restricting conclusions about the used pupil-tracking algorithms. Here, we developed an open-source pupillometry platform consisting of hardware and software competitive with high-end commercial stereo eye-tracking systems. Our goal was to make a professional remote pupil measurement pipeline for laboratory conditions accessible for everyone. This work's core outcome is an integrated cross-platform (macOS, Windows and Linux) pupillometry software called PupilEXT, featuring a user-friendly graphical interface covering the relevant requirements of professional pupil response research. We offer a selection of six state-of-the-art open-source pupil detection algorithms (Starburst, Swirski, ExCuSe, ElSe, PuRe and PuReST) to perform the pupil measurement. A developed 120-fps pupillometry demo system was able to achieve a calibration accuracy of 0.003 mm and an averaged temporal pupil measurement detection accuracy of 0.0059 mm in stereo mode. The PupilEXT software has extended features in pupil detection, measurement validation, image acquisition, data acquisition, offline pupil measurement, camera calibration, stereo vision, data visualization and system independence, all combined in a single open-source interface, available at https://github.com/openPupil/Open-PupilEXT.

Melanopic Limits of Metamer Spectral Optimisation in Multi-Channel Smart Lighting Systems

Modern indoor lighting faces the challenge of finding an appropriate balance between energy consumption, legal requirements, visual performance, and the circadian effectiveness of a spectrum. Multi-channel LED luminaires have the option of keeping image-forming metrics steady while varying the melanopic radiance through metamer spectra for non-visual purposes. Here, we propose the theoretical concept of an automated smart lighting system that is designed to satisfy the user’s visual preference through neural networks while triggering the non-visual pathway via metamers. To quantify the melanopic limits of metamers at a steady chromaticity point, we have used 561 chromaticity coordinates along the Planckian locus (2700 K to 7443 K, ±Duv 0 to 0.048) as optimisation targets and generated the spectra by using a 6-channel, 8-channel, and 11-channel LED combination at three different luminance levels. We have found that in a best-case scenario, the melanopic radiance can be varied up to 65% while keeping the chromaticity coordinates constant (Δu′v′≤7.05×10−5) by using metamer spectra. The highest melanopic metamer contrast can be reached near the Planckian locus between 3292 and 4717 K within a Duv range of −0.009 to 0.006. Additionally, we publish over 1.2 million optimised spectra generated by multichannel LED luminaires as an open-source dataset along with this work.

Neural differences between chromatic- and luminance-driven attentional salience in visual search

Previous electroencephalographic research on attentional salience did not fully capture the complexities of low-level vision, which relies on both cone-opponent chromatic and cone-additive luminance mechanisms. We systematically varied color and luminance contrast using a visual search task for a higher contrast target to assess the degree to which the salience-computing attentional mechanisms are constrained by low-level visual inputs. In our first experiment, stimuli were defined by contrast that isolated chromatic or luminance mechanisms. In our second experiment, targets were defined by contrasts that isolated or combined achromatic and chromatic mechanisms. In both experiments, event-related potential waveforms contralateral and ipsilateral to the target were qualitatively different for chromatic- compared to luminance-defined stimuli. The same was true of the difference waves computed from these waveforms, with isoluminant stimuli eliciting a mid-latency posterior contralateral negativity (PCN) component and achromatic stimuli eliciting a complex of multiple components, including an early posterior contralateral positivity followed by a late-latency PCN. Combining color with luminance resulted in waveform and difference wave patterns equivalent to those of achromatic stimuli. When large levels of chromaticity contrast were added to targets with small levels of luminance contrast, PCN latency was speeded. In conclusion, the mechanisms underlying attentional salience are constrained by the low-level inputs they receive. Furthermore, speeded PCN latencies for stimuli that combine color and luminance signals compared to stimuli that contain luminance alone demonstrate that color and luminance channels are integrated during pre-attentive visual processing, before top-down allocation of attention is triggered.

The Pupillary Response to the Unknown: Novelty Versus Familiarity

Object recognition is a type of perception that enables observers to recognize familiar shapes and categorize them into real-world identities. In this preregistered study, we aimed to determine whether pupil size changes occur during the perception and recognition of identifiable objects. We compared pupil size changes for familiar objects, nonobjects, and random noise. Nonobjects and noise produced greater pupil dilation than familiar objects. Contrary to previous evidence showing greater pupil dilation to stimuli with more perceptual and affective content, these results indicate a greater pupil dilation to stimuli that are unidentifiable. This is consistent with the relative salience of novelty compared to familiarity at the physiological level driving the pupil response.

Combining S-cone and luminance signals adversely affects discrimination of objects within backgrounds

The visual system processes objects embedded in complex scenes that vary in both luminance and colour. In such scenes, colour contributes to the segmentation of objects from backgrounds, but does it also affect perceptual organisation of object contours which are already defined by luminance signals, or are these processes unaffected by colour's presence? We investigated if luminance and chromatic signals comparably sustain processing of objects embedded in backgrounds, by varying contrast along the luminance dimension and along the two cone-opponent colour directions. In the first experiment thresholds for object/non-object discrimination of Gaborised shapes were obtained in the presence and absence of background clutter. Contrast of the component Gabors was modulated along single colour/luminance dimensions or co-modulated along multiple dimensions simultaneously. Background clutter elevated discrimination thresholds only for combined S-(L + M) and L + M signals. The second experiment replicated and extended this finding by demonstrating that the effect was dependent on the presence of relatively high S-(L + M) contrast. These results indicate that S-(L + M) signals impair spatial vision when combined with luminance. Since S-(L + M) signals are characterised by relatively large receptive fields, this is likely to be due to an increase in the size of the integration field over which contour-defining information is summed.

Shape recognition: convexities, concavities and things in between

Visual objects are effortlessly recognized from their outlines, largely irrespective of viewpoint. Previous studies have drawn different conclusions regarding the importance to shape recognition of specific shape features such as convexities and concavities. However, most studies employed familiar objects, or shapes without curves, and did not measure shape recognition across changes in scale and position. We present a novel set of random shapes with well-defined convexities, concavities and inflections (intermediate points), segmented to isolate each feature type. Observers matched the segmented reference shapes to one of two subsequently presented whole-contour shapes (target or distractor) that were re-scaled and re-positioned. For very short segment lengths, performance was significantly higher for convexities than for concavities or intermediate points and for convexities remained constant with increasing segment length. For concavities and intermediate points, performance improved with increasing segment length, reaching convexity performance only for long segments. No significant differences between concavities and intermediates were found. These results show for the first time that closed curvilinear shapes are encoded using the positions of convexities, rather than concavities or intermediate regions. A shape-template model with no free parameters gave an excellent account of the data.

Predictive distractor context facilitates attentional selection of high, but not intermediate and low, salience targets

It is well established that we can focally attend to a specific region in visual space without shifting our eyes, so as to extract action-relevant sensory information from covertly attended locations. The underlying mechanisms that determine how fast we engage our attentional spotlight in visual-search scenarios, however, remain controversial. One dominant view advocated by perceptual decision-making models holds that the times taken for focal-attentional selection are mediated by an internal template that biases perceptual coding and selection decisions exclusively through target-defining feature coding. This notion directly predicts that search times remain unaffected whether or not participants can anticipate the upcoming distractor context. Here we tested this hypothesis by employing an illusory-figure localization task that required participants to search for an invariant target amongst a variable distractor context, which gradually changed--either randomly or predictably--as a function of distractor-target similarity. We observed a graded decrease in internal focal-attentional selection times--correlated with external behavioral latencies--for distractor contexts of higher relative to lower similarity to the target. Critically, for low but not intermediate and high distractor-target similarity, these context-driven effects were cortically and behaviorally amplified when participants could reliably predict the type of distractors. This interactive pattern demonstrates that search guidance signals can integrate information about distractor, in addition to target, identities to optimize distractor-target competition for focal-attentional selection.