Image luminance changes contrast sensitivity in visual cortex

A handheld device for intra-cavity and ex vivo fluorescence imaging of breast conserving surgery margins with 5-aminolevulinic acid

BACKGROUND

Visualization of cancer during breast conserving surgery (BCS) remains challenging; the BCS reoperation rate is reported to be 20-70% of patients. An urgent clinical need exists for real-time intraoperative visualization of breast carcinomas during BCS. We previously demonstrated the ability of a prototype imaging device to identify breast carcinoma in excised surgical specimens following 5-aminolevulinic acid (5-ALA) administration. However, this prototype device was not designed to image the surgical cavity for remaining carcinoma after the excised lumpectomy specimen is removed. A new handheld fluorescence (FL) imaging prototype device, designed to image both excised specimens and within the surgical cavity, was assessed in a clinical trial to evaluate its clinical utility for first-in-human, real-time intraoperative imaging during index BCS.

RESULTS

The imaging device combines consumer-grade imaging sensory technology with miniature light-emitting diodes (LEDs) and multiband optical filtering to capture high-resolution white light (WL) and FL digital images and videos. The technology allows for visualization of protoporphyrin IX (PpIX), which fluoresces red when excited by violet-blue light. To date, n = 17 patients have received 20 mg kg bodyweight (BW) 5-ALA orally 2-4 h before imaging to facilitate the accumulation of PpIX within tumour cells. Tissue types were identified based on their colour appearance. Breast tumours in sectioned lumpectomies appeared red, which contrasted against the green connective tissues and orange-brown adipose tissues. In addition, ductal carcinoma in situ (DCIS) that was missed during intraoperative standard of care was identified at the surgical margin at <1 mm depth. In addition, artifacts due to the surgical drape, illumination, and blood within the surgical cavity were discovered.

CONCLUSIONS

This study has demonstrated the detection of a grossly occult positive margin intraoperatively. Artifacts from imaging within the surgical cavity have been identified, and potential mitigations have been proposed.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT01837225 (Trial start date is September 2010. It was registered to ClinicalTrials.gov retrospectively on April 23, 2013, then later updated on April 9, 2020, to reflect the introduction of the new imaging device.).

In primary visual cortex fMRI responses to chromatic and achromatic stimuli are interdependent and predict contrast detection thresholds

Chromatic and achromatic signals in primary visual cortex have historically been considered independent of each other but have since shown evidence of interdependence. Here, we investigated the combination of two components of a stimulus; an achromatic dynamically changing check background and a chromatic (L-M or S cone) target grating. We found that combinations of chromatic and achromatic signals in primary visual cortex were interdependent, with the dynamic range of responses to chromatic contrast decreasing as achromatic contrast increased. A contrast detection threshold study also revealed interdependence of background and target, with increasing chromatic contrast detection thresholds as achromatic background contrast increased. A model that incorporated a normalising effect of achromatic contrast on chromatic responses, but not vice versa, best predicted our V1 data as well as behavioural thresholds. Further along the visual hierarchy, the dynamic range of chromatic responses was maintained when compared to achromatic responses, which became increasingly compressive.

Predicting Single Neuron Responses of the Primary Visual Cortex with Deep Learning Model

Modeling neuron responses to stimuli can shed light on next-generation technologies such as brain-chip interfaces. Furthermore, high-performing models can serve to help formulate hypotheses and reveal the mechanisms underlying neural responses. Here the state-of-the-art computational model is presented for predicting single neuron responses to natural stimuli in the primary visual cortex (V1) of mice. The algorithm incorporates object positions and assembles multiple models with different train-validation data, resulting in a 15%-30% improvement over the existing models in cross-subject predictions and ranking first in the SENSORIUM 2022 Challenge, which benchmarks methods for neuron-specific prediction based on thousands of images. Importantly, The model reveals evidence that the spatial organizations of V1 are conserved across mice. This model will serve as an important noninvasive tool for understanding and utilizing the response patterns of primary visual cortex neurons.

Orientation and contrast deviance examined: Contrast effects mimic deviant-related negativity yet neither produce the canonical neural correlate of prediction error

The visual mismatch negativity (vMMN) is a negative-going event-related potential (ERP) component that is largest somewhere between 100 and 300 ms after the onset of an unpredictable visual event (i.e., a deviant) in an otherwise predictable sequence of visual events (i.e., standards). Many have argued that the vMMN allows us to monitor our ever-changing visual environment for deviants critical to our survival. Recently, however, it has become unclear whether unpredicted changes in low-level features of visual input, like orientation, can evoke the vMMN. I address this by testing isolated orientation changes, to confirm recent findings, and isolated contrast changes, to determine whether other low-level features of visual input do not evoke the vMMN in a traditional oddball paradigm. Eighteen participants saw sequences of rare, unanticipated, and different deviant stimuli, interspersed among frequent, anticipated, and identical standard stimuli. Stimuli were Gabor patches. Neither deviant produced a vMMN. Therefore, changes in low-level visual properties of well-controlled stimuli-a stimulus in which one property can be manipulated while all others remain unaffected-like Gabor patches do not yield a vMMN.

An open dataset for human SSVEPs in the frequency range of 1-60 Hz

A steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) system relies on the photic driving response to effectively elicit characteristic electroencephalogram (EEG) signals. However, traditional visual stimuli mainly adopt high-contrast black-and-white flickering stimulations, which are easy to cause visual fatigue. This paper presents an SSVEP dataset acquired at a wide frequency range from 1 to 60 Hz with an interval of 1 Hz using flickering stimuli under two different modulation depths. This dataset contains 64-channel EEG data from 30 healthy subjects when they fixated on a single flickering stimulus. The stimulus was rendered on an LCD display with a refresh rate of 240 Hz. Initially, the dataset was rigorously validated through comprehensive data analysis to investigate SSVEP responses and user experiences. Subsequently, BCI performance was evaluated through offline simulations of frequency-coded and phase-coded BCI paradigms. This dataset provides comprehensive and high-quality data for studying and developing SSVEP-based BCI systems.

Contrast Sensitivity of ON and OFF Human Retinal Pathways in Myopia

The human visual cortex processes light and dark stimuli with ON and OFF pathways that are differently modulated by luminance contrast. We have previously demonstrated that ON cortical pathways have higher contrast sensitivity than OFF cortical pathways and the difference increases with luminance range (defined as the maximum minus minimum luminance in the scene). Here, we demonstrate that these ON-OFF cortical differences are already present in the human retina and that retinal responses measured with electroretinography are more affected by reductions in luminance range than cortical responses measured with electroencephalography. Moreover, we show that ON-OFF pathway differences measured with electroretinography become more pronounced in myopia, a visual disorder that elongates the eye and blurs vision at far distance. We find that, as the eye axial length increases across subjects, ON retinal pathways become less responsive, slower in response latency, less sensitive, and less effective and slower at driving pupil constriction. Based on these results, we conclude that myopia is associated with a deficit in ON pathway function that decreases the ability of the retina to process low contrast and regulate retinal illuminance in bright environments.

Effects of normalized summation in the visual illusion of extent

In the present study, the features of summation of effects caused by contextual distracting dots in the length-matching task (a variant of the filled-space illusion) were investigated. In the first two series of psychophysical experiments, the illusion magnitude was measured as a function of the displacement of distractors (either single or double sets of dots) orthogonally to the main axis of the stimulus. It was demonstrated that with increasing displacement, the illusion smoothly decreases for a single set of distractors, while for two sets, the illusion first increases to a certain maximum value, and then gradually decreases. In the third and fourth series of experiments, magnitude of the illusion was measured as a function of the luminance of one set of distracting dots, while the luminance of the other set was fixed. It has been shown that increasing the luminance until the same value is reached for both sets leads to a monotonous growth in the illusion magnitude; after that, the illusion asymptotically decreases to an almost constant level. The theoretical interpretation of the established functional dependencies was performed using a quantitative model based on the assumption that the illusion may arise due to the weighted summation of the distractor-induced normalized neural activity, which leads to the perceptual mislocalization of terminators of stimulus spatial intervals.

Dynamic Recruitment of the Feedforward and Recurrent Mechanism for Black-White Asymmetry in the Primary Visual Cortex

Black and white information is asymmetrically distributed in natural scenes, evokes asymmetric neuronal responses, and causes asymmetric perceptions. Recognizing the universality and essentiality of black-white asymmetry in visual information processing, the neural substrates for black-white asymmetry remain unclear. To disentangle the role of the feedforward and recurrent mechanisms in the generation of cortical black-white asymmetry, we recorded the V1 laminar responses and LGN responses of anesthetized cats of both sexes. In a cortical column, we found that black-white asymmetry starts at the input layer and becomes more pronounced in the output layer. We also found distinct dynamics of black-white asymmetry between the output layer and the input layer. Specifically, black responses dominate in all layers after stimulus onset. After stimulus offset, black and white responses are balanced in the input layer, but black responses still dominate in the output layer. Compared with that in the input layer, the rebound response in the output layer is significantly suppressed. The relative suppression strength evoked by white stimuli is notably stronger and depends on the location within the ON-OFF cortical map. A model with delayed and polarity-selective cortical suppression explains black-white asymmetry in the output layer, within which prominent recurrent connections are identified by Granger causality analysis. In addition to black-white asymmetry in response strength, the interlaminar differences in spatial receptive field varied dynamically. Our findings suggest that the feedforward and recurrent mechanisms are dynamically recruited for the generation of black-white asymmetry in V1.SIGNIFICANCE STATEMENT Black-white asymmetry is universal and essential in visual information processing, yet the neural substrates for cortical black-white asymmetry remain unknown. Leveraging V1 laminar recordings, we provided the first laminar pattern of black-white asymmetry in cat V1 and found distinct dynamics of black-white asymmetry between the output layer and the input layer. Comparing black-white asymmetry across three visual hierarchies, the LGN, V1 input layer, and V1 output layer, we demonstrated that the feedforward and recurrent mechanisms are dynamically recruited for the generation of cortical black-white asymmetry. Our findings not only enhance our understanding of laminar processing within a cortical column but also elucidate how feedforward connections and recurrent connections interact to shape neuronal response properties.

Spatiotemporal visual statistics of aquatic environments in the natural habitats of zebrafish

Animal sensory systems are tightly adapted to the demands of their environment. In the visual domain, research has shown that many species have circuits and systems that exploit statistical regularities in natural visual signals. The zebrafish is a popular model animal in visual neuroscience, but relatively little quantitative data is available about the visual properties of the aquatic habitats where zebrafish reside, as compared to terrestrial environments. Improving our understanding of the visual demands of the aquatic habitats of zebrafish can enhance the insights about sensory neuroscience yielded by this model system. We analyzed a video dataset of zebrafish habitats captured by a stationary camera and compared this dataset to videos of terrestrial scenes in the same geographic area. Our analysis of the spatiotemporal structure in these videos suggests that zebrafish habitats are characterized by low visual contrast and strong motion when compared to terrestrial environments. Similar to terrestrial environments, zebrafish habitats tended to be dominated by dark contrasts, particularly in the lower visual field. We discuss how these properties of the visual environment can inform the study of zebrafish visual behavior and neural processing and, by extension, can inform our understanding of the vertebrate brain.

Differences in visual stimulation between reading and walking and implications for myopia development

Visual input plays an important role in the development of myopia (nearsightedness), a visual disorder that blurs vision at far distances. The risk of myopia progression increases with the time spent reading and decreases with outdoor activity for reasons that remain poorly understood. To investigate the stimulus parameters driving this disorder, we compared the visual input to the retina of humans performing two tasks associated with different risks of myopia progression, reading and walking. Human subjects performed the two tasks while wearing glasses with cameras and sensors that recorded visual scenes and visuomotor activity. When compared with walking, reading black text in white background reduced spatiotemporal contrast in central vision and increased it in peripheral vision, leading to a pronounced reduction in the ratio of central/peripheral strength of visual stimulation. It also made the luminance distribution heavily skewed toward negative dark contrast in central vision and positive light contrast in peripheral vision, decreasing the central/peripheral stimulation ratio of ON visual pathways. It also decreased fixation distance, blink rate, pupil size, and head-eye coordination reflexes dominated by ON pathways. Taken together with previous work, these results support the hypothesis that reading drives myopia progression by understimulating ON visual pathways.

Pulse trains to percepts: A virtual patient describing the perceptual effects of human visual cortical stimulation

The field of cortical sight restoration prostheses is making rapid progress with three clinical trials of visual cortical prostheses underway. However, as yet, we have only limited insight into the perceptual experiences produced by these implants. Here we describe a computational model or 'virtual patient', based on the neurophysiological architecture of V1, which successfully predicts the perceptual experience of participants across a wide range of previously published cortical stimulation studies describing the location, size, brightness and spatiotemporal shape of electrically induced percepts in humans. Our simulations suggest that, in the foreseeable future the perceptual quality of cortical prosthetic devices is likely to be limited by the neurophysiological organization of visual cortex, rather than engineering constraints.

Contrast and Luminance Gain Control in the Macaque's Lateral Geniculate Nucleus

There is substantial variation in the mean and variance of light levels (luminance and contrast) in natural visual scenes. Retinal ganglion cells maintain their sensitivity despite this variation using two adaptive mechanisms, which control how responses depend on luminance and on contrast. However, the nature of each mechanism and their interactions downstream of the retina are unknown. We recorded neurons in the magnocellular and parvocellular layers of the lateral geniculate nucleus (LGN) in anesthetized adult male macaques and characterized how their responses adapt to changes in contrast and luminance. As contrast increases, neurons in the magnocellular layers maintain sensitivity to high temporal frequency stimuli but attenuate sensitivity to low-temporal frequency stimuli. Neurons in the parvocellular layers do not adapt to changes in contrast. As luminance increases, both magnocellular and parvocellular cells increase their sensitivity to high-temporal frequency stimuli. Adaptation to luminance is independent of adaptation to contrast, as previously reported for LGN neurons in the cat. Our results are similar to those previously reported for macaque retinal ganglion cells, suggesting that adaptation to luminance and contrast result from two independent mechanisms that are retinal in origin.

Tablet-based 'ON/OFF' pathway test can distinguish between rod- and cone-dominated diseases

INTRODUCTION

The photopic ON pathway defect is associated with nocturnal vision loss. However, the measurement of ON function to detect a rod-dominated disease (rods affected more than cones) has not been explored. We evaluated whether the psychophysical evaluation of ON/OFF pathways can be used to distinguish cone-dominated from rod-dominated diseases.

METHODS

Thirty-seven patients with inherited retinal diseases were tested using the 'EyeSpeed' [iOS application] on an iPad. The test displayed a random number (1-3) of light or dark targets on a black-and-white noise background. Participants responded on a touch screen indicating the correct number of targets displayed. The outcome variables-reaction time, accuracy and performance index (speed [1/reaction time] * accuracy) to both light and dark targets were assessed for diagnostic ability using standard receiver-operating characteristic (ROC) analysis.

RESULTS

Mean ± standard deviation age and visual acuity for the cone- and rod-dominated groups were 25.15 ± 11.74 years, 0.80 ± 0.25 logMAR and 28.3 ± 14.29 years, 0.48 ± 0.26 logMAR, respectively. The median reaction time to light targets in rod-dominated disease [interquartile range] was 5.28 s [3.17], significantly greater than for patients with cone-dominated disease (2.07 s [0.93]; Mann-Whitney U test, p < 0.001). Amongst all of the outcome variables evaluated, the reaction time to light targets (criterion of ≥2.98 s) exhibited the highest area under the ROC curve (area = 0.89 ± 0.11; p < 0.001), with a sensitivity and specificity of 82.4% and 85% respectively.

CONCLUSIONS

Reaction time to light targets using the ON/OFF pathway paradigm is a valid marker to differentiate between rod- and cone-dominated retinal dystrophies. ON pathway function measured using a tablet-based test could act as a supplemental test in the diagnosis of challenging photoreceptor-specific inherited retinal diseases.

Luminance Contrast Shifts Dominance Balance between ON and OFF Pathways in Human Vision

Human vision processes light and dark stimuli in visual scenes with separate ON and OFF neuronal pathways. In nature, stimuli lighter or darker than their local surround have different spatial properties and contrast distributions (Ratliff et al., 2010; Cooper and Norcia, 2015; Rahimi-Nasrabadi et al., 2021). Similarly, in human vision, we show that luminance contrast affects the perception of lights and darks differently. At high contrast, human subjects of both sexes locate dark stimuli faster and more accurately than light stimuli, which is consistent with a visual system dominated by the OFF pathway. However, at low contrast, they locate light stimuli faster and more accurately than dark stimuli, which is consistent with a visual system dominated by the ON pathway. Luminance contrast was strongly correlated with multiple ON/OFF dominance ratios estimated from light/dark ratios of performance errors, missed targets, or reaction times (RTs). All correlations could be demonstrated at multiple eccentricities of the central visual field with an ON-OFF perimetry test implemented in a head-mounted visual display. We conclude that high-contrast stimuli are processed faster and more accurately by OFF pathways than ON pathways. However, the OFF dominance shifts toward ON dominance when stimulus contrast decreases, as expected from the higher-contrast sensitivity of ON cortical pathways (Kremkow et al., 2014; Rahimi-Nasrabadi et al., 2021). The results highlight the importance of contrast polarity in visual field measurements and predict a loss of low-contrast vision in humans with ON pathway deficits, as demonstrated in animal models (Sarnaik et al., 2014).SIGNIFICANCE STATEMENT ON and OFF retino-thalamo-cortical pathways respond differently to luminance contrast. In both animal models and humans, low contrasts drive stronger responses from ON pathways, whereas high contrasts drive stronger responses from OFF pathways. We demonstrate that these ON-OFF pathway differences have a correlate in human vision. At low contrast, humans locate light targets faster and more accurately than dark targets but, as contrast increases, dark targets become more visible than light targets. We also demonstrate that contrast is strongly correlated with multiple light/dark ratios of visual performance in central vision. These results provide a link between neuronal physiology and human vision while emphasizing the importance of stimulus polarity in measurements of visual fields and contrast sensitivity.

Cortical mechanisms of visual brightness

The primary visual cortex signals the onset of light and dark stimuli with ON and OFF cortical pathways. Here, we demonstrate that both pathways generate similar response increments to large homogeneous surfaces and their response average increases with surface brightness. We show that, in cat visual cortex, response dominance from ON or OFF pathways is bimodally distributed when stimuli are smaller than one receptive field center but unimodally distributed when they are larger. Moreover, whereas small bright stimuli drive opposite responses from ON and OFF pathways (increased versus suppressed activity), large bright surfaces drive similar response increments. We show that this size-brightness relation emerges because strong illumination increases the size of light surfaces in nature and both ON and OFF cortical neurons receive input from ON thalamic pathways. We conclude that visual scenes are perceived as brighter when the average response increments from ON and OFF cortical pathways become stronger.

Mazade et al. find that the visual cortex encodes brightness differently for small than large stimuli. Bright small stimuli drive cortical pathways signaling lights and suppress cortical pathways signaling darks. Conversely, large surfaces drive response increments from both pathways and appear brightest when the response average is strongest.

A neuro-inspired computational model of life-long learning and catastrophic interference, mimicking hippocampus novelty-based dopamine modulation and lateral inhibitory plasticity

The human brain has a remarkable lifelong learning capability to acquire new experiences while retaining previously acquired information. Several hypotheses have been proposed to explain this capability, but the underlying mechanisms are still unclear. Here, we propose a neuro-inspired firing-rate computational model involving the hippocampus and surrounding areas, that encompasses two key mechanisms possibly underlying this capability. The first is based on signals encoded by the neuromodulator dopamine, which is released by novel stimuli and enhances plasticity only when needed. The second is based on a homeostatic plasticity mechanism that involves the lateral inhibitory connections of the pyramidal neurons of the hippocampus. These mechanisms tend to protect neurons that have already been heavily employed in encoding previous experiences. The model was tested with images from the MNIST machine learning dataset, and with more naturalistic images, for its ability to mitigate catastrophic interference in lifelong learning. The results show that the proposed biologically grounded mechanisms can effectively enhance the learning of new stimuli while protecting previously acquired knowledge. The proposed mechanisms could be investigated in future empirical animal experiments and inspire machine learning models.

More than the end: OFF response plasticity as a mnemonic signature of a sound’s behavioral salience

In studying how neural populations in sensory cortex code dynamically varying stimuli to guide behavior, the role of spiking after stimuli have ended has been underappreciated. This is despite growing evidence that such activity can be tuned, experience-and context-dependent and necessary for sensory decisions that play out on a slower timescale. Here we review recent studies, focusing on the auditory modality, demonstrating that this so-called OFF activity can have a more complex temporal structure than the purely phasic firing that has often been interpreted as just marking the end of stimuli. While diverse and still incompletely understood mechanisms are likely involved in generating phasic and tonic OFF firing, more studies point to the continuing post-stimulus activity serving a short-term, stimulus-specific mnemonic function that is enhanced when the stimuli are particularly salient. We summarize these results with a conceptual model highlighting how more neurons within the auditory cortical population fire for longer duration after a sound’s termination during an active behavior and can continue to do so even while passively listening to behaviorally salient stimuli. Overall, these studies increasingly suggest that tonic auditory cortical OFF activity holds an echoic memory of specific, salient sounds to guide behavioral decisions.

Realization of an Artificial Visual Nervous System using an Integrated Optoelectronic Device Array

Human behavior (e.g., the response to any incoming information) has very complex forms and is based on the response to consecutive external stimuli entering varied sensory receptors. Sensory adaptation is an elementary form of the sensory nervous system known to filter out irrelevant information for efficient information transfer from consecutive stimuli. As bioinspired neuromorphic electronic system is developed, the functionality of organs shall be emulated at a higher level than the cell. Because it is important for electronic devices to possess sensory adaptation in spiking neural networks, the authors demonstrate a dynamic, real-time, photoadaptation process to optical irradiation when repeated light stimuli are presented to the artificial photoreceptor. The filtered electrical signal generated by the light and the adapting signal produces a specific range of postsynaptic states through the neurotransistor, demonstrating changes in the response according to the environment, as normally perceived by the human brain. This successfully demonstrates plausible biological sensory adaptation. Further, the ability of this circuit design to accommodate changes in the intensity of bright or dark light by adjusting the sensitivity of the artificial photoreceptor is demonstrated. Thus, the proposed artificial photoreceptor circuits have the potential to advance neuromorphic device technology by providing sensory adaptation capabilities.

Dependence of perceptual saccadic suppression on peri-saccadic image flow properties and luminance contrast polarity

Across saccades, perceptual detectability of brief visual stimuli is strongly diminished. We recently observed that this perceptual suppression phenomenon is jumpstarted in the retina, suggesting that the phenomenon might be significantly more visual in nature than normally acknowledged. Here, we explicitly compared saccadic suppression strength when saccades were made across a uniform image of constant luminance versus when saccades were made across image patches of different luminance, width, and trans-saccadic luminance polarity. We measured perceptual contrast thresholds of human subjects for brief peri-saccadic flashes of positive (luminance increments) or negative (luminance decrements) polarity. Thresholds were >6–7 times higher when saccades translated a luminance stripe or edge across the retina than when saccades were made over a completely uniform image patch. Critically, both background luminance and flash luminance polarity strongly modulated peri-saccadic contrast thresholds. In addition, all of these very same visual dependencies also occurred in the absence of any saccades, but with qualitatively similar rapid translations of image patches across the retina. This similarity of visual dependencies with and without saccades supports the notion that perceptual saccadic suppression may be fundamentally a visual phenomenon, which strongly motivates neurophysiological and theoretical investigations on the role of saccadic eye movement commands in modulating its properties.

Darks and Lights, the 'Yin-Yang' of Vision Depends on Luminance

We all know the disappointment when, after a wonderful snapshot, the details in the photo are at much lower contrast than seen before with our own eyes. A recent study by Rahimi-Nasrabadi et al. revealed that this is because human vision accounts for actual luminance range and for accompanied asymmetric changes in dark and light contrasts.