Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations

Anatomically resolved oscillatory bursts reveal dynamic motifs of thalamocortical activity during naturalistic stimulus viewing

Natural vision requires circuit mechanisms which process complex spatiotemporal stimulus features in parallel. In the mammalian forebrain, one signature of circuit activation is fast oscillatory dynamics, reflected in the local field potential (LFP). Using data from the Allen Neuropixels Visual Coding project, we show that local visual features in naturalistic stimuli induce in mouse primary visual cortex (V1) retinotopically specific oscillations in various frequency bands and V1 layers. Specifically, layer 4 (L4) narrowband gamma was linked to luminance, low-gamma to optic flow, and L4/L5 epsilon oscillations to contrast. These feature-specific oscillations were associated with distinct translaminar spike-phase coupling patterns, which were conserved across a range of stimuli containing the relevant visual features, suggesting that they might constitute feature-specific circuit motifs. Our findings highlight visually induced fast oscillations as markers of dynamic circuit motifs, which may support differential and multiplexed coding of complex visual input and thalamocortical information propagation.

A spherical code of retinal orientation selectivity enables decoding in ensembled and retinotopic operation

Selectivity to orientations of edges is seen at the earliest stages of visual processing in retinal orientation-selective ganglion cells (OSGCs), which are thought to prefer vertical or horizontal orientation. However, because stationary edges are projected on the hemispherical retina as lines of longitude or latitude, how edge orientation is encoded and decoded by the brain is unknown. Here, by mapping the orientation selectivity (OS) of thousands of OSGCs at known retinal locations in mice, we identify three OSGC types whose preferences match two longitudinal fields and a fourth type matching two latitudinal fields, with the members of each field pair being non-orthogonal. A geometric decoder reveals that two OS sensors yield optimal orientation decoding when approaching the deviation from orthogonality we observe for OSGC field pairs. Retinotopically organized decoding generates type-specific variation in decoding efficiency across the visual field. OS tuning is greater in the dorsal retina, possibly reflecting an evolutionary adaptation to an environmental gradient of edges.

Nitric oxide modulates contrast suppression in a subset of mouse retinal ganglion cells

Neuromodulators have major influences on the regulation of neural circuit activity across the nervous system. Nitric oxide (NO) has been shown to be a prominent neuromodulator in many circuits and has been extensively studied in the retina. Here, it has been associated with the regulation of light adaptation, gain control, and gap junctional coupling, but its effect on the retinal output, specifically on the different types of retinal ganglion cells (RGCs), is still poorly understood. In this study, we used two-photon Ca2+ imaging and multi-electrode array (MEA) recordings to measure light-evoked activity of RGCs in the ganglion cell layer in the ex vivo mouse retina. This approach allowed us to investigate the neuromodulatory effects of NO on a cell type-level. Our findings reveal that NO selectively modulates the suppression of temporal responses in a distinct subset of contrast-suppressed RGC types, increasing their activity without altering the spatial properties of their receptive fields. Given that under photopic conditions, NO release is triggered by quick changes in light levels, we propose that these RGC types signal fast contrast changes to higher visual regions. Remarkably, we found that about one-third of the RGC types, recorded using two-photon Ca2+ imaging, exhibited consistent, cell type-specific adaptational response changes throughout an experiment, independent of NO. By employing a sequential-recording paradigm, we could disentangle those additional adaptational response changes from drug-induced modulations. Taken together, our research highlights the selective neuromodulatory effects of NO on RGCs and emphasizes the need of considering non-pharmacological activity changes, like adaptation, in such study designs.

The first interneuron of the mouse visual system is tailored to the natural environment through morphology and electrical coupling

The topographic complexity of the mouse retina has long been underestimated. However, functional gradients exist, which reflect the non-uniform statistics of the visual environment. Horizontal cells are the first visual interneurons that shape the receptive fields of down-stream neurons. We asked whether regional specializations are present in terms of horizontal cell density distributions, morphological properties, localization of gap junction proteins, and the spatial extent of electrical coupling. These key features were asymmetrically organized along the dorsoventral axis. Dorsal cells were less densely distributed, had larger dendritic trees, and electrical coupling was more extensive than in ventral cells. The steepest change occurred at the visual horizon. Our results show that the cellular and synaptic organization of the mouse visual system are adapted to the visual environment at the earliest possible level and that horizontal cells are suited to form the substrate for the global gradient of ganglion cell receptive fields.

Visual Field Asymmetries in Responses to ON and OFF Pathway Biasing Stimuli

Recent reports suggest the ON and OFF pathways are differentially susceptible to selective vision loss in glaucoma. Thus, perimetric assessment of ON- and OFF-pathway function may serve as a useful diagnostic. However, this necessitates a developed understanding of normal ON/OFF pathway function around the visual field and as a function of input intensity. Here, using electroencephalography, we measured ON- and OFF-pathway biased contrast response functions in the upper and lower visual fields. Using the steady-state visually evoked potential paradigm, we flickered achromatic luminance probes according to a saw-tooth waveform, the fast phase of which biased responses towards the ON or OFF pathways. Neural responses from the upper and lower visual fields were simultaneously measured using frequency tagging - probes in the upper visual field modulated at 3.75 Hz, while those in the lower visual field modulated at 3 Hz. We find that responses to OFF/decrements are larger than ON/increments, especially in the lower visual field. In the lower visual field, both ON and OFF responses were well described by a sigmoidal non-linearity. In the upper visual field, the ON pathway function was very similar to that of the lower, but the OFF pathway function showed reduced saturation and more cross-subject variability. Overall, this demonstrates that the relationship between the ON and OFF pathways depends on the visual field location and contrast level, potentially reflective of natural scene statistics.

Neurobehavioral meaning of pupil size

Pupil size is a widely used metric of brain state. It is one of the few signals originating from the brain that can be readily monitored with low-cost devices in basic science, clinical, and home settings. It is, therefore, important to investigate and generate well-defined theories related to specific interpretations of this metric. What exactly does it tell us about the brain? Pupils constrict in response to light and dilate during darkness, but the brain also controls pupil size irrespective of luminosity. Pupil size fluctuations resulting from ongoing "brain states" are used as a metric of arousal, but what is pupil-linked arousal and how should it be interpreted in neural, cognitive, and computational terms? Here, we discuss some recent findings related to these issues. We identify open questions and propose how to answer them through a combination of well-defined tasks, neurocomputational models, and neurophysiological probing of the interconnected loops of causes and consequences of pupil size.

A chromatic feature detector in the retina signals visual context changes

The retina transforms patterns of light into visual feature representations supporting behaviour. These representations are distributed across various types of retinal ganglion cells (RGCs), whose spatial and temporal tuning properties have been studied extensively in many model organisms, including the mouse. However, it has been difficult to link the potentially nonlinear retinal transformations of natural visual inputs to specific ethological purposes. Here, we discover a nonlinear selectivity to chromatic contrast in an RGC type that allows the detection of changes in visual context. We trained a convolutional neural network (CNN) model on large-scale functional recordings of RGC responses to natural mouse movies, and then used this model to search in silico for stimuli that maximally excite distinct types of RGCs. This procedure predicted centre colour opponency in transient suppressed-by-contrast (tSbC) RGCs, a cell type whose function is being debated. We confirmed experimentally that these cells indeed responded very selectively to Green-OFF, UV-ON contrasts. This type of chromatic contrast was characteristic of transitions from ground to sky in the visual scene, as might be elicited by head or eye movements across the horizon. Because tSbC cells performed best among all RGC types at reliably detecting these transitions, we suggest a role for this RGC type in providing contextual information (i.e. sky or ground) necessary for the selection of appropriate behavioural responses to other stimuli, such as looming objects. Our work showcases how a combination of experiments with natural stimuli and computational modelling allows discovering novel types of stimulus selectivity and identifying their potential ethological relevance.

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky

Color is an important visual feature that informs behavior, and the retinal basis for color vision has been studied across various vertebrate species. While many studies have investigated how color information is processed in visual brain areas of primate species, we have limited understanding of how it is organized beyond the retina in other species, including most dichromatic mammals. In this study, we systematically characterized how color is represented in the primary visual cortex (V1) of mice. Using large-scale neuronal recordings and a luminance and color noise stimulus, we found that more than a third of neurons in mouse V1 are color-opponent in their receptive field center, while the receptive field surround predominantly captures luminance contrast. Furthermore, we found that color-opponency is especially pronounced in posterior V1 that encodes the sky, matching the statistics of natural scenes experienced by mice. Using unsupervised clustering, we demonstrate that the asymmetry in color representations across cortex can be explained by an uneven distribution of green-On/UV-Off color-opponent response types that are represented in the upper visual field. Finally, a simple model with natural scene-inspired parametric stimuli shows that green-On/UV-Off color-opponent response types may enhance the detection of 'predatory'-like dark UV-objects in noisy daylight scenes. The results from this study highlight the relevance of color processing in the mouse visual system and contribute to our understanding of how color information is organized in the visual hierarchy across species.

Probabilistic neural transfer function estimation with Bayesian system identification

Neural population responses in sensory systems are driven by external physical stimuli. This stimulus-response relationship is typically characterized by receptive fields, which have been estimated by neural system identification approaches. Such models usually require a large amount of training data, yet, the recording time for animal experiments is limited, giving rise to epistemic uncertainty for the learned neural transfer functions. While deep neural network models have demonstrated excellent power on neural prediction, they usually do not provide the uncertainty of the resulting neural representations and derived statistics, such as most exciting inputs (MEIs), from in silico experiments. Here, we present a Bayesian system identification approach to predict neural responses to visual stimuli, and explore whether explicitly modeling network weight variability can be beneficial for identifying neural response properties. To this end, we use variational inference to estimate the posterior distribution of each model weight given the training data. Tests with different neural datasets demonstrate that this method can achieve higher or comparable performance on neural prediction, with a much higher data efficiency compared to Monte Carlo dropout methods and traditional models using point estimates of the model parameters. At the same time, our variational method provides us with an effectively infinite ensemble, avoiding the idiosyncrasy of any single model, to generate MEIs. This allows us to estimate the uncertainty of stimulus-response function, which we have found to be negatively correlated with the predictive performance at model level and may serve to evaluate models. Furthermore, our approach enables us to identify response properties with credible intervals and to determine whether the inferred features are meaningful by performing statistical tests on MEIs. Finally, in silico experiments show that our model generates stimuli driving neuronal activity significantly better than traditional models in the limited-data regime.

Natural visual behavior and active sensing in the mouse

In the natural world, animals use vision for a wide variety of behaviors not reflected in most laboratory paradigms. Although mice have low-acuity vision, they use their vision for many natural behaviors, including predator avoidance, prey capture, and navigation. They also perform active sensing, moving their head and eyes to achieve behavioral goals and acquire visual information. These aspects of natural vision result in visual inputs and corresponding behavioral outputs that are outside the range of conventional vision studies but are essential aspects of visual function. Here, we review recent studies in mice that have tapped into natural behavior and active sensing to reveal the computational logic of neural circuits for vision.

The vertebrate retina: a window into the evolution of computation in the brain

Animal brains are probably the most complex computational machines on our planet, and like everything in biology, they are the product of evolution. Advances in developmental and palaeobiology have been expanding our general understanding of how nervous systems can change at a molecular and structural level. However, how these changes translate into altered function - that is, into 'computation' - remains comparatively sparsely explored. What, concretely, does it mean for neuronal computation when neurons change their morphology and connectivity, when new neurons appear or old ones disappear, or when transmitter systems are slowly modified over many generations? And how does evolution use these many possible knobs and dials to constantly tune computation to give rise to the amazing diversity in animal behaviours we see today? Addressing these major gaps of understanding benefits from choosing a suitable model system. Here, I present the vertebrate retina as one perhaps unusually promising candidate. The retina is ancient and displays highly conserved core organisational principles across the entire vertebrate lineage, alongside a myriad of adjustments across extant species that were shaped by the history of their visual ecology. Moreover, the computational logic of the retina is readily interrogated experimentally, and our existing understanding of retinal circuits in a handful of species can serve as an anchor when exploring the visual circuit adaptations across the entire vertebrate tree of life, from fish deep in the aphotic zone of the oceans to eagles soaring high up in the sky.

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky

Color is an important visual feature that informs behavior, and the retinal basis for color vision has been studied across various vertebrate species. While many studies have investigated how color information is processed in visual brain areas of primate species, we have limited understanding of how it is organized beyond the retina in other species, including most dichromatic mammals. In this study, we systematically characterized how color is represented in the primary visual cortex (V1) of mice. Using large-scale neuronal recordings and a luminance and color noise stimulus, we found that more than a third of neurons in mouse V1 are color-opponent in their receptive field center, while the receptive field surround predominantly captures luminance contrast. Furthermore, we found that color-opponency is especially pronounced in posterior V1 that encodes the sky, matching the statistics of natural scenes experienced by mice. Using unsupervised clustering, we demonstrate that the asymmetry in color representations across cortex can be explained by an uneven distribution of green-On/UV-Off color-opponent response types that are represented in the upper visual field. Finally, a simple model with natural scene-inspired parametric stimuli shows that green-On/UV-Off color-opponent response types may enhance the detection of "predatory"-like dark UV-objects in noisy daylight scenes. The results from this study highlight the relevance of color processing in the mouse visual system and contribute to our understanding of how color information is organized in the visual hierarchy across species.

Efficient coding of natural images in the mouse visual cortex

How the activity of neurons gives rise to natural vision remains a matter of intense investigation. The mid-level visual areas along the ventral stream are selective to a common class of natural images-textures-but a circuit-level understanding of this selectivity and its link to perception remains unclear. We addressed these questions in mice, first showing that they can perceptually discriminate between textures and statistically simpler spectrally matched stimuli, and between texture types. Then, at the neural level, we found that the secondary visual area (LM) exhibited a higher degree of selectivity for textures compared to the primary visual area (V1). Furthermore, textures were represented in distinct neural activity subspaces whose relative distances were found to correlate with the statistical similarity of the images and the mice's ability to discriminate between them. Notably, these dependencies were more pronounced in LM, where the texture-related subspaces were smaller than in V1, resulting in superior stimulus decoding capabilities. Together, our results demonstrate texture vision in mice, finding a linking framework between stimulus statistics, neural representations, and perceptual sensitivity-a distinct hallmark of efficient coding computations.

Recommendations for measuring and standardizing light for laboratory mammals to improve welfare and reproducibility in animal research

Light enables vision and exerts widespread effects on physiology and behavior, including regulating circadian rhythms, sleep, hormone synthesis, affective state, and cognitive processes. Appropriate lighting in animal facilities may support welfare and ensure that animals enter experiments in an appropriate physiological and behavioral state. Furthermore, proper consideration of light during experimentation is important both when it is explicitly employed as an independent variable and as a general feature of the environment. This Consensus View discusses metrics to use for the quantification of light appropriate for nonhuman mammals and their application to improve animal welfare and the quality of animal research. It provides methods for measuring these metrics, practical guidance for their implementation in husbandry and experimentation, and quantitative guidance on appropriate light exposure for laboratory mammals. The guidance provided has the potential to improve data quality and contribute to reduction and refinement, helping to ensure more ethical animal use.

Ancestral photoreceptor diversity as the basis of visual behaviour

Animal colour vision is based on comparing signals from different photoreceptors. It is generally assumed that processing different spectral types of photoreceptor mainly serves colour vision. Here I propose instead that photoreceptors are parallel feature channels that differentially support visual-motor programmes like motion vision behaviours, prey capture and predator evasion. Colour vision may have emerged as a secondary benefit of these circuits, which originally helped aquatic vertebrates to visually navigate and segment their underwater world. Specifically, I suggest that ancestral vertebrate vision was built around three main systems, including a high-resolution general purpose greyscale system based on ancestral red cones and rods to mediate visual body stabilization and navigation, a high-sensitivity specialized foreground system based on ancestral ultraviolet cones to mediate threat detection and prey capture, and a net-suppressive system based on ancestral green and blue cones for regulating red/rod and ultraviolet circuits. This ancestral strategy probably still underpins vision today, and different vertebrate lineages have since adapted their original photoreceptor circuits to suit their diverse visual ecologies.

Views from 'crabworld': the spatial distribution of light in a tropical mudflat

Natural scene analysis has been extensively used to understand how the invariant structure of the visual environment may have shaped biological image processing strategies. This paper deals with four crucial, but hitherto largely neglected aspects of natural scenes: (1) the viewpoint of specific animals; (2) the fact that image statistics are not independent of the position within the visual field; (3) the influence of the direction of illumination on luminance, spectral and polarization contrast in a scene; and (4) the biologically relevant information content of natural scenes. To address these issues, I recorded the spatial distribution of light in a tropical mudflat with a spectrographic imager equipped with a polarizing filter in an attempt to describe quantitatively the visual environment of fiddler crabs. The environment viewed by the crabs has a distinct structure. Depending on the position of the sun, the luminance, the spectral composition, and the polarization characteristics of horizontal light distribution are not uniform. This is true for both skylight and for reflections from the mudflat surface. The high-contrast feature of the line of horizon dominates the vertical distribution of light and is a discontinuity in terms of luminance, spectral distribution and of image statistics. On a clear day, skylight intensity increases towards the horizon due to multiple scattering, and its spectral composition increasingly resembles that of sunlight. Sky-substratum contrast is highest at short wavelengths. I discuss the consequences of this extreme example of the topography of vision for extracting biologically relevant information from natural scenes.

Treatment with the dual-incretin agonist DA-CH5 demonstrates potent therapeutic effect in a rat model of Wolfram Syndrome

Aim

Wolfram Syndrome (WS) is a rare condition caused by mutations in Wfs1, with a poor prognosis and no cure. Mono-agonists targeting the incretin glucagon-like-peptide 1 (GLP-1) have demonstrated disease-modifying potential in pre-clinical and clinical settings. Dual agonists that target GLP-1 and glucose-dependent insulinotropic polypeptide (GIP-1) are reportedly more efficacious; hence, we evaluated the therapeutic potential of dual incretin agonism in a loss-of-function rat model of WS.

Methods

Eight-month-old Wfs1 knock-out (KO) and wild-type control rats were continuously treated with either the dual agonist DA-CH5 or saline for four months. Glycemic profile, visual acuity and hearing sensitivity were longitudinally monitored pre-treatment, and then at 10.5 and 12 months. Pancreata and retina were harvested for immunohistological analysis.

Results

DA-CH5 therapy reversed glucose intolerance in KO rats and provided lasting anti-diabetogenic protection. Treatment also reversed intra-islet alterations, including reduced endocrine islet area and β-cell density, indicating its regenerative potential. Although no rescue effect was noted for hearing loss, visual acuity and retinal ganglion cell density were better preserved in DA-CH5-treated rats.

Conclusion

We present preclinical evidence for the pleiotropic therapeutic effects of long-term dual incretin agonist treatment; effects were seen despite treatment beginning after symptom-onset, indicating reversal of disease progression. Dual incretins represent a promising therapeutic avenue for WS patients.

Functional subtypes of rodent melanopsin ganglion cells switch roles between night and day illumination

Intrinsically photosensitive retinal ganglion cells (ipRGCs), contain the photopigment melanopsin, and influence both image and non-image forming behaviors. Despite being categorized into multiple types (M1-M6), physiological variability within these types suggests our current understanding of ipRGCs is incomplete. We used multi-electrode array (MEA) recordings and unbiased cluster analysis under synaptic blockade to identify 8 functional clusters of ipRGCs, each with distinct photosensitivity and response timing. We used Cre mice to drive the expression of channelrhodopsin in SON-ipRGCs, enabling the localization of distinct ipRGCs in the dorsal retina. Additionally, we conducted a retrospective unbiased cluster analysis of ipRGC photoresponses to light stimuli across scotopic, mesopic, and photopic intensities, aimed at activating both rod and cone inputs to ipRGCs. Our results revealed shared and distinct synaptic inputs to the identified functional clusters, demonstrating that ipRGCs encode visual information with high fidelity at low light intensities, but poorly at photopic light intensities, when melanopsin activation is highest. Collectively, our findings support a framework with at least 8 functional subtypes of ipRGCs, each encoding luminance with distinct spike outputs, highlighting the inherent functional diversity and complexity of ipRGCs and suggesting a reevaluation of their contributions to retinal function and visual perception under varying light conditions.

Spatiotemporal visual statistics of aquatic environments in the natural habitats of zebrafish

Animal sensory systems are tightly adapted to the demands of their environment. In the visual domain, research has shown that many species have circuits and systems that exploit statistical regularities in natural visual signals. The zebrafish is a popular model animal in visual neuroscience, but relatively little quantitative data is available about the visual properties of the aquatic habitats where zebrafish reside, as compared to terrestrial environments. Improving our understanding of the visual demands of the aquatic habitats of zebrafish can enhance the insights about sensory neuroscience yielded by this model system. We analyzed a video dataset of zebrafish habitats captured by a stationary camera and compared this dataset to videos of terrestrial scenes in the same geographic area. Our analysis of the spatiotemporal structure in these videos suggests that zebrafish habitats are characterized by low visual contrast and strong motion when compared to terrestrial environments. Similar to terrestrial environments, zebrafish habitats tended to be dominated by dark contrasts, particularly in the lower visual field. We discuss how these properties of the visual environment can inform the study of zebrafish visual behavior and neural processing and, by extension, can inform our understanding of the vertebrate brain.

A three-photon head-mounted microscope for imaging all layers of visual cortex in freely moving mice

Advances in head-mounted microscopes have enabled imaging of neuronal activity using genetic tools in freely moving mice but these microscopes are restricted to recording in minimally lit arenas and imaging upper cortical layers. Here we built a 2-g, three-photon excitation-based microscope, containing a z-drive that enabled access to all cortical layers while mice freely behaved in a fully lit environment. The microscope had on-board photon detectors, robust to environmental light, and the arena lighting was timed to the end of each line-scan, enabling functional imaging of activity from cortical layer 4 and layer 6 neurons expressing jGCaMP7f in mice roaming a fully lit or dark arena. By comparing the neuronal activity measured from populations in these layers we show that activity in cortical layer 4 and layer 6 is differentially modulated by lit and dark conditions during free exploration.

Panoramic visual statistics shape retina-wide organization of receptive fields

Statistics of natural scenes are not uniform-their structure varies dramatically from ground to sky. It remains unknown whether these nonuniformities are reflected in the large-scale organization of the early visual system and what benefits such adaptations would confer. Here, by relying on the efficient coding hypothesis, we predict that changes in the structure of receptive fields across visual space increase the efficiency of sensory coding. Using the mouse (Mus musculus) as a model species, we show that receptive fields of retinal ganglion cells change their shape along the dorsoventral retinal axis, with a marked surround asymmetry at the visual horizon, in agreement with our predictions. Our work demonstrates that, according to principles of efficient coding, the panoramic structure of natural scenes is exploited by the retina across space and cell types.

A melanopsin ganglion cell subtype forms a dorsal retinal mosaic projecting to the supraoptic nucleus

Visual input to the hypothalamus from intrinsically photosensitive retinal ganglion cells (ipRGCs) influences several functions including circadian entrainment, body temperature, and sleep. ipRGCs also project to nuclei such as the supraoptic nucleus (SON), which is involved in systemic fluid homeostasis, maternal behavior, social behaviors, and appetite. However, little is known about the SON-projecting ipRGCs or their relationship to well-characterized ipRGC subtypes. Using a GlyT2Cre mouse line, we show a subtype of ipRGCs restricted to the dorsal retina that selectively projects to the SON. These ipRGCs tile a dorsal region of the retina, forming a substrate for encoding ground luminance. Optogenetic activation of their axons demonstrates they release the neurotransmitter glutamate in multiple regions, including the suprachiasmatic nucleus (SCN) and SON. Our results challenge the idea that ipRGC dendrites overlap to optimize photon capture and suggests non-image forming vision operates to sample local regions of the visual field to influence diverse behaviors.

RetinaMOT: rethinking anchor-free YOLOv5 for online multiple object tracking

In recent years, YOLOv5 networks have become a research focus in many fields because they are capable of outperforming state-of-the-art (SOTA) approaches in different computer vision tasks. Nevertheless, there is still room for improvement in YOLOv5 in terms of target tracking. We modified YOLOv5 according to the anchor-free paradigm to be on par with other state-of-the-art tracking paradigms and modified the network backbone to design an efficient module, thus proposing the RetinaYOLO detector, which, after combining state-of-the-art tracking algorithms, achieves state-of-the-art performance: we call it RetinaMOT. To the best of our knowledge, RetinaMOT is the first such approach. The anchor-free paradigm SOTA method for the YOLOv5 architecture and RetinaYOLO outperforms all lightweight YOLO architecture methods on the MS COCO dataset. In this paper, we show the details of the RetinaYOLO backbone, embedding Kalman filtering and the Hungarian algorithm into the network, with one framework used to accomplish two tasks. Our RetinaMOT shows that MOTA metrics reach 74.8, 74.1, and 66.8 on MOT Challenge MOT16, 17, and 20 test datasets, and our method is at the top of the list when compared with state-of-the-art methods.

Joint representations of color and form in mouse visual cortex described by random pooling from rods and cones

Spatial transitions in color can aid any visual perception task, and its neural representation, the "integration of color and form," is thought to begin at primary visual cortex (V1). Integration of color and form is untested in mouse V1, yet studies show that the ventral retina provides the necessary substrate from green-sensitive rods and ultraviolet-sensitive cones. Here, we used two-photon imaging in V1 to measure spatial frequency (SF) tuning along four axes of rod and cone contrast space, including luminance and color. We first reveal that V1's sensitivity to color is similar to luminance, yet average SF tuning is significantly shifted lowpass for color. Next, guided by linear models, we used SF tuning along all four color axes to estimate the proportion of neurons that fall into classic models of color opponency, i.e., "single-," "double-," and "non-opponent." Few neurons (∼6%) fit the criteria for double opponency, which are uniquely tuned for chromatic borders. Most of the population can be described as a unimodal distribution ranging from strongly single-opponent to non-opponent. Consistent with recent studies of the rodent and primate retina, our V1 data are well-described by a simple model in which ON and OFF channels to V1 sample the photoreceptor mosaic randomly. Finally, an analysis comparing color opponency to preferred orientation and retinotopy further validates rods, and not cone M-opsin, as opponent with cone S-opsin in the upper visual field.NEW & NOTEWORTHY This study is the first to show that mouse V1 is highly sensitive to UV-green color contrast. Furthermore, it provides a detailed characterization of "color opponency," which is the putative neural basis for color perception. Finally, using an extremely simple yet novel random wiring model, we account for our observations.

State-dependent pupil dilation rapidly shifts visual feature selectivity

To increase computational flexibility, the processing of sensory inputs changes with behavioural context. In the visual system, active behavioural states characterized by motor activity and pupil dilation1,2 enhance sensory responses, but typically leave the preferred stimuli of neurons unchanged2-9. Here we find that behavioural state also modulates stimulus selectivity in the mouse visual cortex in the context of coloured natural scenes. Using population imaging in behaving mice, pharmacology and deep neural network modelling, we identified a rapid shift in colour selectivity towards ultraviolet stimuli during an active behavioural state. This was exclusively caused by state-dependent pupil dilation, which resulted in a dynamic switch from rod to cone photoreceptors, thereby extending their role beyond night and day vision. The change in tuning facilitated the decoding of ethological stimuli, such as aerial predators against the twilight sky10. For decades, studies in neuroscience and cognitive science have used pupil dilation as an indirect measure of brain state. Our data suggest that, in addition, state-dependent pupil dilation itself tunes visual representations to behavioural demands by differentially recruiting rods and cones on fast timescales.

High-density electrode recordings reveal strong and specific connections between retinal ganglion cells and midbrain neurons

The superior colliculus is a midbrain structure that plays important roles in visually guided behaviors in mammals. Neurons in the superior colliculus receive inputs from retinal ganglion cells but how these inputs are integrated in vivo is unknown. Here, we discovered that high-density electrodes simultaneously capture the activity of retinal axons and their postsynaptic target neurons in the superior colliculus, in vivo. We show that retinal ganglion cell axons in the mouse provide a single cell precise representation of the retina as input to superior colliculus. This isomorphic mapping builds the scaffold for precise retinotopic wiring and functionally specific connection strength. Our methods are broadly applicable, which we demonstrate by recording retinal inputs in the optic tectum in zebra finches. We find common wiring rules in mice and zebra finches that provide a precise representation of the visual world encoded in retinal ganglion cells connections to neurons in retinorecipient areas.

The vertical light-gradient and its potential impact on animal distribution and behavior

The visual environment provides vital cues allowing animals to assess habitat quality, weather conditions or measure time of day. Together with other sensory cues and physiological conditions, the visual environment sets behavioral states that make the animal more prone to engage in some behaviors, and less in others. This master-control of behavior serves a fundamental and essential role in determining the distribution and behavior of all animals. Although it is obvious that visual information contains vital input for setting behavioral states, the precise nature of these visual cues remains unknown. Here we use a recently described method to quantify the distribution of light reaching animals’ eyes in different environments. The method records the vertical gradient (as a function of elevation angle) of intensity, spatial structure and spectral balance. Comparison of measurements from different types of environments, weather conditions, times of day, and seasons reveal that these aspects can be readily discriminated from one another. The vertical gradients of radiance, spatial structure (contrast) and color are thus reliable indicators that are likely to have a strong impact on animal behavior and spatial distribution.

Retinal Encoding of Natural Scenes

An ultimate goal in retina science is to understand how the neural circuit of the retina processes natural visual scenes. Yet most studies in laboratories have long been performed with simple, artificial visual stimuli such as full-field illumination, spots of light, or gratings. The underlying assumption is that the features of the retina thus identified carry over to the more complex scenario of natural scenes. As the application of corresponding natural settings is becoming more commonplace in experimental investigations, this assumption is being put to the test and opportunities arise to discover processing features that are triggered by specific aspects of natural scenes. Here, we review how natural stimuli have been used to probe, refine, and complement knowledge accumulated under simplified stimuli, and we discuss challenges and opportunities along the way toward a comprehensive understanding of the encoding of natural scenes. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

What and Where: Location-Dependent Feature Sensitivity as a Canonical Organizing Principle of the Visual System

Traditionally, functional representations in early visual areas are conceived as retinotopic maps preserving ego-centric spatial location information while ensuring that other stimulus features are uniformly represented for all locations in space. Recent results challenge this framework of relatively independent encoding of location and features in the early visual system, emphasizing location-dependent feature sensitivities that reflect specialization of cortical circuits for different locations in visual space. Here we review the evidence for such location-specific encoding including: (1) systematic variation of functional properties within conventional retinotopic maps in the cortex; (2) novel periodic retinotopic transforms that dramatically illustrate the tight linkage of feature sensitivity, spatial location, and cortical circuitry; and (3) retinotopic biases in cortical areas, and groups of areas, that have been defined by their functional specializations. We propose that location-dependent feature sensitivity is a fundamental organizing principle of the visual system that achieves efficient representation of positional regularities in visual experience, and reflects the evolutionary selection of sensory and motor circuits to optimally represent behaviorally relevant information. Future studies are necessary to discover mechanisms underlying joint encoding of location and functional information, how this relates to behavior, emerges during development, and varies across species.

Face perception: computational insights from phylogeny

Studies of face perception in primates elucidate the psychological and neural mechanisms that support this critical and complex ability. Recent progress in characterizing face perception across species, for example in insects and reptiles, has highlighted the ubiquity over phylogeny of this key ability for social interactions and survival. Here, we review the competence in face perception across species and the types of computation that support this behavior. We conclude that the computational complexity of face perception evinced by a species is not related to phylogenetic status and is, instead, largely a product of environmental context and social and adaptive pressures. Integrating findings across evolutionary data permits the derivation of computational principles that shed further light on primate face perception.

Retinal patterns and the cellular repertoire of neuropsin (Opn5) retinal ganglion cells

Obtaining a parts list of the sensory components of the retina is vital to understanding the effects of light in behavior, health, and disease. Rods, cones, and intrinsically photosensitive retinal ganglion cells (ipRGCs) are the best described photoreceptors in the mammalian retina, but recent functional roles have been proposed for retinal neuropsin (Opn5) - an atypical opsin. However, little is known about the pattern of Opn5 expression in the retina. Using cre (Opn5^cre ) and cre-dependent reporters, we uncover patterns of Opn5 expression and find that Opn5 is restricted to retinal ganglion cells (RGCs). Opn5-RGCs are non-homogenously distributed through the retina, with greater densities of cells located in the dorsotemporal quadrant. In addition to local topology of these cells, using cre-dependent AAV viral tracing, we surveyed their central targets and found that they are biased towards image-forming and image-stabilizing regions. Finally, molecular and electrophysiological profiling reveal that Opn5-RGCs comprise previously defined RGC types which respond optimally to edges and object-motion (F-mini-ONs, HD2, HD1, LEDs, ooDSRGCs, etc.). Together, these data describe the second collection of RGCs that express atypical opsins in the mouse, and expand the roles of image-forming cells in retinal physiology and function. This article is protected by copyright. All rights reserved.

Ancestral circuits for vertebrate color vision emerge at the first retinal synapse

For color vision, retinal circuits separate information about intensity and wavelength. In vertebrates that use the full complement of four “ancestral” cone types, the nature and implementation of this computation remain poorly understood. Here, we establish the complete circuit architecture of outer retinal circuits underlying color processing in larval zebrafish. We find that the synaptic outputs of red and green cones efficiently rotate the encoding of natural daylight in a principal components analysis–like manner to yield primary achromatic and spectrally opponent axes, respectively. Blue cones are tuned to capture most remaining variance when opposed to green cones, while UV cone present a UV achromatic axis for prey capture. We note that fruitflies use essentially the same strategy. Therefore, rotating color space into primary achromatic and chromatic axes at the eye’s first synapse may thus be a fundamental principle of color vision when using more than two spectrally well-separated photoreceptor types.

Mouse vision: La vie en ultraviolet

Visual systems evolve to extract salient information from natural scenes. A new study demonstrates that the specialized organization of ultraviolet-sensitive photoreceptors in the mouse retina optimizes the visibility of aerial predators within the upper visual scenes experienced by mice.

Color vision: More than meets the eye

Mice can discriminate color, but unlike in primates, studies have so far failed to find robust cone-opponent cells in the retina. A new study shows that a sub-region of the mouse visual thalamus is specialized for processing color.