Structure and function come unglued in the visual cortex

Double-opponent spiking neuron array with orientation selectivity for encoding and spatial-chromatic processing

Color spiking encoding and opponent preprocessing are critical for energy-efficient object perception in the human visual system. Emulating the retina and brain's integration of spatial and chromatic spiking signals holds promise for enhancing the efficiency of vision sensors. Here, we introduce an artificial visual neuron array that generates excitatory or inhibitory spiking responses to specific wavelengths with orientation selectivity. The neuron array can function as double-opponent receptive fields for spatial-chromatic opponent preprocessing to color signals, emulating the neural pathway from the retina to the cortex. With the color spiking preprocessing function of the neuron array, the recognition accuracy is improved almost twofold compared to direct perception of underexposure objects, and the noise robustness is also strengthened. This architecture leverages biological mechanisms for simultaneous spike encoding and antagonistic preprocessing of color information, offering the potential for highly efficient neuromorphic vision systems.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.

Circular and unified analysis in network neuroscience

Genuinely new discovery transcends existing knowledge. Despite this, many analyses in systems neuroscience neglect to test new speculative hypotheses against benchmark empirical facts. Some of these analyses inadvertently use circular reasoning to present existing knowledge as new discovery. Here, I discuss that this problem can confound key results and estimate that it has affected more than three thousand studies in network neuroscience over the last decade. I suggest that future studies can reduce this problem by limiting the use of speculative evidence, integrating existing knowledge into benchmark models, and rigorously testing proposed discoveries against these models. I conclude with a summary of practical challenges and recommendations.

The visible gorilla: Unexpected fast-not physically salient-Objects are noticeable

It is widely believed that observers can fail to notice clearly visible unattended objects, even if they are moving. Here, we created parametric tasks to test this belief and report the results of three high-powered experiments (total n = 4,493) indicating that this effect is strongly modulated by the speed of the unattended object. Specifically, fast-but not slow-objects are readily noticeable, whether they are attended or not. These results suggest that fast motion serves as a potent exogenous cue that overrides task-focused attention, showing that fast speeds, not long exposure duration or physical salience, strongly diminish inattentional blindness effects.

A Survey of Brain-Inspired Intelligent Robots: Integration of Vision, Decision, Motion Control, and Musculoskeletal Systems

Current robotic studies are focused on the performance of specific tasks. However, such tasks cannot be generalized, and some special tasks, such as compliant and precise manipulation, fast and flexible response, and deep collaboration between humans and robots, cannot be realized. Brain-inspired intelligent robots imitate humans and animals, from inner mechanisms to external structures, through an integration of visual cognition, decision making, motion control, and musculoskeletal systems. This kind of robot is more likely to realize the functions that current robots cannot realize and become human friends. With the focus on the development of brain-inspired intelligent robots, this article reviews cutting-edge research in the areas of brain-inspired visual cognition, decision making, musculoskeletal robots, motion control, and their integration. It aims to provide greater insight into brain-inspired intelligent robots and attracts more attention to this field from the global research community.

High-Level Visual Encoding Model Framework with Hierarchical Ventral Stream-Optimized Neural Networks

Visual encoding models based on deep neural networks (DNN) show good performance in predicting brain activity in low-level visual areas. However, due to the amount of neural data limitation, DNN-based visual encoding models are difficult to fit for high-level visual areas, resulting in insufficient encoding performance. The ventral stream suggests that higher visual areas receive information from lower visual areas, which is not fully reflected in the current encoding models. In the present study, we propose a novel visual encoding model framework which uses the hierarchy of representations in the ventral stream to improve the model’s performance in high-level visual areas. Under the framework, we propose two categories of hierarchical encoding models from the voxel and the feature perspectives to realize the hierarchical representations. From the voxel perspective, we first constructed an encoding model for the low-level visual area (V1 or V2) and extracted the voxel space predicted by the model. Then we use the extracted voxel space of the low-level visual area to predict the voxel space of the high-level visual area (V4 or LO) via constructing a voxel-to-voxel model. From the feature perspective, the feature space of the first model is extracted to predict the voxel space of the high-level visual area. The experimental results show that two categories of hierarchical encoding models effectively improve the encoding performance in V4 and LO. In addition, the proportion of the best-encoded voxels for different models in V4 and LO show that our proposed models have obvious advantages in prediction accuracy. We find that the hierarchy of representations in the ventral stream has a positive effect on improving the performance of the existing model in high-level visual areas.

Hierarchical Predictive Coding-Based JND Estimation for Image Compression

The human visual system (HVS) is a hierarchical system, in which visual signals are processed hierarchically. In this paper, the HVS is modeled as a three-level communication system and visual perception is divided into three stages according to the hierarchical predictive coding theory. Then, a novel just noticeable distortion (JND) estimation scheme is proposed. In visual perception, the input signals are predicted constantly and spontaneously in each hierarchy, and neural response is evoked by the central residue and inhibited by surrounding residues. These two types' residues are regarded as the positive and negative visual incentives which cause positive and negative perception effects, respectively. In neuroscience, the effect of incentive on observer is measured by the surprise of this incentive. Thus, we propose a surprise-based measurement method to measure both perception effects. Specifically, considering the biased competition of visual attention, we define the product of the residue self-information (i.e., surprise) and the competition biases as the perceptual surprise to measure the positive perception effect. As for the negative perception effect, it is measured by the average surprise (i.e., the local Shannon entropy). The JND threshold of each stage is estimated individually by considering both perception effects. The total JND threshold is finally obtained by non-linear superposition of three stage thresholds. Furthermore, the proposed JND estimation scheme is incorporated into the codec of Versatile Video Coding for image compression. Experimental results show that the proposed JND model outperforms the relevant existing ones, and over 16% of bit rate can be reduced without jeopardizing the perceptual quality.

Anatomy of nerve fiber bundles at micrometer-resolution in the vervet monkey visual system

Although the primate visual system has been extensively studied, detailed spatial organization of white matter fiber tracts carrying visual information between areas has not been fully established. This is mainly due to the large gap between tracer studies and diffusion-weighted MRI studies, which focus on specific axonal connections and macroscale organization of fiber tracts, respectively. Here we used 3D polarization light imaging (3D-PLI), which enables direct visualization of fiber tracts at micrometer resolution, to identify and visualize fiber tracts of the visual system, such as stratum sagittale, inferior longitudinal fascicle, vertical occipital fascicle, tapetum and dorsal occipital bundle in vervet monkey brains. Moreover, 3D-PLI data provide detailed information on cortical projections of these tracts, distinction between neighboring tracts, and novel short-range pathways. This work provides essential information for interpretation of functional and diffusion-weighted MRI data, as well as revision of wiring diagrams based upon observations in the vervet visual system.

Spatial organization of occipital white matter tracts in the common marmoset

The primate brain contains a large number of interconnected visual areas, whose spatial organization and intracortical projections show a high level of conservation across species. One fiber pathway of recent interest is the vertical occipital fasciculus (VOF), which is thought to support communication between dorsal and ventral visual areas in the occipital lobe. A recent comparative diffusion MRI (dMRI) study reported that the VOF in the macaque brain bears a similar topology to that of the human, running superficial and roughly perpendicular to the optic radiation. The present study reports a comparative investigation of the VOF in the common marmoset, a small New World monkey whose lissencephalic brain is approximately tenfold smaller than the macaque and 150-fold smaller than the human. High-resolution ex vivo dMRI of two marmoset brains revealed an occipital white matter structure that closely resembles that of the larger primate species, with one notable difference. Namely, unlike in the macaque and the human, the VOF in the marmoset is spatially fused with other, more anterior vertical tracts, extending anteriorly between the parietal and temporal cortices. We compare several aspects of this continuous structure, which we term the VOF complex (VOF +), and neighboring fasciculi to those of macaques and humans. We hypothesize that the essential topology of the VOF+ is a conserved feature of the posterior cortex in anthropoid primates, with a clearer fragmentation into multiple named fasciculi in larger, more gyrified brains.

A Large-Scale Circuit Mechanism for Hierarchical Dynamical Processing in the Primate Cortex

We developed a large-scale dynamical model of the macaque neocortex, which is based on recently acquired directed- and weighted-connectivity data from tract-tracing experiments, and which incorporates heterogeneity across areas. A hierarchy of timescales naturally emerges from this system: sensory areas show brief, transient responses to input (appropriate for sensory processing), whereas association areas integrate inputs over time and exhibit persistent activity (suitable for decision-making and working memory). The model displays multiple temporal hierarchies, as evidenced by contrasting responses to visual versus somatosensory stimulation. Moreover, slower prefrontal and temporal areas have a disproportionate impact on global brain dynamics. These findings establish a circuit mechanism for "temporal receptive windows" that are progressively enlarged along the cortical hierarchy, suggest an extension of time integration in decision making from local to large circuits, and should prompt a re-evaluation of the analysis of functional connectivity (measured by fMRI or electroencephalography/magnetoencephalography) by taking into account inter-areal heterogeneity.