Speed of processing in the human visual system

Perceptual encoding benefit of visual memorability on visual memory formation

Human observers often exhibit remarkable consistency in remembering specific visual details, such as certain face images. This phenomenon is commonly attributed to visual memorability, a collection of stimulus attributes that enhance the long-term retention of visual information. However, the exact contributions of visual memorability to visual memory formation remain elusive as these effects could emerge anywhere from early perceptual encoding to post-perceptual memory consolidation processes. To clarify this, we tested three key predictions from the hypothesis that visual memorability facilitates early perceptual encoding that supports the formation of visual short-term memory (VSTM) and the retention of visual long-term memory (VLTM). First, we examined whether memorability benefits in VSTM encoding manifest early, even within the constraints of a brief stimulus presentation (100-200 ms; Experiment 1). We achieved this by manipulating stimulus presentation duration in a VSTM change detection task using face images with high- or low-memorability while ensuring they were equally familiar to the participants. Second, we assessed whether this early memorability benefit increases the likelihood of VSTM retention, even with post-stimulus masking designed to interrupt post-perceptual VSTM consolidation processes (Experiment 2). Last, we investigated the durability of memorability benefits by manipulating memory retention intervals from seconds to 24 h (Experiment 3). Across experiments, our data suggest that visual memorability has an early impact on VSTM formation, persisting across variable retention intervals and predicting subsequent VLTM overnight. Combined, these findings highlight that visual memorability enhances visual memory within 100-200 ms following stimulus onset, resulting in robust memory traces resistant to post-perceptual interruption and long-term forgetting.

Behavioral and neural measures of semantic conflict monitoring: Findings from a novel picture-word interference task

Conflict monitoring has been studied extensively using experimental paradigms that manipulate perceptual dimensions of stimuli and responses. The picture-word interference (PWI) task has historically been used to examine semantic conflict, but primarily for the purpose of examining lexical retrieval. In this study, we utilized two novel PWI tasks to assess conflict monitoring in the context of semantic conflict. Participants included nineteen young adults (14F, age = 20.79 ± 3.14) who completed two tasks: Animals and Objects. Task and conflict effects were assessed by examining behavioral (reaction time and accuracy) and neurophysiological (oscillations in theta, alpha, and beta band) measures. Results revealed conflict effects within both tasks, but the pattern of findings differed across the two semantic categories. Participants were slower to respond to unmatched versus matched trials on the Objects task only and were less accurate responding to matched versus unmatched trials in the Animals task only. We also observed task differences, with participants responding more accurately on conflict trials for Animals compared to Objects. Differences in neural oscillations were observed, including between-task differences in low beta oscillations and within-task differences in theta, alpha, and low beta. We also observed significant correlations between task performance and standard measures of cognitive control. This work provides new insights into conflict monitoring, highlighting the importance of examining conflict across different semantic categories, especially in the context of animacy. The findings serve as a benchmark to assess conflict monitoring using PWI tasks across populations of varying cognitive ability.

Evaluating the impact of different hard hats on the peripheral vision of construction workers

Construction safety is always an issue for construction workers. Hence, personal protective equipment plays a critical role in avoiding potential hazards on construction sites. Among these, hard hats protect against head contact with falling objects on construction sites. This study aims to examine the effects of hard hats with different peak lengths on the field of view at different angles in the upward part and to examine the effects of possible field of view losses caused by the hard hat on the reaction times of the workers. A questionnaire was designed and administered to the construction workers. Experiments were then conducted with a group of subjects to assess their peripheral vision level as well as reaction times. The study found that peripheral vision is affected by the peak size of hard hats. The study further revealed that there is a significant relationship between reaction times and hard hat peak size.

A treadmill running research protocol to assess dynamic visual acuity and balance for athletes with and without recent concussion history

Athletes interpret dynamic visual scenes quickly and accurately during physical exertion. It is important to understand how increased exertion may impact vision and cognition following sport-related concussion (SRC).Purpose To examine the effect of a treadmill running research protocol on the assessment of dynamic visual acuity (DVA) and balance for athletes with and without recent history of SRC.Methods Varsity athletes following recent SRC (CONC=12) were compared to athletes without SRC (ATHLETE=19). The DVA task presented a Tumbling 'E' target in four possible orientations during random walk (RW) or horizontal (H) motion at a speed of 30°/s. Participants performed DVA trials standing on a force plate (1000Hz) at four time points: 1) pre-exercise (PRE-EX), 2) immediately (POST1), 3) 10-minutes (POST10), and 4) 20-minutes post- exercise (POST20). Performance was calculated as a change in DVA score from PRE-EX and median response time (RT, ms). Balance control was analyzed using the root mean square of centre of pressure displacement (dCOP).Results Both groups maintained DVA scores for both motion types and exhibited immediate exercise-induced benefits on RT. Both groups had similar change in balance control strategy following treadmill exercise.Conclusion Both groups elicited similar exercise-induced benefits on DVA following exercise. A repeated measures assessment following vigorous exercise may provide meaningful insights about visual and neurocognitive functions for athletes returning to sport following concussion.

Robust inference of causality in high-dimensional dynamical processes from the Information Imbalance of distance ranks

We introduce an approach which allows detecting causal relationships between variables for which the time evolution is available. Causality is assessed by a variational scheme based on the Information Imbalance of distance ranks, a statistical test capable of inferring the relative information content of different distance measures. We test whether the predictability of a putative driven system Y can be improved by incorporating information from a potential driver system X, without explicitly modeling the underlying dynamics and without the need to compute probability densities of the dynamic variables. This framework makes causality detection possible even between high-dimensional systems where only few of the variables are known or measured. Benchmark tests on coupled chaotic dynamical systems demonstrate that our approach outperforms other model-free causality detection methods, successfully handling both unidirectional and bidirectional couplings. We also show that the method can be used to robustly detect causality in human electroencephalography data.

Overt Word Reading and Visual Object Naming in Adults with Dyslexia: Electroencephalography Study in Transparent Orthography

The study aimed to investigate overt reading and naming processes in adult people with dyslexia (PDs) in shallow (transparent) language orthography. The results of adult PDs are compared with adult healthy controls HCs. Comparisons are made in three phases: pre-lexical (150-260 ms), lexical (280-700 ms), and post-lexical stage of processing (750-1000 ms) time window. Twelve PDs and HCs performed overt reading and naming tasks under EEG recording. The word reading and naming task consisted of sparse neighborhoods with closed phonemic onset (words/objects sharing the same onset). For the analysis of the mean ERP amplitude for pre-lexical, lexical, and post-lexical time window, a mixed design ANOVA was performed with the right (F4, FC2, FC6, C4, T8, CP2, CP6, P4) and left (F3, FC5, FC1, T7, C3, CP5, CP1, P7, P3) electrode sites, within-subject factors and group (PD vs. HC) as between-subject factor. Behavioral response latency results revealed significantly prolonged reading latency between HCs and PDs, while no difference was detected in naming response latency. ERP differences were found between PDs and HCs in the right hemisphere's pre-lexical time window (160-200 ms) for word reading aloud. For visual object naming aloud, ERP differences were found between PDs and HCs in the right hemisphere's post-lexical time window (900-1000 ms). The present study demonstrated different distributions of the electric field at the scalp in specific time windows between two groups in the right hemisphere in both word reading and visual object naming aloud, suggesting alternative processing strategies in adult PDs. These results indirectly support the view that adult PDs in shallow language orthography probably rely on the grapho-phonological route during overt word reading and have difficulties with phoneme and word retrieval during overt visual object naming in adulthood.

Have you been there before? Decoding recognition of spatial scenes from fMRI signals in precuneus

One potential application of forensic "brain reading" is to test whether a suspect has previously experienced a crime scene. Here, we investigated whether it is possible to decode real life autobiographic exposure to spatial locations using fMRI. In the first session, participants visited four out of eight possible rooms on a university campus. During a subsequent scanning session, subjects passively viewed pictures and videos from these eight possible rooms (four old, four novel) without giving any responses. A multivariate searchlight analysis was employed that trained a classifier to distinguish between "seen" versus "unseen" stimuli from a subset of six rooms. We found that bilateral precuneus encoded information that can be used to distinguish between previously seen and unseen rooms and that also generalized to the two stimuli left out from training. We conclude that activity in bilateral precuneus is associated with the memory of previously visited rooms, irrespective of the identity of the room, thus supporting a parietal contribution to episodic memory for spatial locations. Importantly, we could decode whether a room was visited in real life without the need of explicit judgments about the rooms. This suggests that recognition is an automatic response that can be decoded from fMRI data, thus potentially supporting forensic applications of concealed information tests for crime scene recognition.

Social prioritisation in scene viewing and the effects of a spatial memory load

When free-viewing scenes, participants tend to preferentially fixate social elements (e.g., people). In the present study, we tested whether this bias would be disrupted by increasing the demands of a secondary dual-task: holding a set of (one or six) spatial locations in memory, presented either simultaneously or sequentially. Following a retention interval, participants judged whether a test location was present in the to-be-remembered stimuli. During the retention interval participants free-viewed scenes containing a social element (a person) and a non-social element (an object) that served as regions of interest. In order to assess the impact of physical salience, the non-social element was presented in both an unaltered baseline version, and in a version where its salience was artificially increased. The results showed that the preference to look at social elements decreased when the demands of the spatial memory task were increased from one to six locations, regardless of presentation mode (simultaneous or sequential). The high-load condition also resulted in more central fixations and reduced exploration of the scene. The results indicate that the social prioritisation effect, and scene viewing more generally, can be affected by a concurrent memory load.

A Hybrid Neural Coding Approach for Pattern Recognition With Spiking Neural Networks

Recently, brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. However, these SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. Given that each neural coding scheme possesses its own merits and drawbacks, these SNNs encounter challenges in achieving optimal performance such as accuracy, response time, efficiency, and robustness, all of which are crucial for practical applications. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes. As an initial exploration in this direction, we propose a hybrid neural coding and learning framework, which encompasses a neural coding zoo with diverse neural coding schemes discovered in neuroscience. Additionally, it incorporates a flexible neural coding assignment strategy to accommodate task-specific requirements, along with novel layer-wise learning methods to effectively implement hybrid coding SNNs. We demonstrate the superiority of the proposed framework on image classification and sound localization tasks. Specifically, the proposed hybrid coding SNNs achieve comparable accuracy to state-of-the-art SNNs, while exhibiting significantly reduced inference latency and energy consumption, as well as high noise robustness. This study yields valuable insights into hybrid neural coding designs, paving the way for developing high-performance neuromorphic systems.

Predictive coding with spiking neurons and feedforward gist signaling

Predictive coding (PC) is an influential theory in neuroscience, which suggests the existence of a cortical architecture that is constantly generating and updating predictive representations of sensory inputs. Owing to its hierarchical and generative nature, PC has inspired many computational models of perception in the literature. However, the biological plausibility of existing models has not been sufficiently explored due to their use of artificial neurons that approximate neural activity with firing rates in the continuous time domain and propagate signals synchronously. Therefore, we developed a spiking neural network for predictive coding (SNN-PC), in which neurons communicate using event-driven and asynchronous spikes. Adopting the hierarchical structure and Hebbian learning algorithms from previous PC neural network models, SNN-PC introduces two novel features: (1) a fast feedforward sweep from the input to higher areas, which generates a spatially reduced and abstract representation of input (i.e., a neural code for the gist of a scene) and provides a neurobiological alternative to an arbitrary choice of priors; and (2) a separation of positive and negative error-computing neurons, which counters the biological implausibility of a bi-directional error neuron with a very high baseline firing rate. After training with the MNIST handwritten digit dataset, SNN-PC developed hierarchical internal representations and was able to reconstruct samples it had not seen during training. SNN-PC suggests biologically plausible mechanisms by which the brain may perform perceptual inference and learning in an unsupervised manner. In addition, it may be used in neuromorphic applications that can utilize its energy-efficient, event-driven, local learning, and parallel information processing nature.

Fast saccades to faces during the feedforward sweep

Saccadic choice tasks use eye movements as a response method, typically in a task where observers are asked to saccade as quickly as possible to an image of a prespecified target category. Using this approach, face-selective saccades have been observed within 100 ms poststimulus. When taking into account oculomotor processing, this suggests that faces can be detected in as little as 70 to 80 ms. It has therefore been suggested that face detection must occur during the initial feedforward sweep, since this latency leaves little time for feedback processing. In the current experiment, we tested this hypothesis using backward masking-a technique shown to primarily disrupt feedback processing while leaving feedforward activation relatively intact. Based on minimum saccadic reaction time, we found that face detection benefited from ultra-fast, accurate saccades within 110 to 160 ms and that these eye movements are obtainable even under extreme masking conditions that limit perceptual awareness. However, masking did significantly increase the median SRT for faces. In the manual responses, we found remarkable detection accuracy for faces and houses, even when participants indicated having no visual experience of the test images. These results provide evidence for the view that the saccadic bias to faces is initiated by coarse information used to categorize faces in the feedforward sweep but that, in most cases, additional processing is required to quickly reach the threshold for saccade initiation.

Predictive coding networks for temporal prediction

One of the key problems the brain faces is inferring the state of the world from a sequence of dynamically changing stimuli, and it is not yet clear how the sensory system achieves this task. A well-established computational framework for describing perceptual processes in the brain is provided by the theory of predictive coding. Although the original proposals of predictive coding have discussed temporal prediction, later work developing this theory mostly focused on static stimuli, and key questions on neural implementation and computational properties of temporal predictive coding networks remain open. Here, we address these questions and present a formulation of the temporal predictive coding model that can be naturally implemented in recurrent networks, in which activity dynamics rely only on local inputs to the neurons, and learning only utilises local Hebbian plasticity. Additionally, we show that temporal predictive coding networks can approximate the performance of the Kalman filter in predicting behaviour of linear systems, and behave as a variant of a Kalman filter which does not track its own subjective posterior variance. Importantly, temporal predictive coding networks can achieve similar accuracy as the Kalman filter without performing complex mathematical operations, but just employing simple computations that can be implemented by biological networks. Moreover, when trained with natural dynamic inputs, we found that temporal predictive coding can produce Gabor-like, motion-sensitive receptive fields resembling those observed in real neurons in visual areas. In addition, we demonstrate how the model can be effectively generalized to nonlinear systems. Overall, models presented in this paper show how biologically plausible circuits can predict future stimuli and may guide research on understanding specific neural circuits in brain areas involved in temporal prediction.

Metacognitive evaluation of postdecisional perceptual representations

Perceptual confidence is thought to arise from metacognitive processes that evaluate the underlying perceptual decision evidence. We investigated whether metacognitive access to perceptual evidence is constrained by the hierarchical organization of visual cortex, where high-level representations tend to be more readily available for explicit scrutiny. We found that the ability of human observers to evaluate their confidence did depend on whether they performed a high-level or low-level task on the same stimuli, but was also affected by manipulations that occurred long after the perceptual decision. Confidence in low-level perceptual decisions degraded with more time between the decision and the response cue, especially when backward masking was present. Confidence in high-level tasks was immune to backward masking and benefitted from additional time. These results can be explained by a model assuming confidence heavily relies on postdecisional internal representations of visual stimuli that degrade over time, where high-level representations are more persistent.

A stimulus exposure of 50 ms elicits the uncanny valley effect

The uncanny valley (UV) effect captures the observation that artificial entities with near-human appearances tend to create feelings of eeriness. Researchers have proposed many hypotheses to explain the UV effect, but the visual processing mechanisms of the UV have yet to be fully understood. In the present study, we examined if the UV effect is as accessible in brief stimulus exposures compared to long stimulus exposures (Experiment 1). Forty-one participants, aged 21-31, rated each human-robot face presented for either a brief (50 ms) or long duration (3 s) in terms of attractiveness, eeriness, and humanness (UV indices) in a 7-point Likert scale. We found that brief and long exposures to stimuli generated a similar UV effect. This suggests that the UV effect is accessible at early visual processing. We then examined the effect of exposure duration on the categorisation of visual stimuli in Experiment 2. Thirty-three participants, aged 21-31, categorised faces as either human or robot in a two-alternative forced choice task. Their response accuracy and variance were recorded. We found that brief stimulus exposures generated significantly higher response variation and errors than the long exposure condition. This indicated that participants were more uncertain in categorising faces in the brief exposure condition due to insufficient time. Further comparisons between Experiment 1 and 2 revealed that the eeriest faces were not the hardest to categorise. Overall, these findings indicate (1) that both the UV effect and categorical uncertainty can be elicited through brief stimulus exposure, but (2) that categorical uncertainty is unlikely to cause the UV effect. These findings provide insights towards the perception of robotic faces and implications for the design of robots, androids, avatars, and artificial intelligence agents.

Visual features are processed before navigational affordances in the human brain

To navigate through their immediate environment humans process scene information rapidly. How does the cascade of neural processing elicited by scene viewing to facilitate navigational planning unfold over time? To investigate, we recorded human brain responses to visual scenes with electroencephalography and related those to computational models that operationalize three aspects of scene processing (2D, 3D, and semantic information), as well as to a behavioral model capturing navigational affordances. We found a temporal processing hierarchy: navigational affordance is processed later than the other scene features (2D, 3D, and semantic) investigated. This reveals the temporal order with which the human brain computes complex scene information and suggests that the brain leverages these pieces of information to plan navigation.

The development of visual categorization based on high-level cues

This study investigated the development of rapid visual object categorization. N = 20 adults (Experiment 1), N = 21 five to six-year-old children (Experiment 2), and N = 140 four-, seven-, and eleven-month-old infants (Experiment 3; all predominantly White, 81 females, data collected in 2013-2020) participated in a fast periodic visual stimulation electroencephalographic task. Similar categorization of animal and furniture stimuli emerged in children and adults, with responses much reduced by phase-scrambling (R2  = .34-.73). Categorization was observed from 4 months, but only at 11 months, high-level cues enhanced performance (R2  = .11). Thus, first signs of rapid categorization were evident from 4 months, but similar categorization patterns as in adults were recorded only from 11 months on.

Human Visual Cortex and Deep Convolutional Neural Network Care Deeply about Object Background

Deep convolutional neural networks (DCNNs) are able to partially predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks. We compared the activity of four DCNN architectures with EEG recordings obtained from 62 human participants during an object categorization task. Previous physiological studies on object categorization have highlighted the importance of figure-ground segregation-the ability to distinguish objects from their backgrounds. Therefore, we investigated whether figure-ground segregation could explain the predictive power of DCNNs. Using a stimulus set consisting of identical target objects embedded in different backgrounds, we examined the influence of object background versus object category within both EEG and DCNN activity. Crucially, the recombination of naturalistic objects and experimentally controlled backgrounds creates a challenging and naturalistic task, while retaining experimental control. Our results showed that early EEG activity (< 100 msec) and early DCNN layers represent object background rather than object category. We also found that the ability of DCNNs to predict EEG activity is primarily influenced by how both systems process object backgrounds, rather than object categories. We demonstrated the role of figure-ground segregation as a potential prerequisite for recognition of object features, by contrasting the activations of trained and untrained (i.e., random weights) DCNNs. These findings suggest that both human visual cortex and DCNNs prioritize the segregation of object backgrounds and target objects to perform object categorization. Altogether, our study provides new insights into the mechanisms underlying object categorization as we demonstrated that both human visual cortex and DCNNs care deeply about object background.

Evidence for Multiscale Multiplexed Representation of Visual Features in EEG

Distinct neural processes such as sensory and memory processes are often encoded over distinct timescales of neural activations. Animal studies have shown that this multiscale coding strategy is also implemented for individual components of a single process, such as individual features of a multifeature stimulus in sensory coding. However, the generalizability of this encoding strategy to the human brain has remained unclear. We asked if individual features of visual stimuli were encoded over distinct timescales. We applied a multiscale time-resolved decoding method to electroencephalography (EEG) collected from human subjects presented with grating visual stimuli to estimate the timescale of individual stimulus features. We observed that the orientation and color of the stimuli were encoded in shorter timescales, whereas spatial frequency and the contrast of the same stimuli were encoded in longer timescales. The stimulus features appeared in temporally overlapping windows along the trial supporting a multiplexed coding strategy. These results provide evidence for a multiplexed, multiscale coding strategy in the human visual system.

Mechanisms underlying category learning in the human ventral occipito-temporal cortex

The human ventral occipito-temporal cortex (VOTC) has evolved into specialized regions that process specific categories, such as words, tools, and animals. The formation of these areas is driven by bottom-up visual and top-down nonvisual experiences. However, the specific mechanisms through which top-down nonvisual experiences modulate category-specific regions in the VOTC are still unknown. To address this question, we conducted a study in which participants were trained for approximately 13 h to associate three sets of novel meaningless figures with different top-down nonvisual features: the wordlike category with word features, the non-wordlike category with nonword features, and the visual familiarity condition with no nonvisual features. Pre- and post-training functional MRI (fMRI) experiments were used to measure brain activity during stimulus presentation. Our results revealed that training induced a categorical preference for the two training categories within the VOTC. Moreover, the locations of two training category-specific regions exhibited a notable overlap. Remarkably, within the overlapping category-specific region, training resulted in a dissociation in activation intensity and pattern between the two training categories. These findings provide important insights into how different nonvisual categorical information is encoded in the human VOTC.

A Wearable Asynchronous Brain-Computer Interface Based on EEG-EOG Signals With Fewer Channels

OBJECTIVE

Brain-computer interfaces (BCIs) have tremendous application potential in communication, mechatronic control and rehabilitation. However, existing BCI systems are bulky, expensive and require laborious preparation before use. This study proposes a practical and user-friendly BCI system without compromising performance.

METHODS

A hybrid asynchronous BCI system was developed based on an elaborately designed wearable electroencephalography (EEG) amplifier that is compact, easy to use and offers a high signal-to-noise ratio (SNR). The wearable BCI system can detect P300 signals by processing EEG signals from three channels and operates asynchronously by integrating blink detection.

RESULT

The wearable EEG amplifier obtains high quality EEG signals and introduces preprocessing capabilities to BCI systems. The wearable BCI system achieves an average accuracy of 94.03±4.65%, an average information transfer rate (ITR) of 31.42±7.39 bits/min and an average false-positive rate (FPR) of 1.78%.

CONCLUSION

The experimental results demonstrate the feasibility and practicality of the developed wearable EEG amplifier and BCI system.

SIGNIFICANCE

Wearable asynchronous BCI systems with fewer channels are possible, indicating that BCI applications can be transferred from the laboratory to real-world scenarios.

Learning beyond sensations: How dreams organize neuronal representations

Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive processing theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known to generate virtual experiences, such as during imagination and dreaming, that go beyond previously experienced inputs. Here, we suggest that virtual experiences may be just as relevant as actual sensory inputs in shaping cortical representations. In particular, we discuss two complementary learning principles that organize representations through the generation of virtual experiences. First, "adversarial dreaming" proposes that creative dreams support a cortical implementation of adversarial learning in which feedback and feedforward pathways engage in a productive game of trying to fool each other. Second, "contrastive dreaming" proposes that the invariance of neuronal representations to irrelevant factors of variation is acquired by trying to map similar virtual experiences together via a contrastive learning process. These principles are compatible with known cortical structure and dynamics and the phenomenology of sleep thus providing promising directions to explain cortical learning beyond the classical predictive processing paradigm.

Modulating the difficulty of a visual oddball-like task and P3m amplitude

It is often necessary to modulate the difficulty of an experimental task without changing physical stimulus characteristics that are known to modulate event-related potentials. Here, we developed a new, oddball-like visual discrimination task with varying levels of difficulty despite using almost identical visual stimuli. Gabor patches of one orientation served as frequent standard stimuli with 75% probability. Gabor patches with a slightly different orientation served as infrequent target stimuli (25% probability). Analyzing the behavioral outcomes revealed a successful modulation of task difficulty, i.e. the hard condition revealed decreased d' values and longer reaction times for standard stimuli. In addition, we recorded MEG and computed event-related fields in response to the stimuli. In line with our expectation, the amplitude of the P3m was reduced in the hard condition. We localized the sources of the P3m with a focus on those that are modulated by changes in task difficulty. The sources of P3m modulation by difficulty were found primarily in the centro-parietal regions of both hemispheres. Additionally, we found significant differences in source activity between the easy and hard conditions in parts of the pre and post-central gyrus and inferior parietal lobe. Our findings are in line with previous research suggesting that the brain areas responsible for the conventional P3m generators also contribute to a modulation by task difficulty.

Hybrid visual and memory search for scenes and objects with variable viewpoints

In hybrid search, observers search visual arrays for any of several target types held in memory. The key finding in hybrid search is that response times (RTs) increase as a linear function of the number of items in a display (visual set size), but RTs increase linearly with the log of the memory set size. Previous experiments have shown this result for specific targets (find exactly this picture of a boot on a blank background) and for broad categorical targets (find any animal). Arguably, these are rather unnatural situations. In the real world, objects are parts of scenes and are seen from multiple viewpoints. The present experiments generalize the hybrid search findings to scenes (Experiment 1) and multiple viewpoints (Experiment 2). The results replicated the basic pattern of hybrid search results: RTs increased logarithmically with the number of scene photos/categories held in memory. Experiment 3 controls the experiment for which viewpoints were seen in an initial learning phase. The results replicate the findings of Experiment 2. Experiment 4 compares hybrid search for specific viewpoints, variable viewpoints, and categorical targets. Search difficulty increases from specific viewpoints to variable viewpoints and then to categorical targets. The results of the four experiments show the generality of logarithmic search through memory in hybrid search.

Early modulations of neural oscillations during the processing of emotional body language

The processing of threat-related emotional body language (EBL) has been shown to engage sensorimotor cortical areas early on and induce freezing in the observers' motor system, particularly when observing fearful EBL. To provide insights into the interplay between somatosensory and motor areas during observation of EBL, here, we used high-density electroencephalography (hd-EEG) in healthy humans while they observed EBL stimuli involving fearful and neutral expressions. To capture early sensorimotor brain response, we focused on P100 fronto-central event-related potentials (ERPs) and event-related desynchronization/synchronization (ERD/ERS) in the mu-alpha (8-13 Hz) and lower beta (13-20 Hz) bands over the primary motor (M1) and somatosensory (S1) cortices. Source-level ERP and ERD/ERS analyses were conducted using eLORETA. Results revealed higher P100 amplitudes in motor and premotor channels for 'Neutral' compared with 'Fear'. Additionally, analysis of ERD/ERS showed increased beta band desynchronization in M1 for 'Neutral', and the opposite pattern in S1. Source-level estimation showed significant differences between conditions mainly observed in the beta band over sensorimotor areas. These findings provide high-temporal resolution evidence suggesting that seeing fearful EBL induces early activation of somatosensory areas, which in turn could suppress M1 activity. These findings highlight early dynamics within the observer's sensorimotor system and hint at a sensorimotor mechanism supporting freezing during the processing of EBL.

Hierarchical Bayesian Causality Network to Extract High-Level Semantic Information in Visual Cortex

Functional MRI (fMRI) is a brain signal with high spatial resolution, and visual cognitive processes and semantic information in the brain can be represented and obtained through fMRI. In this paper, we design single-graphic and matched/unmatched double-graphic visual stimulus experiments and collect 12 subjects' fMRI data to explore the brain's visual perception processes. In the double-graphic stimulus experiment, we focus on the high-level semantic information as "matching", and remove tail-to-tail conjunction by designing a model to screen the matching-related voxels. Then, we perform Bayesian causal learning between fMRI voxels based on the transfer entropy, establish a hierarchical Bayesian causal network (HBcausalNet) of the visual cortex, and use the model for visual stimulus image reconstruction. HBcausalNet achieves an average accuracy of 70.57% and 53.70% in single- and double-graphic stimulus image reconstruction tasks, respectively, higher than HcorrNet and HcasaulNet. The results show that the matching-related voxel screening and causality analysis method in this paper can extract the "matching" information in fMRI, obtain a direct causal relationship between matching information and fMRI, and explore the causal inference process in the brain. It suggests that our model can effectively extract high-level semantic information in brain signals and model effective connections and visual perception processes in the visual cortex of the brain.

The Stroop effect and mental imagery

The classic Stroop task is very simple: you have to name the color of words printed on a page. If these words are color words (like "red" or "blue"), where the color named and the color it is printed in are different (say, "red" printed in blue), the reaction time increases significantly. My aim is to argue that the existing psychological explanations of the Stroop effect need to be supplemented. The Stroop effect is not exclusively about access to motor control. It is also, to a large extent, about interferences in perceptual processing. To put it briefly, reading the color word triggers-laterally and automatically-visual imagery of the color and this interferes with the processing of the perceived color of the word. In other words, the Stroop effect is to a large extent a sensory phenomenon, and it has less to do with attention, conflict monitoring, or other higher-level phenomena.

Clarifying status of DNNs as models of human vision

On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that deep neural networks (DNNs) will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response.

Movement of the stimulated finger in a Go/NoGo task enhances attention directed to that finger as evidenced by P300 amplitude modulation

Somatosensory cues and the optimal allocation of attentional resources are critical for motor performance, but it is uncertain how movement of a body part modulates directed attention and the processing of somatosensory signals originating from that same body part. The current study measured motor reaction time (RT) and the P300 event-related potential during a required movement response to stimulation of the same body part in a Go/NoGo task under multiple response. In the Movement Condition, participants were instructed to extend their right index finger in response to mild electrical stimulation of the same finger (Go signal) or remain still when receiving electrical stimulation to the fifth right finger (NoGo signal). Movement RTs and P300 amplitudes and latencies were measured under varying Go signal 50% probabilities. In other trial blocks, participants were required to count Go signals but not respond with movement or to ignore all signals while engaged in an unrelated task. Mean RT in the Movement Condition was 234.5 ms. P300 response amplitudes at midline electrodes (Fz, Cz, Pz) were the largest in the Movement Condition. The P300 amplitude at parietal electrode site Pz was significantly greater during Movement Condition trials than during Count Condition trials. The increase in P300 amplitude during trials requiring movement of the same body part receiving somatosensory stimulation suggests that movement itself modulates the attentional resources allocated to that body part.

Cortico-spinal modularity in the parieto-frontal system: A new perspective on action control

Classical neurophysiology suggests that the motor cortex (MI) has a unique role in action control. In contrast, this review presents evidence for multiple parieto-frontal spinal command modules that can bypass MI. Five observations support this modular perspective: (i) the statistics of cortical connectivity demonstrate functionally-related clusters of cortical areas, defining functional modules in the premotor, cingulate, and parietal cortices; (ii) different corticospinal pathways originate from the above areas, each with a distinct range of conduction velocities; (iii) the activation time of each module varies depending on task, and different modules can be activated simultaneously; (iv) a modular architecture with direct motor output is faster and less metabolically expensive than an architecture that relies on MI, given the slow connections between MI and other cortical areas; (v) lesions of the areas composing parieto-frontal modules have different effects from lesions of MI. Here we provide examples of six cortico-spinal modules and functions they subserve: module 1) arm reaching, tool use and object construction; module 2) spatial navigation and locomotion; module 3) grasping and observation of hand and mouth actions; module 4) action initiation, motor sequences, time encoding; module 5) conditional motor association and learning, action plan switching and action inhibition; module 6) planning defensive actions. These modules can serve as a library of tools to be recombined when faced with novel tasks, and MI might serve as a recombinatory hub. In conclusion, the availability of locally-stored information and multiple outflow paths supports the physiological plausibility of the proposed modular perspective.

Camouflage Detection and Its Association with Cognitive Style: A Functional Connectivity Study

Background: Individual differences exist in performance in tasks that require visual search, such as camouflage detection (CD). Field dependence/independence (FD/I), as assessed using the Group Embedded Figures Test (GEFT), is an extensively studied dimension of cognitive style that classifies participants based on their visual perceptual styles. Materials and Methods: In the present study, we utilized fMRI on 46 healthy participants to investigate the underlying neural mechanisms specific to the cognitive styles of FD/FI while performing a CD task using both activation magnitude and an exploratory functional connectivity (FC) analysis. Group differences between high and low performers on the two extremes of the accuracy continuum of GEFT were studied. Results: No statistically significant group differences were observed using whole-brain voxel-wise comparison. However, the exploratory FC analysis revealed an enhanced communication between various regions subserving the cognitive traits required for visual search by FI participants over and above their FD counterparts. Conclusion: These enhanced connectivities suggest additional recruitment of cognitive functions to provide computational support that might facilitate superior performance in CD task by the participants who display a field-independent cognitive style.

Multilevel and multifaceted brain response features in spiking, ERP and ERD: experimental observation and simultaneous generation in a neuronal network model with excitation-inhibition balance

Brain as a dynamic system responds to stimulations with specific patterns affected by its inherent ongoing dynamics. The patterns are manifested across different levels of organization-from spiking activity of neurons to collective oscillations in local field potential (LFP) and electroencephalogram (EEG). The multilevel and multifaceted response activities show patterns seemingly distinct and non-comparable from each other, but they should be coherently related because they are generated from the same underlying neural dynamic system. A coherent understanding of the interrelationships between different levels/aspects of activity features is important for understanding the complex brain functions. Here, based on analysis of data from human EEG, monkey LFP and neuronal spiking, we demonstrated that the brain response activities from different levels of neural system are highly coherent: the external stimulus simultaneously generated event-related potentials, event-related desynchronization, and variation in neuronal spiking activities that precisely match with each other in the temporal unfolding. Based on a biologically plausible but generic network of conductance-based integrate-and-fire excitatory and inhibitory neurons with dense connections, we showed that the multiple key features can be simultaneously produced at critical dynamical regimes supported by excitation-inhibition (E-I) balance. The elucidation of the inherent coherency of various neural response activities and demonstration of a simple dynamical neural circuit system having the ability to simultaneously produce multiple features suggest the plausibility of understanding high-level brain function and cognition from elementary and generic neuronal dynamics.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09889-w.

Impact of aversive affect on neural mechanisms of categorization decisions

INTRODUCTION

Many theories contend that evidence accumulation is a critical component of decision-making. Cognitive accumulation models typically interpret two main parameters: a drift rate and decision threshold. The former is the rate of accumulation, based on the quality of evidence, and the latter is the amount of evidence required for a decision. Some studies have found neural signals that mimic evidence accumulators and can be described by the two parameters. However, few studies have related these neural parameters to experimental manipulations of sensory data or memory representations. Here, we investigated the influence of affective salience on neural accumulation parameters. High affective salience has been repeatedly shown to influence decision-making, yet its effect on neural evidence accumulation has been unexamined.

METHODS

The current study used a two-choice object categorization task of body images (feet or hands). Half the images in each category were high in affective salience because they contained highly aversive features (gore and mutilation). To study such quick categorization decisions with a relatively slow technique like functional magnetic resonance imaging, we used a gradual reveal paradigm to lengthen cognitive processing time through the gradual "unmasking" of stimuli.

RESULTS

Because the aversive features were task-irrelevant, high affective salience produced a distractor effect, slowing decision time. In visual accumulation regions of interest, high affective salience produced a longer time to peak activation. Unexpectedly, the later peak appeared to be the product of changes to both drift rate and decision threshold. The drift rate for high affective salience was shallower, and the decision threshold was greater. To our knowledge, this is the first demonstration of an experimental manipulation of sensory data or memory representations that changed the neural decision threshold.

CONCLUSION

These findings advance our knowledge of the neural mechanisms underlying affective responses in general and the influence of high affective salience on object representations and categorization decisions.

Statistical characteristics of comic panel viewing times

Comics are a bimodal form of art involving a mixture of text and images. Since comics require a combination of various cognitive processes to comprehend their contents, the analysis of human comic reading behavior sheds light on how humans process such bimodal forms of media. In this paper, we particularly focus on the viewing times of each comic panel as a quantitative measure of attention, and analyze the statistical characteristics of the distributions of comic panel viewing times. We create a user interface that presents comics in a panel-wise manner, and measure the viewing times of each panel through a user study experiment. We collected data from 18 participants reading 7 comic book volumes resulting in over 99,000 viewing time data points, which will be released publicly. The results show that the average viewing times are proportional to the text length contained in the panel's speech bubbles, with a rate of proportion differing for each reader, despite the bimodal setting. Additionally, we find that the viewing time for all users follows a common heavy-tailed distribution.

An exact mapping from ReLU networks to spiking neural networks

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an exact mapping from a network with Rectified Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our constructive proof, we assume that an arbitrary multi-layer ReLU network with or without convolutional layers, batch normalization and max pooling layers was trained to high performance on some training set. Furthermore, we assume that we have access to a representative example of input data used during training and to the exact parameters (weights and biases) of the trained ReLU network. The mapping from deep ReLU networks to SNNs causes zero percent drop in accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and PASS. More generally our work shows that an arbitrary deep ReLU network can be replaced by an energy-efficient single-spike neural network without any loss of performance.

Spatiotemporal cortical dynamics for visual scene processing as revealed by EEG decoding

The human visual system rapidly recognizes the categories and global properties of complex natural scenes. The present study investigated the spatiotemporal dynamics of neural signals involved in visual scene processing using electroencephalography (EEG) decoding. We recorded visual evoked potentials from 11 human observers for 232 natural scenes, each of which belonged to one of 13 natural scene categories (e.g., a bedroom or open country) and had three global properties (naturalness, openness, and roughness). We trained a deep convolutional classification model of the natural scene categories and global properties using EEGNet. Having confirmed that the model successfully classified natural scene categories and the three global properties, we applied Grad-CAM to the EEGNet model to visualize the EEG channels and time points that contributed to the classification. The analysis showed that EEG signals in the occipital electrodes at short latencies (approximately 80 ~ ms) contributed to the classifications, whereas those in the frontal electrodes at relatively long latencies (200 ~ ms) contributed to the classification of naturalness and the individual scene category. These results suggest that different global properties are encoded in different cortical areas and with different timings, and that the combination of the EEGNet model and Grad-CAM can be a tool to investigate both temporal and spatial distribution of natural scene processing in the human brain.

Hierarchical temporal prediction captures motion processing along the visual pathway

Visual neurons respond selectively to features that become increasingly complex from the eyes to the cortex. Retinal neurons prefer flashing spots of light, primary visual cortical (V1) neurons prefer moving bars, and those in higher cortical areas favor complex features like moving textures. Previously, we showed that V1 simple cell tuning can be accounted for by a basic model implementing temporal prediction - representing features that predict future sensory input from past input (Singer et al., 2018). Here, we show that hierarchical application of temporal prediction can capture how tuning properties change across at least two levels of the visual system. This suggests that the brain does not efficiently represent all incoming information; instead, it selectively represents sensory inputs that help in predicting the future. When applied hierarchically, temporal prediction extracts time-varying features that depend on increasingly high-level statistics of the sensory input.

Clarifying the nature of stochastic fluctuations and accumulation processes in spontaneous movements

Experiments on choice-predictive brain signals have played an important role in the debate on free will. In a seminal study, Benjamin Libet and colleagues found that a negative-going EEG signal, the readiness potential (RP), can be observed over motor-related brain regions even hundreds of ms before the time of the conscious decision to move. If the early onset of the readiness potential is taken as an indicator of the "brain's decision to move" this could mean that this decision is made early, by unconscious brain activity, rather than later, at the time when the subject believes to have decided. However, an alternative kind of interpretation, involving ongoing stochastic fluctuations, has recently been brought to light. This stochastic decision model (SDM) takes its inspiration from leaky accumulator models of perceptual decision making. It suggests that the RP originates from an accumulation of ongoing stochastic fluctuations. In this view, the decision happens only at a much later stage when an accumulated noisy signal (plus imperative) reaches a threshold. Here, we clarify a number of confusions regarding both the evidence for the stochastic decision model as well as the interpretation that it offers. We will explore several points that we feel are in need of clarification: (a) the empirical evidence for the role of stochastic fluctuations is so far only indirect; (b) the interpretation of animal studies is unclear; (c) a model that is deterministic during the accumulation stage can explain the data in a similar way; (d) the primary focus in the literature has been on the role of random fluctuations whereas the deterministic aspects of the model have been largely ignored; (e) contrary to the original interpretation, the deterministic component of the model is quantitatively the dominant input into the accumulator; and finally (f) there is confusion regarding the role of "imperative" in the SDM and its link to "evidence" in perceptual decision making. Our aim is not to rehabilitate the role of the RP in the free will debate. Rather we aim to address some confusions regarding the evidence for accumulators playing a role in these preparatory brain processes.

Depressive states in healthy subjects lead to biased processing in frontal-parietal ERPs during emotional stimuli

Subthreshold depressive (sD) states and major depression are considered to occur on a continuum, and there are only quantitative and not qualitative differences between depressive states in healthy individuals and patients with depression. sD is showing a progressively increasing prevalence and has a lifelong impact, and the social and clinical impacts of sD are no less than those of major depressive disorder (MDD). Because depression leads to biased cognition, patients with depression and healthy individuals show different visual processing properties. However, it remains unclear whether there are significant differences in visual information recognition among healthy individuals with various depressive states. In this study, we investigated the event-related potentials (ERPs) and event-related spectrum perturbation (ERSP) of healthy individuals with various depressive states during the perception of emotional visual stimulation. We show that different neural activities can be detected even among healthy individuals. We divided healthy participants into high, middle, and low depressive state groups and found that participants in a high depressive state had a lower P300 amplitude and significant differences in fast and slow neural responses in the frontal and parietal lobes. We anticipate our study to provide useful parameters for assessing the evaluation of depressive states in healthy individuals.

Human-as-a-robot performance in mixed reality teleultrasound

PURPOSE

In "human teleoperation" (HT), mixed reality (MR) and haptics are used to tightly couple an expert leader to a human follower [1]. To determine the feasibility of HT for teleultrasound, we quantify the ability of humans to track a position and/or force trajectory via MR cues. The human response time, precision, frequency response, and step response were characterized, and several rendering methods were compared.

METHODS

Volunteers (n=11) performed a series of tasks as the follower in our HT system. The tasks involved tracking pre-recorded series of motions and forces while pose and force were recorded. The volunteers then performed frequency response tests and filled out a questionnaire.

RESULTS

Following force and pose simultaneously was more difficult but did not lead to significant performance degradation versus following one at a time. On average, subjects tracked positions, orientations, and forces with RMS tracking errors of [Formula: see text] mm, [Formula: see text], [Formula: see text] N, steady-state errors of [Formula: see text] mm, [Formula: see text] N, and lags of [Formula: see text] ms, respectively. Performance decreased with input frequency, depending on the input amplitude.

CONCLUSION

Teleoperating a person through MR is a novel concept with many possible applications. However, it is unknown what performance is achievable and which approaches work best. This paper thus characterizes human tracking ability in MR HT for teleultrasound, which is important for designing future tightly coupled guidance and training systems using MR.

Suppressive Control of Incentive Salience in Real-World Human Vision

Reward-related activity in the dopaminergic midbrain is thought to guide animal behavior, in part by boosting the perceptual and attentional processing of reward-predictive environmental stimuli. In line with this incentive salience hypothesis, studies of human visual search have shown that simple synthetic stimuli, such as lines, shapes, or Gabor patches, capture attention to their location when they are characterized by reward-associated visual features, such as color. In the real world, however, we commonly search for members of a category of visually heterogeneous objects, such as people, cars, or trees, where category examples do not share low-level features. Is attention captured to examples of a reward-associated real-world object category? Here, we have human participants search for targets in photographs of city and landscapes that contain task-irrelevant examples of a reward-associated category. We use the temporal precision of EEG machine learning and ERPs to show that these distractors acquire incentive salience and draw attention, but do not capture it. Instead, we find evidence of rapid, stimulus-triggered attentional suppression, such that the neural encoding of these objects is degraded relative to neutral objects. Humans appear able to suppress the incentive salience of reward-associated objects when they know these objects will be irrelevant, supporting the rapid deployment of attention to other objects that might be more useful. Incentive salience is thought to underlie key behaviors in eating disorders and addiction, among other conditions, and the kind of suppression identified here likely plays a role in mediating the attentional biases that emerge in these circumstances.Significance Statement Like other animals, humans are prone to notice and interact with environmental objects that have proven rewarding in earlier experience. However, it is common that such objects have no immediate strategic use and are therefore distracting. Do these reward-associated real-world objects capture our attention, despite our strategic efforts otherwise? Or are we able to strategically control the impulse to notice them? Here we use machine learning classification of human electrical brain activity to show that we can establish strategic control over the salience of naturalistic reward-associated objects. These objects draw our attention, but do not necessarily capture it, and this kind of control may play an important role in mediating conditions like eating disorder and addiction.

Exploring Trade-Offs in Spiking Neural Networks

Spiking neural networks (SNNs) have emerged as a promising alternative to traditional deep neural networks for low-power computing. However, the effectiveness of SNNs is not solely determined by their performance but also by their energy consumption, prediction speed, and robustness to noise. The recent method Fast & Deep, along with others, achieves fast and energy-efficient computation by constraining neurons to fire at most once. Known as time-to-first-spike (TTFS), this constraint, however, restricts the capabilities of SNNs in many aspects. In this work, we explore the relationships of performance, energy consumption, speed, and stability when using this constraint. More precisely, we highlight the existence of trade-offs where performance and robustness are gained at the cost of sparsity and prediction latency. To improve these trade-offs, we propose a relaxed version of Fast & Deep that allows for multiple spikes per neuron. Our experiments show that relaxing the spike constraint provides higher performance while also benefiting from faster convergence, similar sparsity, comparable prediction latency, and better robustness to noise compared to TTFS SNNs. By highlighting the limitations of TTFS and demonstrating the advantages of unconstrained SNNs, we provide valuable insight for the development of effective learning strategies for neuromorphic computing.

Effects of scaling direction on adults' spatial scaling in different perceptual domains

The current study investigated adults' strategies of spatial scaling from memory in three perceptual conditions (visual, haptic, and visuo-haptic) when scaling up and down. Following previous research, we predicted the usage of mental transformation strategies. In all conditions, participants (N = 90, aged 19-28 years) were presented with tactile, colored graphics which allowed to visually and haptically explore spatial information. Participants were first asked to encode a map including a target. Then, they were instructed to place a response object at the same place on an empty, constant-sized referent space. Maps had five different sizes resulting in five scaling factors (3:1, 2:1, 1:1, 1:2, 1:3). This manipulation also allowed assessing potentially symmetric effects of scaling direction on adults' responses. Response times and absolute errors served as dependent variables. In line with our hypotheses, the changes in these dependent variables were best explained by a quadratic function which suggests the usage of mental transformation strategies for spatial scaling. There were no differences between perceptual conditions concerning the influence of scaling factor on dependent variables. Results revealed symmetric effects of scaling direction on participants' accuracy whereas there were small differences for response times. Our findings highlight the usage of mental transformation strategies in adults' spatial scaling, irrespective of perceptual modality and scaling direction.

Civil inattention-On the sources of relational segregation

The article employs ethnomethodological conversation analysis (CA) and experimental video analysis to scrutinize the gaze behavior of urban passersby. We operationalize Goffman's concept of civil inattention to make it an empirical research object with defined boundaries. Video analysis enabled measurement of gaze lengths to establish measures for "normal" gazes within civil inattention and to account for their breaches. We also studied the dependence of gazing behavior on the recipient's social appearance by comparing the unmarked condition, the experimenter wearing casual, indistinctive clothes, to marked conditions, the experimenter wearing either a distinct sunhat or an abaya and niqab. The breaches of civil inattention toward marked gaze recipients were 10-fold compared to unmarked recipients. Furthermore, the analysis points out the commonality of hitherto unknown micro gazes and multiple gazes. Together the findings suggest the existence of subconscious monitoring beneath the public social order, which pre-structures interaction order, and indicates that stigmatization is a source for relational segregation.

Graphical Abstract in Scientific Research

A graphical abstract (GA) summarizes the key and important findings of an article graphically, potentially stimulating researchers to view the published manuscript. A GA should enhance dissemination, augment engagement, and impact clinical practice. Infographics play a key role in a quicker understanding of the significant findings of a manuscript. Few level 1 studies reported that GAs enhanced the engagement of readers on social media when compared to plain text abstracts. With the evolution of Industry 4.0, 5.0, and 6.0, GA plays a major role in understanding the technical aspects of various technologies. This article outlines tips to prepare an effective GA and reports the impact of GAs on research and clinical translation.

Saccadic trajectories deviate toward or away from optimally informative visual features

The saccades' path is influenced by visual distractors, making their trajectory curve away or toward them. Previous research suggested that the more salient the distractor, the more pronounced is the curvature. We investigate the saliency of spatial visual features, predicted by a constrained maximum entropy model to be optimal or non-optimal information carriers in fast vision, by using them as distractors in a saccadic task. Their effect was compared to that of luminance-based control distractors. Optimal features evoke a larger saccadic curvature compared to non-optimal features, and the magnitude and direction of deviation change as a function of the delay between distractor and saccade onset. Effects were similar to those found with high-luminance versus low-luminance distractors. Therefore, model-predicted information optimal features interfere with target-oriented saccades, following a dynamic attraction-repulsion pattern. This suggests that the visuo-oculomotor system rapidly and automatically processes optimally informative features while programming visually guided eye movements.

BrainCog: A spiking neural network based, brain-inspired cognitive intelligence engine for brain-inspired AI and brain simulation

Spiking neural networks (SNNs) serve as a promising computational framework for integrating insights from the brain into artificial intelligence (AI). Existing software infrastructures based on SNNs exclusively support brain simulation or brain-inspired AI, but not both simultaneously. To decode the nature of biological intelligence and create AI, we present the brain-inspired cognitive intelligence engine (BrainCog). This SNN-based platform provides essential infrastructure support for developing brain-inspired AI and brain simulation. BrainCog integrates different biological neurons, encoding strategies, learning rules, brain areas, and hardware-software co-design as essential components. Leveraging these user-friendly components, BrainCog incorporates various cognitive functions, including perception and learning, decision-making, knowledge representation and reasoning, motor control, social cognition, and brain structure and function simulations across multiple scales. BORN is an AI engine developed by BrainCog, showcasing seamless integration of BrainCog's components and cognitive functions to build advanced AI models and applications.

Objects sharpen visual scene representations: evidence from MEG decoding

Real-world scenes consist of objects, defined by local information, and scene background, defined by global information. Although objects and scenes are processed in separate pathways in visual cortex, their processing interacts. Specifically, previous studies have shown that scene context makes blurry objects look sharper, an effect that can be observed as a sharpening of object representations in visual cortex from around 300 ms after stimulus onset. Here, we use MEG to show that objects can also sharpen scene representations, with the same temporal profile. Photographs of indoor (closed) and outdoor (open) scenes were blurred such that they were difficult to categorize on their own but easily disambiguated by the inclusion of an object. Classifiers were trained to distinguish MEG response patterns to intact indoor and outdoor scenes, presented in an independent run, and tested on degraded scenes in the main experiment. Results revealed better decoding of scenes with objects than scenes alone and objects alone from 300 ms after stimulus onset. This effect was strongest over left posterior sensors. These findings show that the influence of objects on scene representations occurs at similar latencies as the influence of scenes on object representations, in line with a common predictive processing mechanism.

Hybrid predictive coding: Inferring, fast and slow

Predictive coding is an influential model of cortical neural activity. It proposes that perceptual beliefs are furnished by sequentially minimising "prediction errors"-the differences between predicted and observed data. Implicit in this proposal is the idea that successful perception requires multiple cycles of neural activity. This is at odds with evidence that several aspects of visual perception-including complex forms of object recognition-arise from an initial "feedforward sweep" that occurs on fast timescales which preclude substantial recurrent activity. Here, we propose that the feedforward sweep can be understood as performing amortized inference (applying a learned function that maps directly from data to beliefs) and recurrent processing can be understood as performing iterative inference (sequentially updating neural activity in order to improve the accuracy of beliefs). We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner by describing both in terms of a dual optimization of a single objective function. We show that the resulting scheme can be implemented in a biologically plausible neural architecture that approximates Bayesian inference utilising local Hebbian update rules. We demonstrate that our hybrid predictive coding model combines the benefits of both amortized and iterative inference-obtaining rapid and computationally cheap perceptual inference for familiar data while maintaining the context-sensitivity, precision, and sample efficiency of iterative inference schemes. Moreover, we show how our model is inherently sensitive to its uncertainty and adaptively balances iterative and amortized inference to obtain accurate beliefs using minimum computational expense. Hybrid predictive coding offers a new perspective on the functional relevance of the feedforward and recurrent activity observed during visual perception and offers novel insights into distinct aspects of visual phenomenology.

Waves traveling over a map of visual space can ignite short-term predictions of sensory input

Recent analyses have found waves of neural activity traveling across entire visual cortical areas in awake animals. These traveling waves modulate the excitability of local networks and perceptual sensitivity. The general computational role of these spatiotemporal patterns in the visual system, however, remains unclear. Here, we hypothesize that traveling waves endow the visual system with the capacity to predict complex and naturalistic inputs. We present a network model whose connections can be rapidly and efficiently trained to predict individual natural movies. After training, a few input frames from a movie trigger complex wave patterns that drive accurate predictions many frames into the future solely from the network's connections. When the recurrent connections that drive waves are randomly shuffled, both traveling waves and the ability to predict are eliminated. These results suggest traveling waves may play an essential computational role in the visual system by embedding continuous spatiotemporal structures over spatial maps.