Temporal Encoding and Multispike Learning Framework for Efficient Recognition of Visual Patterns

Low-power and lightweight spiking transformer for EEG-based auditory attention detection

EEG signal analysis can be used to study brain activity and the function and structure of neural networks, helping to understand neural mechanisms such as cognition, emotion, and behavior. EEG-based auditory attention detection is using EEG signals to determine an individual's level of attention to specific auditory stimuli. In this technique, researchers record and analyze a subject's electrical activity to infer whether an individual is paying attention to a specific auditory stimulus. The model deployed in edge devices will be greatly convenient for subjects to use. However, most of the existing EEG-based auditory attention detection models use traditional neural network models, and their high computing load makes deployment on edge devices challenging. We present a pioneering approach in the form of a binarized spiking Transformer for EEG-based auditory attention detection, which is characterized by high accuracy, low power consumption, and lightweight design, making it highly suitable for deployment on edge devices. In terms of low power consumption, the network is constructed using spiking neurons, which emit sparse and binary spike sequences, which can effectively reduce computing power consumption. In terms of lightweight, we use a post-training quantization strategy to quantize the full-precision network weights into binary weights, which greatly reduces the model size. In addition, the structure of the Transformer ensures that the model can learn effective information and ensure its high performance. We verify the model through mainstream datasets, and experimental results show that our model performance can exceed the existing state-of-the-art models, and the model size can be reduced by more than 21 times compared with the original full-precision network counterpart.

Spike-Based Motion Estimation for Object Tracking Through Bio-Inspired Unsupervised Learning

Neuromorphic vision sensors, whose pixels output events/spikes asynchronously with a high temporal resolution according to the scene radiance change, are naturally appropriate for capturing high-speed motion in the scenes. However, how to utilize the events/spikes to smoothly track high-speed moving objects is still a challenging problem. Existing approaches either employ time-consuming iterative optimization, or require large amounts of labeled data to train the object detector. To this end, we propose a bio-inspired unsupervised learning framework, which takes advantage of the spatiotemporal information of events/spikes generated by neuromorphic vision sensors to capture the intrinsic motion patterns. Without off-line training, our models can filter the redundant signals with dynamic adaption module based on short-term plasticity, and extract the motion patterns with motion estimation module based on the spike-timing-dependent plasticity. Combined with the spatiotemporal and motion information of the filtered spike stream, the traditional DBSCAN clustering algorithm and Kalman filter can effectively track multiple targets in extreme scenes. We evaluate the proposed unsupervised framework for object detection and tracking tasks on synthetic data, publicly available event-based datasets, and spiking camera datasets. The experiment results show that the proposed model can robustly detect and smoothly track the moving targets on various challenging scenarios and outperforms state-of-the-art approaches.

Astrocytes mediate analogous memory in a multi-layer neuron–astrocyte network

Modeling the neuronal processes underlying short-term working memory remains the focus of many theoretical studies in neuroscience. In this paper, we propose a mathematical model of a spiking neural network (SNN) which simulates the way a fragment of information is maintained as a robust activity pattern for several seconds and the way it completely disappears if no other stimuli are fed to the system. Such short-term memory traces are preserved due to the activation of astrocytes accompanying the SNN. The astrocytes exhibit calcium transients at a time scale of seconds. These transients further modulate the efficiency of synaptic transmission and, hence, the firing rate of neighboring neurons at diverse timescales through gliotransmitter release. We demonstrate how such transients continuously encode frequencies of neuronal discharges and provide robust short-term storage of analogous information. This kind of short-term memory can store relevant information for seconds and then completely forget it to avoid overlapping with forthcoming patterns. The SNN is inter-connected with the astrocytic layer by local inter-cellular diffusive connections. The astrocytes are activated only when the neighboring neurons fire synchronously, e.g., when an information pattern is loaded. For illustration, we took grayscale photographs of people’s faces where the shades of gray correspond to the level of applied current which stimulates the neurons. The astrocyte feedback modulates (facilitates) synaptic transmission by varying the frequency of neuronal firing. We show how arbitrary patterns can be loaded, then stored for a certain interval of time, and retrieved if the appropriate clue pattern is applied to the input.