Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks

BISNN: bio-information-fused spiking neural networks for enhanced EEG-based emotion recognition

Spiking neural networks (SNNs), known for their rich spatio-temporal dynamics, have recently gained considerable attention in EEG-based emotion recognition. However, conventional model training approaches often fail to fully exploit the capabilities of SNNs, posing challenges for effective EEG data analysis. In this work, we propose a novel bio-information-fused SNN (BISNN) model to enhance EEG-based emotion recognition. The BISNN model incorporates biologically plausible intrinsic parameters into spiking neurons and is initialized with a structurally equivalent pre-trained ANN model. By constructing a bio-information-fused loss function, the BISNN model enables simultaneous training under dual constraints. Extensive experiments on benchmark EEG-based emotion datasets demonstrate that the BISNN model achieves competitive performance compared to state-of-the-art methods. Additionally, ablation studies investigating various components further elucidate the mechanisms underlying the model's effectiveness and evolution, aligning well with previous findings.

Ternary spike-based neuromorphic signal processing system

Deep Neural Networks (DNNs) have been successfully implemented across various signal processing fields, resulting in significant enhancements in performance. However, DNNs generally require substantial computational resources, leading to significant economic costs and posing challenges for their deployment on resource-constrained edge devices. In this study, we take advantage of spiking neural networks (SNNs) and quantization technologies to develop an energy-efficient and lightweight neuromorphic signal processing system. Our system is characterized by two principal innovations: a threshold-adaptive encoding (TAE) method and a quantized ternary SNN (QT-SNN). The TAE method can efficiently encode time-varying analog signals into sparse ternary spike trains, thereby reducing energy and memory demands for signal processing. QT-SNN, compatible with ternary spike trains from the TAE method, quantifies both membrane potentials and synaptic weights to reduce memory requirements while maintaining performance. Extensive experiments are conducted on two typical signal-processing tasks: speech and electroencephalogram recognition. The results demonstrate that our neuromorphic signal processing system achieves state-of-the-art (SOTA) performance with a 94% reduced memory requirement. Furthermore, through theoretical energy consumption analysis, our system shows 7.5× energy saving compared to other SNN works. The efficiency and efficacy of the proposed system highlight its potential as a promising avenue for energy-efficient signal processing.

A Reinforced, Event-Driven, and Attention-Based Convolution Spiking Neural Network for Multivariate Time Series Prediction

Despite spiking neural networks (SNNs) inherently exceling at processing time series due to their rich spatio-temporal information and efficient event-driven computing, the challenge of extracting complex correlations between variables in multivariate time series (MTS) remains to be addressed. This paper proposes a reinforced, event-driven, and attention-based convolution SNN model (REAT-CSNN) with three novel features. First, a joint Gramian Angular Field and Rate (GAFR) coding scheme is proposed to convert MTS into spike images, preserving the inherent features in MTS, such as the temporal patterns and spatio-temporal correlations between time series. Second, an advanced LIF-pooling strategy is developed, which is then theoretically and empirically proved to be effective in preserving more features from the regions of interest in spike images than average-pooling strategies. Third, a convolutional block attention mechanism (CBAM) is redesigned to support spike-based input, enhancing event-driven characteristics in weighting operations while maintaining outstanding capability to capture the information encoded in spike images. Experiments on multiple MTS data sets, such as stocks and PM2.5 data sets, demonstrate that our model rivals, and even surpasses, some CNN- and RNN-based techniques, with up to 3% better performance, while consuming significantly less energy.

A novel approach to enhancing biomedical signal recognition via hybrid high-order information bottleneck driven spiking neural networks

Biomedical signals, encapsulating vital physiological information, are pivotal in elucidating human traits and conditions, serving as a cornerstone for advancing human-machine interfaces. Nonetheless, the fidelity of biomedical signal interpretation is frequently compromised by pervasive noise sources such as skin, motion, and equipment interference, posing formidable challenges to precision recognition tasks. Concurrently, the burgeoning adoption of intelligent wearable devices illuminates a societal shift towards enhancing life and work through technological integration. This surge in popularity underscores the imperative for efficient, noise-resilient biomedical signal recognition methodologies, a quest that is both challenging and profoundly impactful. This study proposes a novel approach to enhancing biomedical signal recognition. The proposed approach employs a hierarchical information bottleneck mechanism within SNNs, quantifying the mutual information in different orders based on the depth of information flow in the network. Subsequently, these mutual information, together with the network's output and category labels, are restructured based on information theory principles to form the loss function used for training. A series of theoretical analyses and substantial experimental results have shown that this method can effectively compress noise in the data, and on the premise of low computational cost, it can also significantly outperform its vanilla counterpart in terms of classification performance.

Fault-Tolerant Attitude Tracking Control Driven by Spiking NNs for Unmanned Aerial Vehicles

In this article, we proposed a novel fault-tolerant control scheme for quadrotor unmanned aerial vehicles (UAVs) based on spiking neural networks (SNNs), which leverages the inherent features of neural network computing to significantly enhance the reliability and robustness of UAV flight control. Traditional control methods are known to be inadequate in dealing with complex and real-time sensor data, which results in poor performance and reduced robustness in fault-tolerant control. In contrast, the temporal processing, parallelism, and nonlinear capacity of SNNs enable the fault-tolerant control scheme to process vast amounts of sensory data with the ability to accurately identify and respond to faults. Furthermore, SNNs can learn and adjust to new environments and fault conditions, providing effective and adaptive flight control. The proposed SNN-based fault-tolerant control scheme demonstrates significant improvements in control accuracy and robustness compared with conventional methods, indicating its potential applicability and suitability for a range of UAV flight control scenarios.

Memory-Dependent Computation and Learning in Spiking Neural Networks Through Hebbian Plasticity

Spiking neural networks (SNNs) are the basis for many energy-efficient neuromorphic hardware systems. While there has been substantial progress in SNN research, artificial SNNs still lack many capabilities of their biological counterparts. In biological neural systems, memory is a key component that enables the retention of information over a huge range of temporal scales, ranging from hundreds of milliseconds up to years. While Hebbian plasticity is believed to play a pivotal role in biological memory, it has so far been analyzed mostly in the context of pattern completion and unsupervised learning in artificial and SNNs. Here, we propose that Hebbian plasticity is fundamental for computations in biological and artificial spiking neural systems. We introduce a novel memory-augmented SNN architecture that is enriched by Hebbian synaptic plasticity. We show that Hebbian enrichment renders SNNs surprisingly versatile in terms of their computational as well as learning capabilities. It improves their abilities for out-of-distribution generalization, one-shot learning, cross-modal generative association, language processing, and reward-based learning. This suggests that powerful cognitive neuromorphic systems can be built based on this principle.

Memristor-Based Neural Network Circuit of Associative Memory With Overshadowing and Emotion Congruent Effect

Most memristor-based neural network circuits consider only a single pattern of overshadowing or emotion, but the relationship between overshadowing and emotion is ignored. In this article, a memristor-based neural network circuit of associative memory with overshadowing and emotion congruent effect is designed, and overshadowing under multiple emotions is taken into account. The designed circuit mainly consists of an emotion module, a memory module, an inhibition module, and a feedback module. The generation and recovery of different emotions are realized by the emotion module. The functions of overshadowing under different emotions and recovery from overshadowing are achieved by the inhibition module and the memory module. Finally, the blocking caused by long-term overshadowing is implemented by the feedback module. The proposed circuit can be applied to bionic emotional robots and offers some references for brain-like systems.

Spiking Neural Networks with Adaptive Membrane Time Constant for Event-Based Tracking

The brain-inspired Spiking Neural Networks (SNNs) work in an event-driven manner and have an implicit recurrence in neuronal membrane potential to memorize information over time, which are inherently suitable to handle temporal event-based streams. Despite their temporal nature and recent approaches advancements, these methods have predominantly been assessed on event-based classification tasks. In this paper, we explore the utility of SNNs for event-based tracking tasks. Specifically, we propose a brain-inspired adaptive Leaky Integrate-and-Fire neuron (BA-LIF) that can adaptively adjust the membrane time constant according to the inputs, thereby accelerating the leakage of meaningless noise features and reducing the decay of valuable information. SNNs composed of our proposed BA-LIF neurons can achieve high performance without a careful and time-consuming trial-by-error initialization on the membrane time constant. The adaptive capability of our network is further improved by introducing an extra temporal feature aggregator (TFA) that assigns attention weights over the temporal dimension. Extensive experiments on various event-based tracking datasets validate the effectiveness of our proposed method. We further validate the generalization capability of our method by applying it to other event-classification tasks.

Event-Driven Spiking Learning Algorithm Using Aggregated Labels

Traditional spiking learning algorithm aims to train neurons to spike at a specific time or on a particular frequency, which requires precise time and frequency labels in the training process. While in reality, usually only aggregated labels of sequential patterns are provided. The aggregate-label (AL) learning is proposed to discover these predictive features in distracting background streams only by aggregated spikes. It has achieved much success recently, but it is still computationally intensive and has limited use in deep networks. To address these issues, we propose an event-driven spiking aggregate learning algorithm (SALA) in this article. Specifically, to reduce the computational complexity, we improve the conventional spike-threshold-surface (STS) calculation in AL learning by analytical calculating voltage peak values in spiking neurons. Then we derive the algorithm to multilayers by event-driven strategy using aggregated spikes. We conduct comprehensive experiments on various tasks including temporal clue recognition, segmented and continuous speech recognition, and neuromorphic image classification. The experimental results demonstrate that the new STS method improves the efficiency of AL learning significantly, and the proposed algorithm outperforms the conventional spiking algorithm in various temporal clue recognition tasks.

Enhancing SNN-based spatio-temporal learning: A benchmark dataset and Cross-Modality Attention model

Spiking Neural Networks (SNNs), renowned for their low power consumption, brain-inspired architecture, and spatio-temporal representation capabilities, have garnered considerable attention in recent years. Similar to Artificial Neural Networks (ANNs), high-quality benchmark datasets are of great importance to the advances of SNNs. However, our analysis indicates that many prevalent neuromorphic datasets lack strong temporal correlation, preventing SNNs from fully exploiting their spatio-temporal representation capabilities. Meanwhile, the integration of event and frame modalities offers more comprehensive visual spatio-temporal information. Yet, the SNN-based cross-modality fusion remains underexplored. In this work, we present a neuromorphic dataset called DVS-SLR that can better exploit the inherent spatio-temporal properties of SNNs. Compared to existing datasets, it offers advantages in terms of higher temporal correlation, larger scale, and more varied scenarios. In addition, our neuromorphic dataset contains corresponding frame data, which can be used for developing SNN-based fusion methods. By virtue of the dual-modal feature of the dataset, we propose a Cross-Modality Attention (CMA) based fusion method. The CMA model efficiently utilizes the unique advantages of each modality, allowing for SNNs to learn both temporal and spatial attention scores from the spatio-temporal features of event and frame modalities, subsequently allocating these scores across modalities to enhance their synergy. Experimental results demonstrate that our method not only improves recognition accuracy but also ensures robustness across diverse scenarios.

Low Latency and Sparse Computing Spiking Neural Networks With Self-Driven Adaptive Threshold Plasticity

Spiking neural networks (SNNs) have captivated the attention worldwide owing to their compelling advantages in low power consumption, high biological plausibility, and strong robustness. However, the intrinsic latency associated with SNNs during inference poses a significant challenge, impeding their further development and application. This latency is caused by the need for spiking neurons to collect electrical stimuli and generate spikes only when their membrane potential exceeds a firing threshold. Considering the firing threshold plays a crucial role in SNN performance, this article proposes a self-driven adaptive threshold plasticity (SATP) mechanism, wherein neurons autonomously adjust the firing thresholds based on their individual state information using unsupervised learning rules, of which the adjustment is triggered by their own firing events. SATP is based on the principle of maximizing the information contained in the output spike rate distribution of each neuron. This article derives the mathematical expression of SATP and provides extensive experimental results, demonstrating that SATP effectively reduces SNN inference latency, further reduces the computation density while improving computational accuracy, so that SATP facilitates SNN models to be with low latency, sparse computing, and high accuracy.

Synchronized stepwise control of firing and learning thresholds in a spiking randomly connected neural network toward hardware implementation

Spiking randomly connected neural network (RNN) hardware is promising as ultimately low power devices for temporal data processing at the edge. Although the potential of RNNs for temporal data processing has been demonstrated, randomness of the network architecture often causes performance degradation. To mitigate such degradation, self-organization mechanism using intrinsic plasticity (IP) and synaptic plasticity (SP) should be implemented in the spiking RNN. Therefore, we propose hardware-oriented models of these functions. To implement the function of IP, a variable firing threshold is introduced to each excitatory neuron in the RNN that changes stepwise in accordance with its activity. We also define other thresholds for SP that synchronize with the firing threshold, which determine the direction of stepwise synaptic update that is executed on receiving a pre-synaptic spike. To discuss the effectiveness of our model, we perform simulations of temporal data learning and anomaly detection using publicly available electrocardiograms (ECGs) with a spiking RNN. We observe that the spiking RNN with our IP and SP models realizes the true positive rate of 1 with the false positive rate being suppressed at 0 successfully, which does not occur otherwise. Furthermore, we find that these thresholds as well as the synaptic weights can be reduced to binary if the RNN architecture is appropriately designed. This contributes to minimization of the circuit of the neuronal system having IP and SP.

A Spatial-Channel-Temporal-Fused Attention for Spiking Neural Networks

Spiking neural networks (SNNs) mimic brain computational strategies, and exhibit substantial capabilities in spatiotemporal information processing. As an essential factor for human perception, visual attention refers to the dynamic process for selecting salient regions in biological vision systems. Although visual attention mechanisms have achieved great success in computer vision applications, they are rarely introduced into SNNs. Inspired by experimental observations on predictive attentional remapping, we propose a new spatial-channel-temporal-fused attention (SCTFA) module that can guide SNNs to efficiently capture underlying target regions by utilizing accumulated historical spatial-channel information in the present study. Through a systematic evaluation on three event stream datasets (DVS Gesture, SL-Animals-DVS, and MNIST-DVS), we demonstrate that the SNN with the SCTFA module (SCTFA-SNN) not only significantly outperforms the baseline SNN (BL-SNN) and two other SNN models with degenerated attention modules, but also achieves competitive accuracy with the existing state-of-the-art (SOTA) methods. Additionally, our detailed analysis shows that the proposed SCTFA-SNN model has strong robustness to noise and outstanding stability when faced with incomplete data, while maintaining acceptable complexity and efficiency. Overall, these findings indicate that incorporating appropriate cognitive mechanisms of the brain may provide a promising approach to elevate the capabilities of SNNs.

Composing recurrent spiking neural networks using locally-recurrent motifs and risk-mitigating architectural optimization

In neural circuits, recurrent connectivity plays a crucial role in network function and stability. However, existing recurrent spiking neural networks (RSNNs) are often constructed by random connections without optimization. While RSNNs can produce rich dynamics that are critical for memory formation and learning, systemic architectural optimization of RSNNs is still an open challenge. We aim to enable systematic design of large RSNNs via a new scalable RSNN architecture and automated architectural optimization. We compose RSNNs based on a layer architecture called Sparsely-Connected Recurrent Motif Layer (SC-ML) that consists of multiple small recurrent motifs wired together by sparse lateral connections. The small size of the motifs and sparse inter-motif connectivity leads to an RSNN architecture scalable to large network sizes. We further propose a method called Hybrid Risk-Mitigating Architectural Search (HRMAS) to systematically optimize the topology of the proposed recurrent motifs and SC-ML layer architecture. HRMAS is an alternating two-step optimization process by which we mitigate the risk of network instability and performance degradation caused by architectural change by introducing a novel biologically-inspired "self-repairing" mechanism through intrinsic plasticity. The intrinsic plasticity is introduced to the second step of each HRMAS iteration and acts as unsupervised fast self-adaptation to structural and synaptic weight modifications introduced by the first step during the RSNN architectural "evolution." We demonstrate that the proposed automatic architecture optimization leads to significant performance gains over existing manually designed RSNNs: we achieve 96.44% on TI46-Alpha, 94.66% on N-TIDIGITS, 90.28% on DVS-Gesture, and 98.72% on N-MNIST. To the best of the authors' knowledge, this is the first work to perform systematic architecture optimization on RSNNs.

Weighted residual network for SAR automatic target recognition with data augmentation

Introduction

Decades of research have been dedicated to overcoming the obstacles inherent in synthetic aperture radar (SAR) automatic target recognition (ATR). The rise of deep learning technologies has brought a wave of new possibilities, demonstrating significant progress in the field. However, challenges like the susceptibility of SAR images to noise, the requirement for large-scale training datasets, and the often protracted duration of model training still persist.

Methods

This paper introduces a novel data augmentation strategy to address these issues. Our method involves the intentional addition and subsequent removal of speckle noise to artificially enlarge the scope of training data through noise perturbation. Furthermore, we propose a modified network architecture named weighted ResNet, which incorporates residual strain controls for enhanced performance. This network is designed to be computationally efficient and to minimize the amount of training data required.

Results

Through rigorous experimental analysis, our research confirms that the proposed data augmentation method, when used in conjunction with the weighted ResNet model, significantly reduces the time needed for training. It also improves the SAR ATR capabilities.

Discussion

Compared to existing models and methods tested, the combination of our data augmentation scheme and the weighted ResNet framework achieves higher computational efficiency and better recognition accuracy in SAR ATR applications. This suggests that our approach could be a valuable advancement in the field of SAR image analysis.

Robust data hiding for JPEG images with invertible neural network

JPEG compression will cause severe distortion to the shared compressed image, which brings great challenges to extracting messages correctly from the stego image. To address such challenges, we propose a novel end-to-end robust data hiding scheme for JPEG images. The embedding and extracting secret messages on the quantized discrete cosine transform (DCT) coefficients are implemented by the bi-directional process of the invertible neural network (INN), which can provide intrinsic robustness against lossy JPEG compression. We design a JPEG compression attack module to simulate the JPEG compression process, which helps the network automatically learn how to recover the secret message from JPEG compressed image. Experimental results have demonstrated that our method achieves strong robustness against lossy JPEG compression, and also significantly improves the security compared with the existing data hiding methods on the premise of ensuring image quality and high capacity. For example, the detection error of our method against XuNet has been increased by 3.45% over the existing data hiding methods.

STSC-SNN: Spatio-Temporal Synaptic Connection with temporal convolution and attention for spiking neural networks

Spiking neural networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a Spatio-Temporal Synaptic Connection SNN (STSC-SNN) model to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Specifically, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance via varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.