Efficient Spike-Driven Learning With Dendritic Event-Based Processing

Spiking neural networks on FPGA: A survey of methodologies and recent advancements

The mimicry of the biological brain's structure in information processing enables spiking neural networks (SNNs) to exhibit significantly reduced power consumption compared to conventional systems. Consequently, these networks have garnered heightened attention and spurred extensive research endeavors in recent years, proposing various structures to achieve low power consumption, high speed, and improved recognition ability. However, researchers are still in the early stages of developing more efficient neural networks that more closely resemble the biological brain. This development and research require suitable hardware for execution with appropriate capabilities, and field-programmable gate array (FPGA) serves as a highly qualified candidate compared to existing hardware such as central processing unit (CPU) and graphics processing unit (GPU). FPGA, with parallel processing capabilities similar to the brain, lower latency and power consumption, and higher throughput, is highly eligible hardware for assisting in the development of spiking neural networks. In this review, an attempt has been made to facilitate researchers' path to further develop this field by collecting and examining recent works and the challenges that hinder the implementation of these networks on FPGA.

Graphene-Based Artificial Dendrites for Bio-Inspired Learning in Spiking Neuromorphic Systems

Analog neuromorphic computing systems emulate the parallelism and connectivity of the human brain, promising greater expressivity and energy efficiency compared to those of digital systems. Though many devices have emerged as candidates for artificial neurons and artificial synapses, there have been few device candidates for artificial dendrites. In this work, we report on biocompatible graphene-based artificial dendrites (GrADs) that can implement dendritic processing. By using a dual side-gate configuration, current applied through a Nafion membrane can be used to control device conductance across a trilayer graphene channel, showing spatiotemporal responses of leaky recurrent, alpha, and Gaussian dendritic potentials. The devices can be variably connected to enable higher-order neuronal responses, and we show through data-driven spiking neural network simulations that spiking activity is reduced by ≤15% without accuracy loss while low-frequency operation is stabilized. This positions the GrADs as strong candidates for energy efficient bio-interfaced spiking neural networks.

Leveraging dendritic properties to advance machine learning and neuro-inspired computing

The brain is a remarkably capable and efficient system. It can process and store huge amounts of noisy and unstructured information, using minimal energy. In contrast, current artificial intelligence (AI) systems require vast resources for training while still struggling to compete in tasks that are trivial for biological agents. Thus, brain-inspired engineering has emerged as a promising new avenue for designing sustainable, next-generation AI systems. Here, we describe how dendritic mechanisms of biological neurons have inspired innovative solutions for significant AI problems, including credit assignment in multi-layer networks, catastrophic forgetting, and high-power consumption. These findings provide exciting alternatives to existing architectures, showing how dendritic research can pave the way for building more powerful and energy efficient artificial learning systems.

Optimizing event-based neural networks on digital neuromorphic architecture: a comprehensive design space exploration

Neuromorphic processors promise low-latency and energy-efficient processing by adopting novel brain-inspired design methodologies. Yet, current neuromorphic solutions still struggle to rival conventional deep learning accelerators' performance and area efficiency in practical applications. Event-driven data-flow processing and near/in-memory computing are the two dominant design trends of neuromorphic processors. However, there remain challenges in reducing the overhead of event-driven processing and increasing the mapping efficiency of near/in-memory computing, which directly impacts the performance and area efficiency. In this work, we discuss these challenges and present our exploration of optimizing event-based neural network inference on SENECA, a scalable and flexible neuromorphic architecture. To address the overhead of event-driven processing, we perform comprehensive design space exploration and propose spike-grouping to reduce the total energy and latency. Furthermore, we introduce the event-driven depth-first convolution to increase area efficiency and latency in convolutional neural networks (CNNs) on the neuromorphic processor. We benchmarked our optimized solution on keyword spotting, sensor fusion, digit recognition and high resolution object detection tasks. Compared with other state-of-the-art large-scale neuromorphic processors, our proposed optimizations result in a 6× to 300× improvement in energy efficiency, a 3× to 15× improvement in latency, and a 3× to 100× improvement in area efficiency. Our optimizations for event-based neural networks can be potentially generalized to a wide range of event-based neuromorphic processors.

Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

It is widely believed the brain-inspired spiking neural networks have the capability of processing temporal information owing to their dynamic attributes. However, how to understand what kind of mechanisms contributing to the learning ability and exploit the rich dynamic properties of spiking neural networks to satisfactorily solve complex temporal computing tasks in practice still remains to be explored. In this article, we identify the importance of capturing the multi-timescale components, based on which a multi-compartment spiking neural model with temporal dendritic heterogeneity, is proposed. The model enables multi-timescale dynamics by automatically learning heterogeneous timing factors on different dendritic branches. Two breakthroughs are made through extensive experiments: the working mechanism of the proposed model is revealed via an elaborated temporal spiking XOR problem to analyze the temporal feature integration at different levels; comprehensive performance benefits of the model over ordinary spiking neural networks are achieved on several temporal computing benchmarks for speech recognition, visual recognition, electroencephalogram signal recognition, and robot place recognition, which shows the best-reported accuracy and model compactness, promising robustness and generalization, and high execution efficiency on neuromorphic hardware. This work moves neuromorphic computing a significant step toward real-world applications by appropriately exploiting biological observations.

A 22-pJ/spike 73-Mspikes/s 130k-compartment neural array transceiver with conductance-based synaptic and membrane dynamics

Neuromorphic cognitive computing offers a bio-inspired means to approach the natural intelligence of biological neural systems in silicon integrated circuits. Typically, such circuits either reproduce biophysical neuronal dynamics in great detail as tools for computational neuroscience, or abstract away the biology by simplifying the functional forms of neural computation in large-scale systems for machine intelligence with high integration density and energy efficiency. Here we report a hybrid which offers biophysical realism in the emulation of multi-compartmental neuronal network dynamics at very large scale with high implementation efficiency, and yet with high flexibility in configuring the functional form and the network topology. The integrate-and-fire array transceiver (IFAT) chip emulates the continuous-time analog membrane dynamics of 65 k two-compartment neurons with conductance-based synapses. Fired action potentials are registered as address-event encoded output spikes, while the four types of synapses coupling to each neuron are activated by address-event decoded input spikes for fully reconfigurable synaptic connectivity, facilitating virtual wiring as implemented by routing address-event spikes externally through synaptic routing table. Peak conductance strength of synapse activation specified by the address-event input spans three decades of dynamic range, digitally controlled by pulse width and amplitude modulation (PWAM) of the drive voltage activating the log-domain linear synapse circuit. Two nested levels of micro-pipelining in the IFAT architecture improve both throughput and efficiency of synaptic input. This two-tier micro-pipelining results in a measured sustained peak throughput of 73 Mspikes/s and overall chip-level energy efficiency of 22 pJ/spike. Non-uniformity in digitally encoded synapse strength due to analog mismatch is mitigated through single-point digital offset calibration. Combined with the flexibly layered and recurrent synaptic connectivity provided by hierarchical address-event routing of registered spike events through external memory, the IFAT lends itself to efficient large-scale emulation of general biophysical spiking neural networks, as well as rate-based mapping of rectified linear unit (ReLU) neural activations.

BalanceHRNet: An effective network for bottom-up human pose estimation

In the study of human pose estimation, which is widely used in safety and sports scenes, the performance of deep learning methods is greatly reduced in high overlap rate and crowded scenes. Therefore, we propose a bottom-up model, called BalanceHRNet, which is based on balanced high-resolution module and a new branch attention module. BalanceHRNet draws on the multi-branch structure and fusion method of a popular model HigherHRNet. And our model overcomes the shortcoming of HigherHRNet that cannot obtain a large enough receptive field. Specifically, through the connecting structure in balanced high-resolution module, we can connect almost all convolutional layers and obtain a sufficiently large receptive field. At the same time, the multi-resolution representation can be maintained due to the use of balanced high-resolution module, which enable our model to recognize objects with richer scales and obtain more complex semantics information. And for branch fusion method, we design branch attention to obtain the importance of different branches at different stages. Finally, our model improves the accuracy while ensuring a smaller amount of computation than HigherHRNet. The CrowdPose dataset is used as test dataset, and HigherHRNet, AlphaPose, OpenPose and so on are taken as comparison models. The AP measured by BalanceHRNet is 63.0%, increased by 3.1% compared to best model - HigherHRNet. We also demonstrate the effectiveness of our network through the COCO(2017) keypoint detection dataset. Compared with HigherHRNet-w32, the AP of our model is improved by 1.6%.

Lightweight ResGRU: a deep learning-based prediction of SARS-CoV-2 (COVID-19) and its severity classification using multimodal chest radiography images

The new COVID-19 emerged in a town in China named Wuhan in December 2019, and since then, this deadly virus has infected 324 million people worldwide and caused 5.53 million deaths by January 2022. Because of the rapid spread of this pandemic, different countries are facing the problem of a shortage of resources, such as medical test kits and ventilators, as the number of cases increased uncontrollably. Therefore, developing a readily available, low-priced, and automated approach for COVID-19 identification is the need of the hour. The proposed study uses chest radiography images (CRIs) such as X-rays and computed tomography (CTs) to detect chest infections, as these modalities contain important information about chest infections. This research introduces a novel hybrid deep learning model named Lightweight ResGRU that uses residual blocks and a bidirectional gated recurrent unit to diagnose non-COVID and COVID-19 infections using pre-processed CRIs. Lightweight ResGRU is used for multi-modal two-class classification (normal and COVID-19), three-class classification (normal, COVID-19, and viral pneumonia), four-class classification (normal, COVID-19, viral pneumonia, and bacterial pneumonia), and COVID-19 severity types' classification (i.e., atypical appearance, indeterminate appearance, typical appearance, and negative for pneumonia). The proposed architecture achieved f-measure of 99.0%, 98.4%, 91.0%, and 80.5% for two-class, three-class, four-class, and COVID-19 severity level classifications, respectively, on unseen data. A large dataset is created by combining and changing different publicly available datasets. The results prove that radiologists can adopt this method to screen chest infections where test kits are limited.

BitBrain and Sparse Binary Coincidence (SBC) memories: Fast, robust learning and inference for neuromorphic architectures

We present an innovative working mechanism (the SBC memory) and surrounding infrastructure (BitBrain) based upon a novel synthesis of ideas from sparse coding, computational neuroscience and information theory that enables fast and adaptive learning and accurate, robust inference. The mechanism is designed to be implemented efficiently on current and future neuromorphic devices as well as on more conventional CPU and memory architectures. An example implementation on the SpiNNaker neuromorphic platform has been developed and initial results are presented. The SBC memory stores coincidences between features detected in class examples in a training set, and infers the class of a previously unseen test example by identifying the class with which it shares the highest number of feature coincidences. A number of SBC memories may be combined in a BitBrain to increase the diversity of the contributing feature coincidences. The resulting inference mechanism is shown to have excellent classification performance on benchmarks such as MNIST and EMNIST, achieving classification accuracy with single-pass learning approaching that of state-of-the-art deep networks with much larger tuneable parameter spaces and much higher training costs. It can also be made very robust to noise. BitBrain is designed to be very efficient in training and inference on both conventional and neuromorphic architectures. It provides a unique combination of single-pass, single-shot and continuous supervised learning; following a very simple unsupervised phase. Accurate classification inference that is very robust against imperfect inputs has been demonstrated. These contributions make it uniquely well-suited for edge and IoT applications.

Evolving deep convolutional neutral network by hybrid sine-cosine and extreme learning machine for real-time COVID19 diagnosis from X-ray images

The COVID19 pandemic globally and significantly has affected the life and health of many communities. The early detection of infected patients is effective in fighting COVID19. Using radiology (X-Ray) images is, perhaps, the fastest way to diagnose the patients. Thereby, deep Convolutional Neural Networks (CNNs) can be considered as applicable tools to diagnose COVID19 positive cases. Due to the complicated architecture of a deep CNN, its real-time training and testing become a challenging problem. This paper proposes using the Extreme Learning Machine (ELM) instead of the last fully connected layer to address this deficiency. However, the parameters' stochastic tuning of ELM's supervised section causes the final model unreliability. Therefore, to cope with this problem and maintain network reliability, the sine-cosine algorithm was utilized to tune the ELM's parameters. The designed network is then benchmarked on the COVID-Xray-5k dataset, and the results are verified by a comparative study with canonical deep CNN, ELM optimized by cuckoo search, ELM optimized by genetic algorithm, and ELM optimized by whale optimization algorithm. The proposed approach outperforms comparative benchmarks with a final accuracy of 98.83% on the COVID-Xray-5k dataset, leading to a relative error reduction of 2.33% compared to a canonical deep CNN. Even more critical, the designed network's training time is only 0.9421 ms and the overall detection test time for 3100 images is 2.721 s.

A brain-inspired robot pain model based on a spiking neural network

Introduction

Pain is a crucial function for organisms. Building a "Robot Pain" model inspired by organisms' pain could help the robot learn self-preservation and extend longevity. Most previous studies about robots and pain focus on robots interacting with people by recognizing their pain expressions or scenes, or avoiding obstacles by recognizing dangerous objects. Robots do not have human-like pain capacity and cannot adaptively respond to danger. Inspired by the evolutionary mechanisms of pain emergence and the Free Energy Principle (FEP) in the brain, we summarize the neural mechanisms of pain and construct a Brain-inspired Robot Pain Spiking Neural Network (BRP-SNN) with spike-time-dependent-plasticity (STDP) learning rule and population coding method.

Methods

The proposed model can quantify machine injury by detecting the coupling relationship between multi-modality sensory information and generating "robot pain" as an internal state.

Results

We provide a comparative analysis with the results of neuroscience experiments, showing that our model has biological interpretability. We also successfully tested our model on two tasks with real robots-the alerting actual injury task and the preventing potential injury task.

Discussion

Our work has two major contributions: (1) It has positive implications for the integration of pain concepts into robotics in the intelligent robotics field. (2) Our summary of pain's neural mechanisms and the implemented computational simulations provide a new perspective to explore the nature of pain, which has significant value for future pain research in the cognitive neuroscience field.

Neuromorphic Context-Dependent Learning Framework With Fault-Tolerant Spike Routing

Neuromorphic computing is a promising technology that realizes computation based on event-based spiking neural networks (SNNs). However, fault-tolerant on-chip learning remains a challenge in neuromorphic systems. This study presents the first scalable neuromorphic fault-tolerant context-dependent learning (FCL) hardware framework. We show how this system can learn associations between stimulation and response in two context-dependent learning tasks from experimental neuroscience, despite possible faults in the hardware nodes. Furthermore, we demonstrate how our novel fault-tolerant neuromorphic spike routing scheme can avoid multiple fault nodes successfully and can enhance the maximum throughput of the neuromorphic network by 0.9%-16.1% in comparison with previous studies. By utilizing the real-time computational capabilities and multiple-fault-tolerant property of the proposed system, the neuronal mechanisms underlying the spiking activities of neuromorphic networks can be readily explored. In addition, the proposed system can be applied in real-time learning and decision-making applications, brain-machine integration, and the investigation of brain cognition during learning.

Outer-synchronization criterions for asymmetric recurrent time-varying neural networks described by differential-algebraic system via data-sampling principles

Asymmetric recurrent time-varying neural networks (ARTNNs) can enable realistic brain-like models to help scholars explore the mechanisms of the human brain and thus realize the applications of artificial intelligence, whose dynamical behaviors such as synchronization has attracted extensive research interest due to its superior applicability and flexibility. In this paper, we examined the outer-synchronization of ARTNNs, which are described by the differential-algebraic system (DAS). By designing appropriate centralized and decentralized data-sampling approaches which fully account for information gathering at the times tk and tki. Using the characteristics of integral inequalities and the theory of differential equations, several novel suitable outer-synchronization conditions were established. Those conditions facilitate the analysis and applications of dynamical behaviors of ARTNNs. The superiority of the theoretical results was then demonstrated by using a numerical example.

Sim-to-real transfer and reality gap modeling in model predictive control for autonomous driving

The main challenge for the adoption of autonomous driving is to ensure an adequate level of safety. Considering the almost infinite variability of possible scenarios that autonomous vehicles would have to face, the use of autonomous driving simulators is becoming of utmost importance. Simulation suites allow the used of automated validation techniques in a wide variety of scenarios, and enable the development of closed-loop validation methods, such as machine learning and reinforcement learning approaches. However, simulation tools suffer from a standing flaw in that there is a noticeable gap between the simulation conditions and real-world scenarios. Although the use of simulators powers most of the research around autonomous driving, and is generally used within all domains it is divided into, there is an inherent source of error given the stochastic nature of activities performed in real world, which are unreplicable in computer environments. This paper proposes a new approach to assess the real-to-sim gap for path tracking systems. The aim is to narrow down the sources of error between simulation results and real-world conditions, and to evaluate the performance of the simulation suite in the design process by employing the information extracted from gap analysis, which adds a new dimension of development against other approaches for autonomous driving. A real-time model predictive controller (MPC) based on adaptive potential fields was developed and validated using the CARLA simulator. Both the path planning and vehicle control systems where tested in real traffic conditions. The error between the simulator and the real data acquisition was evaluated using the Pearson correlation coefficient (PCC) and the max normalized cross-correlation (MNCC). The controller was further evaluated on a process of sim-to-real transfer, and was finally tested both in simulation and real traffic conditions. A comparison was performed against an optimal-control ILQR-based model predictive controller was carried out to further showcase the validity of this approach.

UAV Localization Algorithm Based on Factor Graph Optimization in Complex Scenes

With the increasingly widespread application of UAV intelligence, the need for autonomous navigation and positioning is becoming more and more important. To solve the problem that UAV cannot perform localization in complex scenes, a new multi-source fusion framework factor graph optimization algorithm is used for UAV localization state estimation in this paper, which is based on IMU/GNSS/VO multi-source sensors. Based on the factor graph model and the iSAM incremental inference algorithm, a multi-source fusion model of IMU/GNSS/VO is established, including the IMU pre-integration factor, IMU bias factor, GNSS factor, and VO factor. Mathematical simulations and validations on the EuRoC dataset show that, when the selected sliding window size is 30, the factor graph optimization (FGO) algorithm can not only meet the requirements of real time and accuracy at the same time, but it also achieves a plug-and-play function in the event of local sensor failures. Finally, compared with the traditional federated Kalman algorithm and the adaptive federated Kalman algorithm, the positioning accuracy of the FGO algorithm in this paper is improved by 1.5-2-fold, and can effectively improve autonomous navigation system robustness and flexibility in complex scenarios. Moreover, the multi-source fusion framework in this paper is a general algorithm framework that can satisfy other scenarios and other types of sensor combinations.

Reframing control methods for parameters optimization in adversarial image generation

Training procedures for deep networks require the setting of several hyper-parameters that strongly affect the obtained results. The problem is even worse in adversarial learning strategies used for image generation where a proper balancing of the discriminative and generative networks is fundamental for an effective training. In this work we propose a novel hyper-parameters optimization strategy based on the use of Proportional-Integral (PI) and Proportional-Integral-Derivative (PID) controllers. Both open loop and closed loop schemes for the tuning of a single parameter or of multiple parameters together are proposed allowing an efficient parameter tuning without resorting to computationally demanding trial-and-error schemes. We applied the proposed strategies to the widely used BEGAN and CycleGAN models: They allowed to achieve a more stable training that converges faster. The obtained images are also sharper with a slightly better quality both visually and according to the FID and FCN metrics. Image translation results also showed better background preservation and less color artifacts with respect to CycleGAN.

Online subspace learning and imputation by Tensor-Ring decomposition

This paper considers the completion problem of a partially observed high-order streaming data, which is cast as an online low-rank tensor completion problem. Though the online low-rank tensor completion problem has drawn lots of attention in recent years, most of them are designed based on the traditional decomposition method, such as CP and Tucker. Inspired by the advantages of Tensor Ring decomposition over the traditional decompositions in expressing high-order data and its superiority in missing values estimation, this paper proposes two online subspace learning and imputation methods based on Tensor Ring decomposition. Specifically, we first propose an online Tensor Ring subspace learning and imputation model by formulating an exponentially weighted least squares with Frobenium norm regularization of TR-cores. Then, two commonly used optimization algorithms, i.e. alternating recursive least squares and stochastic-gradient algorithms, are developed to solve the proposed model. Numerical experiments show that the proposed methods are more effective to exploit the time-varying subspace in comparison with the conventional Tensor Ring completion methods. Besides, the proposed methods are demonstrated to be superior to obtain better results than state-of-the-art online methods in streaming data completion under varying missing ratios and noise.

Rural Workplace Sustainable Development of Smart Rural Governance Workplace Platform for Efficient Enterprise Performances

In the long developmental process, China's agriculture has transformed from organic agriculture to inorganic agriculture. New technologies have made the modernization of agriculture possible. However, most older people who are engaged in agriculture may not completely understand the modernization of agriculture. Based on the limitations of traditional image target detection methods, a deep learning-based pest target detection and recognition method is proposed from a blockchain perspective, to analyze and research agricultural data supervision and governance and explore the effectiveness of deep learning methods in crop pest detection and recognition. The comparative analysis demonstrates that the average precision (AP) of GA-CPN-LAR (global activation-characteristic pyramid network-local activation region) increases by 4.2% compared with other methods. Whether under the Inception or ResNet-50 backbone networks, the AP of GA-CPN-LAR is significantly better than other methods. Compared with the ResNet-50 backbone network, GA-CPN-LAR has higher accuracy and recall rates under Inception. Precision-recall curve measurement shows that the proposed method can significantly reduce the false detection rate and missed detection rate. The GA-CPN-LAR model proposed here has a higher AP value on the MPD dataset than the other target detection methods, which can be increased by 4.2%. Besides, the accuracy and recall of the GA-CPN-LAR method corresponding to two representative pests under the initial feature extractor are higher than the MPD dataset baseline. In addition, the research results of the MPD dataset and AgriPest dataset also show that the pest target detection method based on convolutional neural networks (CNNs) has a good presentation effect and can significantly reduce false detection and missed detection. Moreover, the pest regulation based on blockchain and deep learning comprehensively considers global and local feature extraction and pattern recognition, which positively impacts the conscientization of agricultural data processing and promotes the sustainable development of rural areas.

Heterogeneous Ensemble-Based Spike-Driven Few-Shot Online Learning

Spiking neural networks (SNNs) are regarded as a promising candidate to deal with the major challenges of current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the few-shot learning performance of artificial neural networks. Importantly, existing spike-based few-shot learning models do not target robust learning based on spatiotemporal dynamics and superior machine learning theory. In this paper, we propose a novel spike-based framework with the entropy theory, namely, heterogeneous ensemble-based spike-driven few-shot online learning (HESFOL). The proposed HESFOL model uses the entropy theory to establish the gradient-based few-shot learning scheme in a recurrent SNN architecture. We examine the performance of the HESFOL model based on the few-shot classification tasks using spiking patterns and the Omniglot data set, as well as the few-shot motor control task using an end-effector. Experimental results show that the proposed HESFOL scheme can effectively improve the accuracy and robustness of spike-driven few-shot learning performance. More importantly, the proposed HESFOL model emphasizes the application of modern entropy-based machine learning methods in state-of-the-art spike-driven learning algorithms. Therefore, our study provides new perspectives for further integration of advanced entropy theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

SAM: A Unified Self-Adaptive Multicompartmental Spiking Neuron Model for Learning With Working Memory

Working memory is a fundamental feature of biological brains for perception, cognition, and learning. In addition, learning with working memory, which has been show in conventional artificial intelligence systems through recurrent neural networks, is instrumental to advanced cognitive intelligence. However, it is hard to endow a simple neuron model with working memory, and to understand the biological mechanisms that have resulted in such a powerful ability at the neuronal level. This article presents a novel self-adaptive multicompartment spiking neuron model, referred to as SAM, for spike-based learning with working memory. SAM integrates four major biological principles including sparse coding, dendritic non-linearity, intrinsic self-adaptive dynamics, and spike-driven learning. We first describe SAM's design and explore the impacts of critical parameters on its biological dynamics. We then use SAM to build spiking networks to accomplish several different tasks including supervised learning of the MNIST dataset using sequential spatiotemporal encoding, noisy spike pattern classification, sparse coding during pattern classification, spatiotemporal feature detection, meta-learning with working memory applied to a navigation task and the MNIST classification task, and working memory for spatiotemporal learning. Our experimental results highlight the energy efficiency and robustness of SAM in these wide range of challenging tasks. The effects of SAM model variations on its working memory are also explored, hoping to offer insight into the biological mechanisms underlying working memory in the brain. The SAM model is the first attempt to integrate the capabilities of spike-driven learning and working memory in a unified single neuron with multiple timescale dynamics. The competitive performance of SAM could potentially contribute to the development of efficient adaptive neuromorphic computing systems for various applications from robotics to edge computing.

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

Limited text speech synthesis with electroglottograph based on Bi-LSTM and modified Tacotron-2

This paper proposes a framework of applying only the EGG signal for speech synthesis in the limited categories of contents scenario. EGG is a sort of physiological signal which can reflect the trends of the vocal cord movement. Note that EGG’s different acquisition method contrasted with speech signals, we exploit its application in speech synthesis under the following two scenarios. (1) To synthesize speeches under high noise circumstances, where clean speech signals are unavailable. (2) To enable dumb people who retain vocal cord vibration to speak again. Our study consists of two stages, EGG to text and text to speech. The first is a text content recognition model based on Bi-LSTM, which converts each EGG signal sample into the corresponding text with a limited class of contents. This model achieves 91.12% accuracy on the validation set in a 20-class content recognition experiment. Then the second step synthesizes speeches with the corresponding text and the EGG signal. Based on modified Tacotron-2, our model gains the Mel cepstral distortion (MCD) of 5.877 and the mean opinion score (MOS) of 3.87, which is comparable with the state-of-the-art performance and achieves an improvement by 0.42 and a relatively smaller model size than the origin Tacotron-2. Considering to introduce the characteristics of speakers contained in EGG to the final synthesized speech, we put forward a fine-grained fundamental frequency modification method, which adjusts the fundamental frequency according to EGG signals and achieves a lower MCD of 5.781 and a higher MOS of 3.94 than that without modification.

Functional Implications of Dale's Law in Balanced Neuronal Network Dynamics and Decision Making

The notion that a neuron transmits the same set of neurotransmitters at all of its post-synaptic connections, typically known as Dale's law, is well supported throughout the majority of the brain and is assumed in almost all theoretical studies investigating the mechanisms for computation in neuronal networks. Dale's law has numerous functional implications in fundamental sensory processing and decision-making tasks, and it plays a key role in the current understanding of the structure-function relationship in the brain. However, since exceptions to Dale's law have been discovered for certain neurons and because other biological systems with complex network structure incorporate individual units that send both positive and negative feedback signals, we investigate the functional implications of network model dynamics that violate Dale's law by allowing each neuron to send out both excitatory and inhibitory signals to its neighbors. We show how balanced network dynamics, in which large excitatory and inhibitory inputs are dynamically adjusted such that input fluctuations produce irregular firing events, are theoretically preserved for a single population of neurons violating Dale's law. We further leverage this single-population network model in the context of two competing pools of neurons to demonstrate that effective decision-making dynamics are also produced, agreeing with experimental observations from honeybee dynamics in selecting a food source and artificial neural networks trained in optimal selection. Through direct comparison with the classical two-population balanced neuronal network, we argue that the one-population network demonstrates more robust balanced activity for systems with less computational units, such as honeybee colonies, whereas the two-population network exhibits a more rapid response to temporal variations in network inputs, as required by the brain. We expect this study will shed light on the role of neurons violating Dale's law found in experiment as well as shared design principles across biological systems that perform complex computations.

Predicting Spike Features of Hodgkin-Huxley-Type Neurons With Simple Artificial Neural Network

Hodgkin-Huxley (HH)-type model is the most famous computational model for simulating neural activity. It shows the highest accuracy in capturing neuronal spikes, and its model parameters have definite physiological meanings. However, HH-type models are computationally expensive. To address this problem, a previous study proposed a spike prediction module (SPM) to predict whether a spike will take place 1 ms later based on three voltage values with intervals of 1 ms. Although SPM does well, it fails to evaluate the informative features of the spike. In this study, the feature prediction module (FPM) based on simple artificial neural network (ANN) was proposed to predict spike features including maximum voltage, minimum voltage, and dropping interval. Nine different HH-type models were adopted whose firing patterns cover most of the firing behaviors observed in the brain. Voltage and spike feature samples under constant external input current were collected for training and testing. Experiment results illustrated that the combination of SPM and FPM can accurately predict the spiking part of different HH-type models and can generalize to unseen types of input current. The combination of SPM and FPM may offer a possible way to simulate the action potentials of biological neurons with high accuracy and efficiency.

Automated Detection of COVID-19 Using Deep Learning Approaches with Paper-Based ECG Reports

One of the pandemics that have caused many deaths is the Coronavirus disease 2019 (COVID-19). It first appeared in late 2019, and many deaths are increasing day by day until now. Therefore, the early diagnosis of COVID-19 has become a salient issue. Additionally, the current diagnosis methods have several demerits, and a new investigation is required to enhance the diagnosis performance. In this paper, a set of phases are performed, such as collecting data, filtering and augmenting images, extracting features, and classifying ECG images. The data were obtained from two publicly available ECG image datasets, and one of them contained COVID ECG reports. A set of preprocessing methods are applied to the ECG images, and data augmentation is performed to balance the ECG images based on the classes. A deep learning approach based on a convolutional neural network (CNN) is performed for feature extraction. Four different pre-trained models are applied, such as Vgg16, Vgg19, ResNet-101, and Xception. Moreover, an ensemble of Xception and the temporary convolutional network (TCN), which is named ECGConvnet, is proposed. Finally, the results obtained from the former models are fed to four main classifiers. These classifiers are softmax, random forest (RF), multilayer perception (MLP), and support vector machine (SVM). The former classifiers are used to evaluate the diagnosis ability of the proposed methods. The classification scenario is based on fivefold cross-validation. Seven experiments are presented to evaluate the performance of the ECGConvnet. Three of them are multi-class, and the remaining are binary class diagnosing. Six out of seven experiments diagnose COVID-19 patients. The aforementioned experimental results indicated that ECGConvnet has the highest performance over other pre-trained models, and the SVM classifier showed higher accuracy in comparison with the other classifiers. The resulting accuracies from ECGConvnet based on SVM are (99.74%, 98.6%, 99.1% on the multi-class diagnosis tasks) and (99.8% on one of the binary-class diagnoses, while the remaining achieved 100%). It is possible to develop an automatic diagnosis system for COVID based on deep learning using ECG data.

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Artificial neural networks (ANNs), like convolutional neural networks (CNNs), have achieved the state-of-the-art results for many machine learning tasks. However, inference with large-scale full-precision CNNs must cause substantial energy consumption and memory occupation, which seriously hinders their deployment on mobile and embedded systems. Highly inspired from biological brain, spiking neural networks (SNNs) are emerging as new solutions because of natural superiority in brain-like learning and great energy efficiency with event-driven communication and computation. Nevertheless, training a deep SNN remains a main challenge and there is usually a big accuracy gap between ANNs and SNNs. In this paper, we introduce a hardware-friendly conversion algorithm called "scatter-and-gather" to convert quantized ANNs to lossless SNNs, where neurons are connected with ternary {-1,0,1} synaptic weights. Each spiking neuron is stateless and more like original McCulloch and Pitts model, because it fires at most one spike and need be reset at each time step. Furthermore, we develop an incremental mapping framework to demonstrate efficient network deployments on a reconfigurable neuromorphic chip. Experimental results show our spiking LeNet on MNIST and VGG-Net on CIFAR-10 datasetobtain 99.37% and 91.91% classification accuracy, respectively. Besides, the presented mapping algorithm manages network deployment on our neuromorphic chip with maximum resource efficiency and excellent flexibility. Our four-spike LeNet and VGG-Net on chip can achieve respective real-time inference speed of 0.38 ms/image, 3.24 ms/image, and an average power consumption of 0.28 mJ/image and 2.3 mJ/image at 0.9 V, 252 MHz, which is nearly two orders of magnitude more efficient than traditional GPUs.

An inertial neural network approach for robust time-of-arrival localization considering clock asynchronization

This paper presents an inertial neural network to solve the source localization optimization problem with l₁-norm objective function based on the time of arrival (TOA) localization technique. The convergence and stability of the inertial neural network are analyzed by the Lyapunov function method. An inertial neural network iterative approach is further used to find a better solution among the solutions with different inertial parameters. Furthermore, the clock asynchronization is considered in the TOA l₁-norm model for more general real applications, and the corresponding inertial neural network iterative approach is addressed. The numerical simulations and real data are both considered in the experiments. In the simulation experiments, the noise contains uncorrelated zero-mean Gaussian noise and uniform distributed outliers. In the real experiments, the data is obtained by using the ultra wide band (UWB) technology hardware modules. Whether or not there is clock asynchronization, the results show that the proposed approach always can find a more accurate source position compared with some of the existing algorithms, which implies that the proposed approach is more effective than the compared ones.