CerebelluMorphic: Large-Scale Neuromorphic Model and Architecture for Supervised Motor Learning

Memory-Dependent Computation and Learning in Spiking Neural Networks Through Hebbian Plasticity

Spiking neural networks (SNNs) are the basis for many energy-efficient neuromorphic hardware systems. While there has been substantial progress in SNN research, artificial SNNs still lack many capabilities of their biological counterparts. In biological neural systems, memory is a key component that enables the retention of information over a huge range of temporal scales, ranging from hundreds of milliseconds up to years. While Hebbian plasticity is believed to play a pivotal role in biological memory, it has so far been analyzed mostly in the context of pattern completion and unsupervised learning in artificial and SNNs. Here, we propose that Hebbian plasticity is fundamental for computations in biological and artificial spiking neural systems. We introduce a novel memory-augmented SNN architecture that is enriched by Hebbian synaptic plasticity. We show that Hebbian enrichment renders SNNs surprisingly versatile in terms of their computational as well as learning capabilities. It improves their abilities for out-of-distribution generalization, one-shot learning, cross-modal generative association, language processing, and reward-based learning. This suggests that powerful cognitive neuromorphic systems can be built based on this principle.

Power spectral analysis of voltage-gated channels in neurons

This article develops a fundamental insight into the behavior of neuronal membranes, focusing on their responses to stimuli measured with power spectra in the frequency domain. It explores the use of linear and nonlinear (quadratic sinusoidal analysis) approaches to characterize neuronal function. It further delves into the random theory of internal noise of biological neurons and the use of stochastic Markov models to investigate these fluctuations. The text also discusses the origin of conductance noise and compares different power spectra for interpreting this noise. Importantly, it introduces a novel sequential chemical state model, named p 2, which is more general than the Hodgkin-Huxley formulation, so that the probability for an ion channel to be open does not imply exponentiation. In particular, it is demonstrated that the p 2 (without exponentiation) and n 4 (with exponentiation) models can produce similar neuronal responses. A striking relationship is also shown between fluctuation and quadratic power spectra, suggesting that voltage-dependent random mechanisms can have a significant impact on deterministic nonlinear responses, themselves known to have a crucial role in the generation of action potentials in biological neural networks.

Brain-inspired biomimetic robot control: a review

Complex robotic systems, such as humanoid robot hands, soft robots, and walking robots, pose a challenging control problem due to their high dimensionality and heavy non-linearities. Conventional model-based feedback controllers demonstrate robustness and stability but struggle to cope with the escalating system design and tuning complexity accompanying larger dimensions. In contrast, data-driven methods such as artificial neural networks excel at representing high-dimensional data but lack robustness, generalization, and real-time adaptiveness. In response to these challenges, researchers are directing their focus to biological paradigms, drawing inspiration from the remarkable control capabilities inherent in the human body. This has motivated the exploration of new control methods aimed at closely emulating the motor functions of the brain given the current insights in neuroscience. Recent investigation into these Brain-Inspired control techniques have yielded promising results, notably in tasks involving trajectory tracking and robot locomotion. This paper presents a comprehensive review of the foremost trends in biomimetic brain-inspired control methods to tackle the intricacies associated with controlling complex robotic systems.

Artificial cerebellum on FPGA: realistic real-time cerebellar spiking neural network model capable of real-world adaptive motor control

The cerebellum plays a central role in motor control and learning. Its neuronal network architecture, firing characteristics of component neurons, and learning rules at their synapses have been well understood in terms of anatomy and physiology. A realistic artificial cerebellum with mimetic network architecture and synaptic plasticity mechanisms may allow us to analyze cerebellar information processing in the real world by applying it to adaptive control of actual machines. Several artificial cerebellums have previously been constructed, but they require high-performance hardware to run in real-time for real-world machine control. Presently, we implemented an artificial cerebellum with the size of 104 spiking neuron models on a field-programmable gate array (FPGA) which is compact, lightweight, portable, and low-power-consumption. In the implementation three novel techniques are employed: (1) 16-bit fixed-point operation and randomized rounding, (2) fully connected spike information transmission, and (3) alternative memory that uses pseudo-random number generators. We demonstrate that the FPGA artificial cerebellum runs in real-time, and its component neuron models behave as those in the corresponding artificial cerebellum configured on a personal computer in Python. We applied the FPGA artificial cerebellum to the adaptive control of a machine in the real world and demonstrated that the artificial cerebellum is capable of adaptively reducing control error after sudden load changes. This is the first implementation and demonstration of a spiking artificial cerebellum on an FPGA applicable to real-world adaptive control. The FPGA artificial cerebellum may provide neuroscientific insights into cerebellar information processing in adaptive motor control and may be applied to various neuro-devices to augment and extend human motor control capabilities.

Building a realistic, scalable memory model with independent engrams using a homeostatic mechanism

Memory formation is usually associated with Hebbian learning and synaptic plasticity, which changes the synaptic strengths but omits structural changes. A recent study suggests that structural plasticity can also lead to silent memory engrams, reproducing a conditioned learning paradigm with neuron ensembles. However, this study is limited by its way of synapse formation, enabling the formation of only one memory engram. Overcoming this, our model allows the formation of many engrams simultaneously while retaining high neurophysiological accuracy, e.g., as found in cortical columns. We achieve this by substituting the random synapse formation with the Model of Structural Plasticity. As a homeostatic model, neurons regulate their activity by growing and pruning synaptic elements based on their current activity. Utilizing synapse formation based on the Euclidean distance between the neurons with a scalable algorithm allows us to easily simulate 4 million neurons with 343 memory engrams. These engrams do not interfere with one another by default, yet we can change the simulation parameters to form long-reaching associations. Our model's analysis shows that homeostatic engram formation requires a certain spatiotemporal order of events. It predicts that synaptic pruning precedes and enables synaptic engram formation and that it does not occur as a mere compensatory response to enduring synapse potentiation as in Hebbian plasticity with synaptic scaling. Our model paves the way for simulations addressing further inquiries, ranging from memory chains and hierarchies to complex memory systems comprising areas with different learning mechanisms.

Investigation on Vision System: Digital FPGA Implementation in Case of Retina Rod Cells

The development of prostheses and treatments for illnesses and recovery has recently been centered on hardware modeling for various delicate biological components, including the nervous system, brain, eyes, and heart. The retina, being the thinnest and deepest layer of the eye, is of particular interest. In this study, we employ the Nyquist-Based Approximation of Retina Rod Cell (NBAoRRC) approach, which has been adapted to utilize Look-Up Tables (LUTs) rather than original functions, to implement rod cells in the retina using cost-effective hardware. In modern mathematical models, numerous nonlinear functions are used to represent the activity of these cells. However, these nonlinear functions would require a substantial amount of hardware for direct implementation and may not meet the required speed constraints. The proposed method eliminates the need for multiplication functions and utilizes a fast, cost-effective rod cell device. Simulation results demonstrate the extent to which the proposed model aligns with the behavior of the primary rod cell model, particularly in terms of dynamic behavior. Based on the results of hardware implementation using the Field-Programmable Gate Arrays (FPGA) board Virtex-5, the proposed model is shown to be reliable, consume 30 percent less power than the primary model, and have reduced hardware resource requirements. Based on the results of hardware implementation using the reconfigurable FPGA board Virtex-5, the proposed model is reliable, uses 30% less power consumption than the primary model in the worth state of the set of approximation method, and has a reduced hardware resource requirement. In fact, using the proposed model, this reduction in the power consumption can be achieved. Finally, in this article, by using the LUT which is systematically sampled (Nyquist rate), we were able to remove all costly operators in terms of hardware (digital) realization and achieve very good results in the field of digital implementation in two scales of network and single neuron.

An Integrated Neurorobotics Model of the Cerebellar-Basal Ganglia Circuitry

This work presents a neurorobotics model of the brain that integrates the cerebellum and the basal ganglia regions to coordinate movements in a humanoid robot. This cerebellar-basal ganglia circuitry is well known for its relevance to the motor control used by most mammals. Other computational models have been designed for similar applications in the robotics field. However, most of them completely ignore the interplay between neurons from the basal ganglia and cerebellum. Recently, neuroscientists indicated that neurons from both regions communicate not only at the level of the cerebral cortex but also at the subcortical level. In this work, we built an integrated neurorobotics model to assess the capacity of the network to predict and adjust the motion of the hands of a robot in real time. Our model was capable of performing different movements in a humanoid robot by respecting the sensorimotor loop of the robot and the biophysical features of the neuronal circuitry. The experiments were executed in simulation and the real world. We believe that our proposed neurorobotics model can be an important tool for new studies on the brain and a reference toward new robot motor controllers.

Exploring spiking neural networks: a comprehensive analysis of mathematical models and applications

This article presents a comprehensive analysis of spiking neural networks (SNNs) and their mathematical models for simulating the behavior of neurons through the generation of spikes. The study explores various models, including LIF and NLIF, for constructing SNNs and investigates their potential applications in different domains. However, implementation poses several challenges, including identifying the most appropriate model for classification tasks that demand high accuracy and low-performance loss. To address this issue, this research study compares the performance, behavior, and spike generation of multiple SNN models using consistent inputs and neurons. The findings of the study provide valuable insights into the benefits and challenges of SNNs and their models, emphasizing the significance of comparing multiple models to identify the most effective one. Moreover, the study quantifies the number of spiking operations required by each model to process the same inputs and produce equivalent outputs, enabling a thorough assessment of computational efficiency. The findings provide valuable insights into the benefits and limitations of SNNs and their models. The research underscores the significance of comparing different models to make informed decisions in practical applications. Additionally, the results reveal essential variations in biological plausibility and computational efficiency among the models, further emphasizing the importance of selecting the most suitable model for a given task. Overall, this study contributes to a deeper understanding of SNNs and offers practical guidelines for using their potential in real-world scenarios.

ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network

Recent developments in artificial neural networks and their learning algorithms have enabled new research directions in computer vision, language modeling, and neuroscience. Among various neural network algorithms, spiking neural networks (SNNs) are well-suited for understanding the behavior of biological neural circuits. In this work, we propose to guide the training of a sparse SNN in order to replace a sub-region of a cultured hippocampal network with limited hardware resources. To verify our approach with a realistic experimental setup, we record spikes of cultured hippocampal neurons with a microelectrode array (in vitro). The main focus of this work is to dynamically cut unimportant synapses during SNN training on the fly so that the model can be realized on resource-constrained hardware, e.g., implantable devices. To do so, we adopt a simple STDP learning rule to easily select important synapses that impact the quality of spike timing learning. By combining the STDP rule with online supervised learning, we can precisely predict the spike pattern of the cultured network in real-time. The reduction in the model complexity, i.e., the reduced number of connections, significantly reduces the required hardware resources, which is crucial in developing an implantable chip for the treatment of neurological disorders. In addition to the new learning algorithm, we prototype a sparse SNN hardware on a small FPGA with pipelined execution and parallel computing to verify the possibility of real-time replacement. As a result, we can replace a sub-region of the biological neural circuit within 22 μs using 2.5 × fewer hardware resources, i.e., by allowing 80% sparsity in the SNN model, compared to the fully-connected SNN model. With energy-efficient algorithms and hardware, this work presents an essential step toward real-time neuroprosthetic computation.

The computational power of the human brain

At the end of the 20th century, analog systems in computer science have been widely replaced by digital systems due to their higher computing power. Nevertheless, the question keeps being intriguing until now: is the brain analog or digital? Initially, the latter has been favored, considering it as a Turing machine that works like a digital computer. However, more recently, digital and analog processes have been combined to implant human behavior in robots, endowing them with artificial intelligence (AI). Therefore, we think it is timely to compare mathematical models with the biology of computation in the brain. To this end, digital and analog processes clearly identified in cellular and molecular interactions in the Central Nervous System are highlighted. But above that, we try to pinpoint reasons distinguishing in silico computation from salient features of biological computation. First, genuinely analog information processing has been observed in electrical synapses and through gap junctions, the latter both in neurons and astrocytes. Apparently opposed to that, neuronal action potentials (APs) or spikes represent clearly digital events, like the yes/no or 1/0 of a Turing machine. However, spikes are rarely uniform, but can vary in amplitude and widths, which has significant, differential effects on transmitter release at the presynaptic terminal, where notwithstanding the quantal (vesicular) release itself is digital. Conversely, at the dendritic site of the postsynaptic neuron, there are numerous analog events of computation. Moreover, synaptic transmission of information is not only neuronal, but heavily influenced by astrocytes tightly ensheathing the majority of synapses in brain (tripartite synapse). At least at this point, LTP and LTD modifying synaptic plasticity and believed to induce short and long-term memory processes including consolidation (equivalent to RAM and ROM in electronic devices) have to be discussed. The present knowledge of how the brain stores and retrieves memories includes a variety of options (e.g., neuronal network oscillations, engram cells, astrocytic syncytium). Also epigenetic features play crucial roles in memory formation and its consolidation, which necessarily guides to molecular events like gene transcription and translation. In conclusion, brain computation is not only digital or analog, or a combination of both, but encompasses features in parallel, and of higher orders of complexity.

Adaptive STDP-based on-chip spike pattern detection

A spiking neural network (SNN) is a bottom-up tool used to describe information processing in brain microcircuits. It is becoming a crucial neuromorphic computational model. Spike-timing-dependent plasticity (STDP) is an unsupervised brain-like learning rule implemented in many SNNs and neuromorphic chips. However, a significant performance gap exists between ideal model simulation and neuromorphic implementation. The performance of STDP learning in neuromorphic chips deteriorates because the resolution of synaptic efficacy in such chips is generally restricted to 6 bits or less, whereas simulations employ the entire 64-bit floating-point precision available on digital computers. Previously, we introduced a bio-inspired learning rule named adaptive STDP and demonstrated via numerical simulation that adaptive STDP (using only 4-bit fixed-point synaptic efficacy) performs similarly to STDP learning (using 64-bit floating-point precision) in a noisy spike pattern detection model. Herein, we present the experimental results demonstrating the performance of adaptive STDP learning. To the best of our knowledge, this is the first study that demonstrates unsupervised noisy spatiotemporal spike pattern detection to perform well and maintain the simulation performance on a mixed-signal CMOS neuromorphic chip with low-resolution synaptic efficacy. The chip was designed in Taiwan Semiconductor Manufacturing Company (TSMC) 250 nm CMOS technology node and comprises a soma circuit and 256 synapse circuits along with their learning circuitry.

Trends and Challenges in AIoT/IIoT/IoT Implementation

For the next coming years, metaverse, digital twin and autonomous vehicle applications are the leading technologies for many complex applications hitherto inaccessible such as health and life sciences, smart home, smart agriculture, smart city, smart car and logistics, Industry 4.0, entertainment (video game) and social media applications, due to recent tremendous developments in process modeling, supercomputing, cloud data analytics (deep learning, etc.), communication network and AIoT/IIoT/IoT technologies. AIoT/IIoT/IoT is a crucial research field because it provides the essential data to fuel metaverse, digital twin, real-time Industry 4.0 and autonomous vehicle applications. However, the science of AIoT is inherently multidisciplinary, and therefore, it is difficult for readers to understand its evolution and impacts. Our main contribution in this article is to analyze and highlight the trends and challenges of the AIoT technology ecosystem including core hardware (MCU, MEMS/NEMS sensors and wireless access medium), core software (operating system and protocol communication stack) and middleware (deep learning on a microcontroller: TinyML). Two low-powered AI technologies emerge: TinyML and neuromorphic computing, but only one AIoT/IIoT/IoT device implementation using TinyML dedicated to strawberry disease detection as a case study. So far, despite the very rapid progress of AIoT/IIoT/IoT technologies, several challenges remain to be overcome such as safety, security, latency, interoperability and reliability of sensor data, which are essential characteristics to meet the requirements of metaverse, digital twin, autonomous vehicle and Industry 4.0. applications.

Highly efficient neuromorphic learning system of spiking neural network with multi-compartment leaky integrate-and-fire neurons

A spiking neural network (SNN) is considered a high-performance learning system that matches the digital circuits and presents higher efficiency due to the architecture and computation of spiking neurons. While implementing a SNN on a field-programmable gate array (FPGA), the gradient back-propagation through layers consumes a surprising number of resources. In this paper, we aim to realize an efficient architecture of SNN on the FPGA to reduce resource and power consumption. The multi-compartment leaky integrate-and-fire (MLIF) model is used to convert spike trains to the plateau potential in dendrites. We accumulate the potential in the apical dendrite during the training period. The average of this accumulative result is the dendritic plateau potential and is used to guide the updates of synaptic weights. Based on this architecture, the SNN is implemented on FPGA efficiently. In the implementation of a neuromorphic learning system, the shift multiplier (shift MUL) module and piecewise linear (PWL) algorithm are used to replace multipliers and complex nonlinear functions to match the digital circuits. The neuromorphic learning system is constructed with resources on FPGA without dataflow between on-chip and off-chip memories. Our neuromorphic learning system performs with higher resource utilization and power efficiency than previous on-chip learning systems.

Instant-Hybrid Neural-Cryptography (IHNC) based on fast machine learning

Nowadays, cryptographic systems’ designers are facing significant challenges in their designs. They have to constantly search for new ideas of fast unbreakable algorithms with a very powerful key generator. In this paper, we propose a novel hybrid neural-cryptography methodology. It depends on new rule of very fast Backpropagation (BP) instant machine learning (ML). This proposed Hybrid Cryptography system is constructed from Encryptor and Decryptor based on the asymmetric Autoencoder type. The Encryptor encrypts and compresses a set of data to be instant code (i-code) using public key. While the Decryptor recovers this i-code (ciphered-data) based on two keys together. The first is the private key and the other is called instant-key (i-key). This i-key is generated from 3 factors as well (the original data itself, the generated i-code and the private key). The i-key is changing periodically with every transformation of plain data set, so it is powerful unpredictable key against the brute force.

Emergence of associative learning in a neuromorphic inference network

OBJECTIVE

In the theoretical framework of predictive coding and active inference, the brain can be viewed as instantiating a rich generative model of the world that predicts incoming sensory data while continuously updating its parameters via minimization of prediction errors. While this theory has been successfully applied to cognitive processes - by modelling the activity of functional neural networks at a mesoscopic scale - the validity of the approach when modelling neurons as an ensemble of inferring agents, in a biologically plausible architecture, remained to be explored.

APPROACH

We modelled a simplified cerebellar circuit with individual neurons acting as Bayesian agents to simulate the classical delayed eyeblink conditioning protocol. Neurons and synapses adjusted their activity to minimize their prediction error, which was used as the network cost function. This cerebellar network was then implemented in hardware by replicating digital neuronal elements via a low-power microcontroller.

MAIN RESULTS

Persistent changes of synaptic strength - that mirrored neurophysiological observations - emerged via local (neurocentric) prediction error minimization, leading to the expression of associative learning. The same paradigm was effectively emulated in low-power hardware showing remarkably efficient performance compared to conventional neuromorphic architectures.

SIGNIFICANCE

These findings show that: i) an ensemble of free energy minimizing neurons - organized in a biological plausible architecture - can recapitulate functional self-organization observed in nature, such as associative plasticity, and ii) a neuromorphic network of inference units can learn unsupervised tasks without embedding predefined learning rules in the circuit, thus providing a potential avenue to a novel form of brain-inspired artificial intelligence.

Combining Optical Coherence Tomography and Fundus Photography to Improve Glaucoma Screening

We aimed to evaluate the accuracy of glaucoma screening using fundus photography combined with optical coherence tomography and determine the agreement between ophthalmologists and ophthalmology residents. We used a comprehensive ophthalmologic examination dataset obtained from 503 cases (1006 eyes). Of the 1006 eyes, 132 had a confirmed glaucoma diagnosis. Overall, 24 doctors, comprising two groups (ophthalmologists and ophthalmology residents, 12 individuals/group), analyzed the data presented in three screening strategies as follows: (1) fundus photography alone, (2) fundus photography + optical coherence tomography, and (3) fundus photography + optical coherence tomography + comprehensive examination. We investigated the diagnostic accuracy (sensitivity and specificity). The respective sensitivity and specificity values for the diagnostic accuracy obtained by 24 doctors, 12 ophthalmologists, and 12 ophthalmology residents were as follows: (1) fundus photography: sensitivity, 55.4%, 55.4%, and 55.4%; specificity, 91.8%, 94.0%, and 89.6%; (2) fundus photography + OCT: sensitivity, 80.0%, 82.3%, and 77.8%; specificity, 91.7%, 92.9%, and 90.6%; and (3) fundus photography + OCT + comprehensive examination: sensitivity 78.4%, 79.8%, and 77.1%; specificity, 92.7%, 94.0%, and 91.3%. The diagnostic accuracy of glaucoma screening significantly increased with optical coherence tomography. Following its addition, ophthalmologists could more effectively improve the diagnostic accuracy than ophthalmology residents. Screening accuracy is improved when optical coherence tomography is added to fundus photography.

SAM: A Unified Self-Adaptive Multicompartmental Spiking Neuron Model for Learning With Working Memory

Working memory is a fundamental feature of biological brains for perception, cognition, and learning. In addition, learning with working memory, which has been show in conventional artificial intelligence systems through recurrent neural networks, is instrumental to advanced cognitive intelligence. However, it is hard to endow a simple neuron model with working memory, and to understand the biological mechanisms that have resulted in such a powerful ability at the neuronal level. This article presents a novel self-adaptive multicompartment spiking neuron model, referred to as SAM, for spike-based learning with working memory. SAM integrates four major biological principles including sparse coding, dendritic non-linearity, intrinsic self-adaptive dynamics, and spike-driven learning. We first describe SAM's design and explore the impacts of critical parameters on its biological dynamics. We then use SAM to build spiking networks to accomplish several different tasks including supervised learning of the MNIST dataset using sequential spatiotemporal encoding, noisy spike pattern classification, sparse coding during pattern classification, spatiotemporal feature detection, meta-learning with working memory applied to a navigation task and the MNIST classification task, and working memory for spatiotemporal learning. Our experimental results highlight the energy efficiency and robustness of SAM in these wide range of challenging tasks. The effects of SAM model variations on its working memory are also explored, hoping to offer insight into the biological mechanisms underlying working memory in the brain. The SAM model is the first attempt to integrate the capabilities of spike-driven learning and working memory in a unified single neuron with multiple timescale dynamics. The competitive performance of SAM could potentially contribute to the development of efficient adaptive neuromorphic computing systems for various applications from robotics to edge computing.

Novel Machine Learning for Big Data Analytics in Intelligent Support Information Management Systems

Two-dimensional 1 arrays of bi-component structures made of cobalt and permalloy elliptical dots with thickness of 25 nm, length 1 mm and width of 225 nm, have been prepared by a self-aligned shadow deposition technique. Brillouin light scattering has been exploited to study the frequency dependence of thermally excited magnetic eigenmodes on the intensity of the external magnetic field, applied along the easy axis of the elements.

Scientific information technology has been developed rapidly. Here, the purposes are to make people's lives more convenient and ensure information management and classification. The machine learning algorithm is improved to obtain the optimized Light Gradient Boosting Machine (LightGBM) algorithm. Then, an Android-based intelligent support information management system is designed based on LightGBM for the big data analysis and classification management of information in the intelligent support information management system. The system is designed with modules of employee registration and login, company announcement notice, attendance and attendance management, self-service, and daily tools with the company as the subject. Furthermore, the performance of the constructed information management system is analyzed through simulations. Results demonstrate that the training time of the optimized LightGBM algorithm can stabilize at about 100s, and the test time can stabilize at 0.68s. Besides, its accuracy rate can reach 89.24%, which is at least 3.6% higher than other machine learning algorithms. Moreover, the acceleration efficiency analysis of each algorithm suggests that the optimized LightGBM algorithm is suitable for processing large amounts of data; its acceleration effect is more apparent, and its acceleration ratio is higher than other algorithms. Hence, the constructed intelligent support information management system can reach a high accuracy while ensuring the error, with apparent acceleration effect. Therefore, this model can provide an experimental reference for information classification and management in various fields.

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

Limited text speech synthesis with electroglottograph based on Bi-LSTM and modified Tacotron-2

This paper proposes a framework of applying only the EGG signal for speech synthesis in the limited categories of contents scenario. EGG is a sort of physiological signal which can reflect the trends of the vocal cord movement. Note that EGG’s different acquisition method contrasted with speech signals, we exploit its application in speech synthesis under the following two scenarios. (1) To synthesize speeches under high noise circumstances, where clean speech signals are unavailable. (2) To enable dumb people who retain vocal cord vibration to speak again. Our study consists of two stages, EGG to text and text to speech. The first is a text content recognition model based on Bi-LSTM, which converts each EGG signal sample into the corresponding text with a limited class of contents. This model achieves 91.12% accuracy on the validation set in a 20-class content recognition experiment. Then the second step synthesizes speeches with the corresponding text and the EGG signal. Based on modified Tacotron-2, our model gains the Mel cepstral distortion (MCD) of 5.877 and the mean opinion score (MOS) of 3.87, which is comparable with the state-of-the-art performance and achieves an improvement by 0.42 and a relatively smaller model size than the origin Tacotron-2. Considering to introduce the characteristics of speakers contained in EGG to the final synthesized speech, we put forward a fine-grained fundamental frequency modification method, which adjusts the fundamental frequency according to EGG signals and achieves a lower MCD of 5.781 and a higher MOS of 3.94 than that without modification.

Periodicity Pitch Perception Part III: Sensibility and Pachinko Volatility

Neuromorphic computer models are used to explain sensory perceptions. Auditory models generate cochleagrams, which resemble the spike distributions in the auditory nerve. Neuron ensembles along the auditory pathway transform sensory inputs step by step and at the end pitch is represented in auditory categorical spaces. In two previous articles in the series on periodicity pitch perception an extended auditory model had been successfully used for explaining periodicity pitch proved for various musical instrument generated tones and sung vowels. In this third part in the series the focus is on octopus cells as they are central sensitivity elements in auditory cognition processes. A powerful numerical model had been devised, in which auditory nerve fibers (ANFs) spike events are the inputs, triggering the impulse responses of the octopus cells. Efficient algorithms are developed and demonstrated to explain the behavior of octopus cells with a focus on a simple event-based hardware implementation of a layer of octopus neurons. The main finding is, that an octopus' cell model in a local receptive field fine-tunes to a specific trajectory by a spike-timing-dependent plasticity (STDP) learning rule with synaptic pre-activation and the dendritic back-propagating signal as post condition. Successful learning explains away the teacher and there is thus no need for a temporally precise control of plasticity that distinguishes between learning and retrieval phases. Pitch learning is cascaded: At first octopus cells respond individually by self-adjustment to specific trajectories in their local receptive fields, then unions of octopus cells are collectively learned for pitch discrimination. Pitch estimation by inter-spike intervals is shown exemplary using two input scenarios: a simple sinus tone and a sung vowel. The model evaluation indicates an improvement in pitch estimation on a fixed time-scale.

Predicting Spike Features of Hodgkin-Huxley-Type Neurons With Simple Artificial Neural Network

Hodgkin-Huxley (HH)-type model is the most famous computational model for simulating neural activity. It shows the highest accuracy in capturing neuronal spikes, and its model parameters have definite physiological meanings. However, HH-type models are computationally expensive. To address this problem, a previous study proposed a spike prediction module (SPM) to predict whether a spike will take place 1 ms later based on three voltage values with intervals of 1 ms. Although SPM does well, it fails to evaluate the informative features of the spike. In this study, the feature prediction module (FPM) based on simple artificial neural network (ANN) was proposed to predict spike features including maximum voltage, minimum voltage, and dropping interval. Nine different HH-type models were adopted whose firing patterns cover most of the firing behaviors observed in the brain. Voltage and spike feature samples under constant external input current were collected for training and testing. Experiment results illustrated that the combination of SPM and FPM can accurately predict the spiking part of different HH-type models and can generalize to unseen types of input current. The combination of SPM and FPM may offer a possible way to simulate the action potentials of biological neurons with high accuracy and efficiency.

Automated Detection of COVID-19 Using Deep Learning Approaches with Paper-Based ECG Reports

One of the pandemics that have caused many deaths is the Coronavirus disease 2019 (COVID-19). It first appeared in late 2019, and many deaths are increasing day by day until now. Therefore, the early diagnosis of COVID-19 has become a salient issue. Additionally, the current diagnosis methods have several demerits, and a new investigation is required to enhance the diagnosis performance. In this paper, a set of phases are performed, such as collecting data, filtering and augmenting images, extracting features, and classifying ECG images. The data were obtained from two publicly available ECG image datasets, and one of them contained COVID ECG reports. A set of preprocessing methods are applied to the ECG images, and data augmentation is performed to balance the ECG images based on the classes. A deep learning approach based on a convolutional neural network (CNN) is performed for feature extraction. Four different pre-trained models are applied, such as Vgg16, Vgg19, ResNet-101, and Xception. Moreover, an ensemble of Xception and the temporary convolutional network (TCN), which is named ECGConvnet, is proposed. Finally, the results obtained from the former models are fed to four main classifiers. These classifiers are softmax, random forest (RF), multilayer perception (MLP), and support vector machine (SVM). The former classifiers are used to evaluate the diagnosis ability of the proposed methods. The classification scenario is based on fivefold cross-validation. Seven experiments are presented to evaluate the performance of the ECGConvnet. Three of them are multi-class, and the remaining are binary class diagnosing. Six out of seven experiments diagnose COVID-19 patients. The aforementioned experimental results indicated that ECGConvnet has the highest performance over other pre-trained models, and the SVM classifier showed higher accuracy in comparison with the other classifiers. The resulting accuracies from ECGConvnet based on SVM are (99.74%, 98.6%, 99.1% on the multi-class diagnosis tasks) and (99.8% on one of the binary-class diagnoses, while the remaining achieved 100%). It is possible to develop an automatic diagnosis system for COVID based on deep learning using ECG data.

CSITime: Privacy-preserving human activity recognition using WiFi channel state information

Human activity recognition (HAR) is an important task in many applications such as smart homes, sports analysis, healthcare services, etc. Popular modalities for human activity recognition involving computer vision and inertial sensors are in the literature for solving HAR, however, they face serious limitations with respect to different illumination, background, clutter, obtrusiveness, and other factors. In recent years, WiFi channel state information (CSI) based activity recognition is gaining momentum due to its many advantages including easy deployability, and cost-effectiveness. This work proposes CSITime, a modified InceptionTime network architecture, a generic architecture for CSI-based human activity recognition. We perceive CSI activity recognition as a multi-variate time series problem. The methodology of CSITime is threefold. First, we pre-process CSI signals followed by data augmentation using two label-mixing strategies - mixup and cutmix to enhance the neural network's learning. Second, in the basic block of CSITime, features from multiple convolutional kernels are concatenated and passed through a self-attention layer followed by a fully connected layer with Mish activation. CSITime network consists of six such blocks followed by a global average pooling layer and a final fully connected layer for the final classification. Third, in the training of the neural network, instead of adopting general training procedures such as early stopping, we use one-cycle policy and cosine annealing to monitor the learning rate. The proposed model has been tested on publicly available benchmark datasets, i.e., ARIL, StanWiFi, and SignFi datasets. The proposed CSITime has achieved accuracy of 98.20%, 98%, and 95.42% on ARIL, StanWiFi, and SignFi datasets, respectively, for WiFi-based activity recognition. This is an improvement on state-of-the-art accuracies by 3.3%, 0.67%, and 0.82% on ARIL, StanWiFi, and SignFi datasets, respectively. In lab-5 users' scenario of the SignFi dataset, which has the training and testing data from different distributions, our model achieved accuracy was 2.17% higher than state-of-the-art, which shows the comparative robustness of our model.

Thermal-based early breast cancer detection using inception V3, inception V4 and modified inception MV4

Breast cancer is one of the most significant causes of death for women around the world. Breast thermography supported by deep convolutional neural networks is expected to contribute significantly to early detection and facilitate treatment at an early stage. The goal of this study is to investigate the behavior of different recent deep learning methods for identifying breast disorders. To evaluate our proposal, we built classifiers based on deep convolutional neural networks modelling inception V3, inception V4, and a modified version of the latter called inception MV4. MV4 was introduced to maintain the computational cost across all layers by making the resultant number of features and the number of pixel positions equal. DMR database was used for these deep learning models in classifying thermal images of healthy and sick patients. A set of epochs 3–30 were used in conjunction with learning rates 1 × 10^–3, 1 × 10^–4 and 1 × 10^–5, Minibatch 10 and different optimization methods. The training results showed that inception V4 and MV4 with color images, a learning rate of 1 × 10^–4, and SGDM optimization method, reached very high accuracy, verified through several experimental repetitions. With grayscale images, inception V3 outperforms V4 and MV4 by a considerable accuracy margin, for any optimization methods. In fact, the inception V3 (grayscale) performance is almost comparable to inception V4 and MV4 (color) performance but only after 20–30 epochs. inception MV4 achieved 7% faster classification response time compared to V4. The use of MV4 model is found to contribute to saving energy consumed and fluidity in arithmetic operations for the graphic processor. The results also indicate that increasing the number of layers may not necessarily be useful in improving the performance.

Always-On Sub-Microwatt Spiking Neural Network Based on Spike-Driven Clock- and Power-Gating for an Ultra-Low-Power Intelligent Device

This paper presents a novel spiking neural network (SNN) classifier architecture for enabling always-on artificial intelligent (AI) functions, such as keyword spotting (KWS) and visual wake-up, in ultra-low-power internet-of-things (IoT) devices. Such always-on hardware tends to dominate the power efficiency of an IoT device and therefore it is paramount to minimize its power dissipation. A key observation is that the input signal to always-on hardware is typically sparse in time. This is a great opportunity that a SNN classifier can leverage because the switching activity and the power consumption of SNN hardware can scale with spike rate. To leverage this scalability, the proposed SNN classifier architecture employs event-driven architecture, especially fine-grained clock generation and gating and fine-grained power gating, to obtain very low static power dissipation. The prototype is fabricated in 65 nm CMOS and occupies an area of 1.99 mm². At 0.52 V supply voltage, it consumes 75 nW at no input activity and less than 300 nW at 100% input activity. It still maintains competitive inference accuracy for KWS and other always-on classification workloads. The prototype achieved a power consumption reduction of over three orders of magnitude compared to the state-of-the-art for SNN hardware and of about 2.3X compared to the state-of-the-art KWS hardware.

Benchmarking Highly Parallel Hardware for Spiking Neural Networks in Robotics

Animal brains still outperform even the most performant machines with significantly lower speed. Nonetheless, impressive progress has been made in robotics in the areas of vision, motion- and path planning in the last decades. Brain-inspired Spiking Neural Networks (SNN) and the parallel hardware necessary to exploit their full potential have promising features for robotic application. Besides the most obvious platform for deploying SNN, brain-inspired neuromorphic hardware, Graphical Processing Units (GPU) are well capable of parallel computing as well. Libraries for generating CUDA-optimized code, like GeNN and affordable embedded systems make them an attractive alternative due to their low price and availability. While a few performance tests exist, there has been a lack of benchmarks targeting robotic applications. We compare the performance of a neural Wavefront algorithm as a representative of use cases in robotics on different hardware suitable for running SNN simulations. The SNN used for this benchmark is modeled in the simulator-independent declarative language PyNN, which allows using the same model for different simulator backends. Our emphasis is the comparison between Nest, running on serial CPU, SpiNNaker, as a representative of neuromorphic hardware, and an implementation in GeNN. Beyond that, we also investigate the differences of GeNN deployed to different hardware. A comparison between the different simulators and hardware is performed with regard to total simulation time, average energy consumption per run, and the length of the resulting path. We hope that the insights gained about performance details of parallel hardware solutions contribute to developing more efficient SNN implementations for robotics.