BiCoSS: Toward Large-Scale Cognition Brain With Multigranular Neuromorphic Architecture

Building a realistic, scalable memory model with independent engrams using a homeostatic mechanism

Memory formation is usually associated with Hebbian learning and synaptic plasticity, which changes the synaptic strengths but omits structural changes. A recent study suggests that structural plasticity can also lead to silent memory engrams, reproducing a conditioned learning paradigm with neuron ensembles. However, this study is limited by its way of synapse formation, enabling the formation of only one memory engram. Overcoming this, our model allows the formation of many engrams simultaneously while retaining high neurophysiological accuracy, e.g., as found in cortical columns. We achieve this by substituting the random synapse formation with the Model of Structural Plasticity. As a homeostatic model, neurons regulate their activity by growing and pruning synaptic elements based on their current activity. Utilizing synapse formation based on the Euclidean distance between the neurons with a scalable algorithm allows us to easily simulate 4 million neurons with 343 memory engrams. These engrams do not interfere with one another by default, yet we can change the simulation parameters to form long-reaching associations. Our model's analysis shows that homeostatic engram formation requires a certain spatiotemporal order of events. It predicts that synaptic pruning precedes and enables synaptic engram formation and that it does not occur as a mere compensatory response to enduring synapse potentiation as in Hebbian plasticity with synaptic scaling. Our model paves the way for simulations addressing further inquiries, ranging from memory chains and hierarchies to complex memory systems comprising areas with different learning mechanisms.

ExaFlexHH: an exascale-ready, flexible multi-FPGA library for biologically plausible brain simulations

Introduction

In-silico simulations are a powerful tool in modern neuroscience for enhancing our understanding of complex brain systems at various physiological levels. To model biologically realistic and detailed systems, an ideal simulation platform must possess: (1) high performance and performance scalability, (2) flexibility, and (3) ease of use for non-technical users. However, most existing platforms and libraries do not meet all three criteria, particularly for complex models such as the Hodgkin-Huxley (HH) model or for complex neuron-connectivity modeling such as gap junctions.

Methods

This work introduces ExaFlexHH, an exascale-ready, flexible library for simulating HH models on multi-FPGA platforms. Utilizing FPGA-based Data-Flow Engines (DFEs) and the dataflow programming paradigm, ExaFlexHH addresses all three requirements. The library is also parameterizable and compliant with NeuroML, a prominent brain-description language in computational neuroscience. We demonstrate the performance scalability of the platform by implementing a highly demanding extended-Hodgkin-Huxley (eHH) model of the Inferior Olive using ExaFlexHH.

Results

Model simulation results show linear scalability for unconnected networks and near-linear scalability for networks with complex synaptic plasticity, with a 1.99 × performance increase using two FPGAs compared to a single FPGA simulation, and 7.96 × when using eight FPGAs in a scalable ring topology. Notably, our results also reveal consistent performance efficiency in GFLOPS per watt, further facilitating exascale-ready computing speeds and pushing the boundaries of future brain-simulation platforms.

Discussion

The ExaFlexHH library shows superior resource efficiency, quantified in FLOPS per hardware resources, benchmarked against other competitive FPGA-based brain simulation implementations.

Investigation on Vision System: Digital FPGA Implementation in Case of Retina Rod Cells

The development of prostheses and treatments for illnesses and recovery has recently been centered on hardware modeling for various delicate biological components, including the nervous system, brain, eyes, and heart. The retina, being the thinnest and deepest layer of the eye, is of particular interest. In this study, we employ the Nyquist-Based Approximation of Retina Rod Cell (NBAoRRC) approach, which has been adapted to utilize Look-Up Tables (LUTs) rather than original functions, to implement rod cells in the retina using cost-effective hardware. In modern mathematical models, numerous nonlinear functions are used to represent the activity of these cells. However, these nonlinear functions would require a substantial amount of hardware for direct implementation and may not meet the required speed constraints. The proposed method eliminates the need for multiplication functions and utilizes a fast, cost-effective rod cell device. Simulation results demonstrate the extent to which the proposed model aligns with the behavior of the primary rod cell model, particularly in terms of dynamic behavior. Based on the results of hardware implementation using the Field-Programmable Gate Arrays (FPGA) board Virtex-5, the proposed model is shown to be reliable, consume 30 percent less power than the primary model, and have reduced hardware resource requirements. Based on the results of hardware implementation using the reconfigurable FPGA board Virtex-5, the proposed model is reliable, uses 30% less power consumption than the primary model in the worth state of the set of approximation method, and has a reduced hardware resource requirement. In fact, using the proposed model, this reduction in the power consumption can be achieved. Finally, in this article, by using the LUT which is systematically sampled (Nyquist rate), we were able to remove all costly operators in terms of hardware (digital) realization and achieve very good results in the field of digital implementation in two scales of network and single neuron.

Efficient digital design of the nonlinear behavior of Hindmarsh-Rose neuron model in large-scale neural population

Spiking networks, as the third generation of neural networks, are of great interest today due to their low power consumption in cognitive processes. This important characteristic has caused the hardware implementation techniques of spiking networks in the form of neuromorphic systems attract a lot of attention. For the first time, the focus is on the digital implementation based on CORDIC approximation of the Hindmarsh-Rose (HR) neuron so that the hardware implementation cost is lower than previous studies. If the digital design of a neuron is done efficient, the possibility of implementing a population of neurons is provided for the feasibility of low-consumption implementation of high-level cognitive processes in hardware, which is considered in this paper through edge detector, noise removal and image magnification spiking networks based on the proposed CORDIC_HR model. While using less hardware resources, the proposed HR neuron model follows the behavior of the original neuron model in the time domain with much less error than previous study. Also, the complex nonlinear behavior of the original and the proposed model of HR neuron through the bifurcation diagram, phase space and nullcline space analysis under different system parameters was investigated and the good follow-up of the proposed model was confirmed from the original model. In addition to the fact that the individual behavior of the original and the proposed neurons is the same, the functional and behavioral performance of the randomly connected neuronal population of original and proposed neuron model is equal. In general, the main contribution of the paper is in presenting an efficient hardware model, which consumes less hardware resources, follows the behavior of the original model with high accuracy, and has an acceptable performance in image processing applications such as noise removal and edge detection.

Artificial Intelligence in Heart Failure: Friend or Foe?

In recent times, there have been notable changes in cardiovascular medicine, propelled by the swift advancements in artificial intelligence (AI). The present work provides an overview of the current applications and challenges of AI in the field of heart failure. It emphasizes the "garbage in, garbage out" issue, where AI systems can produce inaccurate results with skewed data. The discussion covers issues in heart failure diagnostic algorithms, particularly discrepancies between existing models. Concerns about the reliance on the left ventricular ejection fraction (LVEF) for classification and treatment are highlighted, showcasing differences in current scientific perceptions. This review also delves into challenges in implementing AI, including variable considerations and biases in training data. It underscores the limitations of current AI models in real-world scenarios and the difficulty in interpreting their predictions, contributing to limited physician trust in AI-based models. The overarching suggestion is that AI can be a valuable tool in clinicians' hands for treating heart failure patients, as far as existing medical inaccuracies have been addressed before integrating AI into these frameworks.

Exploring spiking neural networks: a comprehensive analysis of mathematical models and applications

This article presents a comprehensive analysis of spiking neural networks (SNNs) and their mathematical models for simulating the behavior of neurons through the generation of spikes. The study explores various models, including LIF and NLIF, for constructing SNNs and investigates their potential applications in different domains. However, implementation poses several challenges, including identifying the most appropriate model for classification tasks that demand high accuracy and low-performance loss. To address this issue, this research study compares the performance, behavior, and spike generation of multiple SNN models using consistent inputs and neurons. The findings of the study provide valuable insights into the benefits and challenges of SNNs and their models, emphasizing the significance of comparing multiple models to identify the most effective one. Moreover, the study quantifies the number of spiking operations required by each model to process the same inputs and produce equivalent outputs, enabling a thorough assessment of computational efficiency. The findings provide valuable insights into the benefits and limitations of SNNs and their models. The research underscores the significance of comparing different models to make informed decisions in practical applications. Additionally, the results reveal essential variations in biological plausibility and computational efficiency among the models, further emphasizing the importance of selecting the most suitable model for a given task. Overall, this study contributes to a deeper understanding of SNNs and offers practical guidelines for using their potential in real-world scenarios.

ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network

Recent developments in artificial neural networks and their learning algorithms have enabled new research directions in computer vision, language modeling, and neuroscience. Among various neural network algorithms, spiking neural networks (SNNs) are well-suited for understanding the behavior of biological neural circuits. In this work, we propose to guide the training of a sparse SNN in order to replace a sub-region of a cultured hippocampal network with limited hardware resources. To verify our approach with a realistic experimental setup, we record spikes of cultured hippocampal neurons with a microelectrode array (in vitro). The main focus of this work is to dynamically cut unimportant synapses during SNN training on the fly so that the model can be realized on resource-constrained hardware, e.g., implantable devices. To do so, we adopt a simple STDP learning rule to easily select important synapses that impact the quality of spike timing learning. By combining the STDP rule with online supervised learning, we can precisely predict the spike pattern of the cultured network in real-time. The reduction in the model complexity, i.e., the reduced number of connections, significantly reduces the required hardware resources, which is crucial in developing an implantable chip for the treatment of neurological disorders. In addition to the new learning algorithm, we prototype a sparse SNN hardware on a small FPGA with pipelined execution and parallel computing to verify the possibility of real-time replacement. As a result, we can replace a sub-region of the biological neural circuit within 22 μs using 2.5 × fewer hardware resources, i.e., by allowing 80% sparsity in the SNN model, compared to the fully-connected SNN model. With energy-efficient algorithms and hardware, this work presents an essential step toward real-time neuroprosthetic computation.

Memristive neural network circuit implementation of associative learning with overshadowing and blocking

In the field of second language acquisition, overshadowing and blocking by cue competition effects in classical conditioning affect the learning and expression of human cognitive associations. In this work, a memristive neural network circuit based on neurobiological mechanisms is proposed, which consists of synapse module, neuron module, and control module. In particular, the designed network introduces an inhibitory interneuron to divide memristive synapses into excitatory and inhibitory memristive synapses, so as to mimic synaptic plasticity better. In addition, the proposed circuit can implement six functions of second language acquisition conditioning, including learning, overshadowing, blocking, recovery from overshadowing, recovery from blocking, and long-term effect of overshadowing over time leading to blocking. Overshadowing, which denotes that the more salient stimulus overshadows the learning of the less salient stimulus when two stimuli differ in salience, reduces the associative strength acquired by the less salient stimulus. Blocking, which indicates that pretraining on one stimulus blocks learning about a second stimulus, inhibits the associative strength acquired by a second stimulus. Finally, the correctness and effectiveness of implementing functions mentioned above are verified by the simulation results in PSPICE. Through further research, the proposed circuit is applied to bionic devices such as social robots or educational robots, which can address language and cognitive disorders via assisted learning and training.

BrainS: Customized multi-core embedded multiple scale neuromorphic system

Research on modeling and mechanisms of the brain remains the most urgent and challenging task. The customized embedded neuromorphic system is one of the most effective approaches for multi-scale simulations ranging from ion channel to network. This paper proposes BrainS, a scalable multi-core embedded neuromorphic system capable of accommodating massive and large-scale simulations. It is designed with rich external extension interfaces to support various types of input/output and communication requirements. The 3D mesh-based topology with an efficient memory access mechanism makes exploring the properties of neuronal networks possible. BrainS operates at 168 MHz and contains a model database ranging from ion channel to network scale within the Fundamental Computing Unit (FCU). At the ion channel scale, the Basic Community Unit (BCU) can perform real-time simulations of a Hodgkin-Huxley (HH) neuron with 16000 ion channels, using 125.54 KB of the SRAM. When the number of ion channels is within 64000, the HH neuron is simulated in real-time by 4 BCUs. At the network scale, the basal ganglia-thalamus (BG-TH) network consisting of 3200 Izhikevich neurons, providing a vital motor regulation function, is simulated in 4 BCUs with a power consumption of 364.8 mW. Overall, BrainS has an excellent performance in real-time and flexible configurability, providing an embedded application solution for multi-scale simulation.

Neuromorphic processor-oriented hybrid Q-format multiplication with adaptive quantization for tiny YOLO3

Deep neural networks (DNNs) have delivered unprecedented achievements in the modern Internet of Everything society, encompassing autonomous driving, expert diagnosis, unmanned supermarkets, etc. It continues to be challenging for researchers and engineers to develop a high-performance neuromorphic processor for deployment in edge devices or embedded hardware. DNNs’ superpower derives from their enormous and complex network architecture, which is computation-intensive, time-consuming, and energy-heavy. Due to the limited perceptual capacity of humans, accurate processing results from DNNs require a substantial amount of computing time, making them redundant in some applications. Utilizing adaptive quantization technology to compress the DNN model with sufficient accuracy is crucial for facilitating the deployment of neuromorphic processors in emerging edge applications. This study proposes a method to boost the development of neuromorphic processors by conducting fixed-point multiplication in a hybrid Q-format using an adaptive quantization technique on the convolution of tiny YOLO3. In particular, this work integrates the sign-bit check and bit roundoff techniques into the arithmetic of fixed-point multiplications to address overflow and roundoff issues within the convolution’s adding and multiplying operations. In addition, a hybrid Q-format multiplication module is developed to assess the proposed method from a hardware perspective. The experimental results prove that the hybrid multiplication with adaptive quantization on the tiny YOLO3’s weights and feature maps possesses a lower error rate than alternative fixed-point representation formats while sustaining the same object detection accuracy. Moreover, the fixed-point numbers represented by Q(6.9) have a suboptimal error rate, which can be utilized as an alternative representation form for the tiny YOLO3 algorithm-based neuromorphic processor design. In addition, the 8-bit hybrid Q-format multiplication module exhibits low power consumption and low latency in contrast to benchmark multipliers.

BitBrain and Sparse Binary Coincidence (SBC) memories: Fast, robust learning and inference for neuromorphic architectures

We present an innovative working mechanism (the SBC memory) and surrounding infrastructure (BitBrain) based upon a novel synthesis of ideas from sparse coding, computational neuroscience and information theory that enables fast and adaptive learning and accurate, robust inference. The mechanism is designed to be implemented efficiently on current and future neuromorphic devices as well as on more conventional CPU and memory architectures. An example implementation on the SpiNNaker neuromorphic platform has been developed and initial results are presented. The SBC memory stores coincidences between features detected in class examples in a training set, and infers the class of a previously unseen test example by identifying the class with which it shares the highest number of feature coincidences. A number of SBC memories may be combined in a BitBrain to increase the diversity of the contributing feature coincidences. The resulting inference mechanism is shown to have excellent classification performance on benchmarks such as MNIST and EMNIST, achieving classification accuracy with single-pass learning approaching that of state-of-the-art deep networks with much larger tuneable parameter spaces and much higher training costs. It can also be made very robust to noise. BitBrain is designed to be very efficient in training and inference on both conventional and neuromorphic architectures. It provides a unique combination of single-pass, single-shot and continuous supervised learning; following a very simple unsupervised phase. Accurate classification inference that is very robust against imperfect inputs has been demonstrated. These contributions make it uniquely well-suited for edge and IoT applications.

Evolving deep convolutional neutral network by hybrid sine-cosine and extreme learning machine for real-time COVID19 diagnosis from X-ray images

The COVID19 pandemic globally and significantly has affected the life and health of many communities. The early detection of infected patients is effective in fighting COVID19. Using radiology (X-Ray) images is, perhaps, the fastest way to diagnose the patients. Thereby, deep Convolutional Neural Networks (CNNs) can be considered as applicable tools to diagnose COVID19 positive cases. Due to the complicated architecture of a deep CNN, its real-time training and testing become a challenging problem. This paper proposes using the Extreme Learning Machine (ELM) instead of the last fully connected layer to address this deficiency. However, the parameters' stochastic tuning of ELM's supervised section causes the final model unreliability. Therefore, to cope with this problem and maintain network reliability, the sine-cosine algorithm was utilized to tune the ELM's parameters. The designed network is then benchmarked on the COVID-Xray-5k dataset, and the results are verified by a comparative study with canonical deep CNN, ELM optimized by cuckoo search, ELM optimized by genetic algorithm, and ELM optimized by whale optimization algorithm. The proposed approach outperforms comparative benchmarks with a final accuracy of 98.83% on the COVID-Xray-5k dataset, leading to a relative error reduction of 2.33% compared to a canonical deep CNN. Even more critical, the designed network's training time is only 0.9421 ms and the overall detection test time for 3100 images is 2.721 s.

Neuromorphic-Based Neuroprostheses for Brain Rewiring: State-of-the-Art and Perspectives in Neuroengineering

Neuroprostheses are neuroengineering devices that have an interface with the nervous system and supplement or substitute functionality in people with disabilities. In the collective imagination, neuroprostheses are mostly used to restore sensory or motor capabilities, but in recent years, new devices directly acting at the brain level have been proposed. In order to design the next-generation of neuroprosthetic devices for brain repair, we foresee the increasing exploitation of closed-loop systems enabled with neuromorphic elements due to their intrinsic energy efficiency, their capability to perform real-time data processing, and of mimicking neurobiological computation for an improved synergy between the technological and biological counterparts. In this manuscript, after providing definitions of key concepts, we reviewed the first exploitation of a real-time hardware neuromorphic prosthesis to restore the bidirectional communication between two neuronal populations in vitro. Starting from that 'case-study', we provide perspectives on the technological improvements for real-time interfacing and processing of neural signals and their potential usage for novel in vitro and in vivo experimental designs. The development of innovative neuroprosthetics for translational purposes is also presented and discussed. In our understanding, the pursuit of neuromorphic-based closed-loop neuroprostheses may spur the development of novel powerful technologies, such as 'brain-prostheses', capable of rewiring and/or substituting the injured nervous system.

Highly efficient neuromorphic learning system of spiking neural network with multi-compartment leaky integrate-and-fire neurons

A spiking neural network (SNN) is considered a high-performance learning system that matches the digital circuits and presents higher efficiency due to the architecture and computation of spiking neurons. While implementing a SNN on a field-programmable gate array (FPGA), the gradient back-propagation through layers consumes a surprising number of resources. In this paper, we aim to realize an efficient architecture of SNN on the FPGA to reduce resource and power consumption. The multi-compartment leaky integrate-and-fire (MLIF) model is used to convert spike trains to the plateau potential in dendrites. We accumulate the potential in the apical dendrite during the training period. The average of this accumulative result is the dendritic plateau potential and is used to guide the updates of synaptic weights. Based on this architecture, the SNN is implemented on FPGA efficiently. In the implementation of a neuromorphic learning system, the shift multiplier (shift MUL) module and piecewise linear (PWL) algorithm are used to replace multipliers and complex nonlinear functions to match the digital circuits. The neuromorphic learning system is constructed with resources on FPGA without dataflow between on-chip and off-chip memories. Our neuromorphic learning system performs with higher resource utilization and power efficiency than previous on-chip learning systems.

Rural Workplace Sustainable Development of Smart Rural Governance Workplace Platform for Efficient Enterprise Performances

In the long developmental process, China's agriculture has transformed from organic agriculture to inorganic agriculture. New technologies have made the modernization of agriculture possible. However, most older people who are engaged in agriculture may not completely understand the modernization of agriculture. Based on the limitations of traditional image target detection methods, a deep learning-based pest target detection and recognition method is proposed from a blockchain perspective, to analyze and research agricultural data supervision and governance and explore the effectiveness of deep learning methods in crop pest detection and recognition. The comparative analysis demonstrates that the average precision (AP) of GA-CPN-LAR (global activation-characteristic pyramid network-local activation region) increases by 4.2% compared with other methods. Whether under the Inception or ResNet-50 backbone networks, the AP of GA-CPN-LAR is significantly better than other methods. Compared with the ResNet-50 backbone network, GA-CPN-LAR has higher accuracy and recall rates under Inception. Precision-recall curve measurement shows that the proposed method can significantly reduce the false detection rate and missed detection rate. The GA-CPN-LAR model proposed here has a higher AP value on the MPD dataset than the other target detection methods, which can be increased by 4.2%. Besides, the accuracy and recall of the GA-CPN-LAR method corresponding to two representative pests under the initial feature extractor are higher than the MPD dataset baseline. In addition, the research results of the MPD dataset and AgriPest dataset also show that the pest target detection method based on convolutional neural networks (CNNs) has a good presentation effect and can significantly reduce false detection and missed detection. Moreover, the pest regulation based on blockchain and deep learning comprehensively considers global and local feature extraction and pattern recognition, which positively impacts the conscientization of agricultural data processing and promotes the sustainable development of rural areas.

Automatic Detection of High-Frequency Oscillations With Neuromorphic Spiking Neural Networks

Interictal high-frequency oscillations (HFO) detected in electroencephalography recordings have been proposed as biomarkers of epileptogenesis, seizure propensity, disease severity, and treatment response. Automatic HFO detectors typically analyze the data offline using complex time-consuming algorithms, which limits their clinical application. Neuromorphic circuits offer the possibility of building compact and low-power processing systems that can analyze data on-line and in real time. In this review, we describe a fully automated detection pipeline for HFO that uses, for the first time, spiking neural networks and neuromorphic technology. We demonstrated that our HFO detection pipeline can be applied to recordings from different modalities (intracranial electroencephalography, electrocorticography, and scalp electroencephalography) and validated its operation in a custom-designed neuromorphic processor. Our HFO detection approach resulted in high accuracy and specificity in the prediction of seizure outcome in patients implanted with intracranial electroencephalography and electrocorticography, and in the prediction of epilepsy severity in patients recorded with scalp electroencephalography. Our research provides a further step toward the real-time detection of HFO using compact and low-power neuromorphic devices. The real-time detection of HFO in the operation room may improve the seizure outcome of epilepsy surgery, while the use of our neuromorphic processor for non-invasive therapy monitoring might allow for more effective medication strategies to achieve seizure control. Therefore, this work has the potential to improve the quality of life in patients with epilepsy by improving epilepsy diagnostics and treatment.

Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

Limited text speech synthesis with electroglottograph based on Bi-LSTM and modified Tacotron-2

This paper proposes a framework of applying only the EGG signal for speech synthesis in the limited categories of contents scenario. EGG is a sort of physiological signal which can reflect the trends of the vocal cord movement. Note that EGG’s different acquisition method contrasted with speech signals, we exploit its application in speech synthesis under the following two scenarios. (1) To synthesize speeches under high noise circumstances, where clean speech signals are unavailable. (2) To enable dumb people who retain vocal cord vibration to speak again. Our study consists of two stages, EGG to text and text to speech. The first is a text content recognition model based on Bi-LSTM, which converts each EGG signal sample into the corresponding text with a limited class of contents. This model achieves 91.12% accuracy on the validation set in a 20-class content recognition experiment. Then the second step synthesizes speeches with the corresponding text and the EGG signal. Based on modified Tacotron-2, our model gains the Mel cepstral distortion (MCD) of 5.877 and the mean opinion score (MOS) of 3.87, which is comparable with the state-of-the-art performance and achieves an improvement by 0.42 and a relatively smaller model size than the origin Tacotron-2. Considering to introduce the characteristics of speakers contained in EGG to the final synthesized speech, we put forward a fine-grained fundamental frequency modification method, which adjusts the fundamental frequency according to EGG signals and achieves a lower MCD of 5.781 and a higher MOS of 3.94 than that without modification.

Periodicity Pitch Perception Part III: Sensibility and Pachinko Volatility

Neuromorphic computer models are used to explain sensory perceptions. Auditory models generate cochleagrams, which resemble the spike distributions in the auditory nerve. Neuron ensembles along the auditory pathway transform sensory inputs step by step and at the end pitch is represented in auditory categorical spaces. In two previous articles in the series on periodicity pitch perception an extended auditory model had been successfully used for explaining periodicity pitch proved for various musical instrument generated tones and sung vowels. In this third part in the series the focus is on octopus cells as they are central sensitivity elements in auditory cognition processes. A powerful numerical model had been devised, in which auditory nerve fibers (ANFs) spike events are the inputs, triggering the impulse responses of the octopus cells. Efficient algorithms are developed and demonstrated to explain the behavior of octopus cells with a focus on a simple event-based hardware implementation of a layer of octopus neurons. The main finding is, that an octopus' cell model in a local receptive field fine-tunes to a specific trajectory by a spike-timing-dependent plasticity (STDP) learning rule with synaptic pre-activation and the dendritic back-propagating signal as post condition. Successful learning explains away the teacher and there is thus no need for a temporally precise control of plasticity that distinguishes between learning and retrieval phases. Pitch learning is cascaded: At first octopus cells respond individually by self-adjustment to specific trajectories in their local receptive fields, then unions of octopus cells are collectively learned for pitch discrimination. Pitch estimation by inter-spike intervals is shown exemplary using two input scenarios: a simple sinus tone and a sung vowel. The model evaluation indicates an improvement in pitch estimation on a fixed time-scale.

Application of Digital Twins in multiple fields

With the development of science and technology, the high-tech industry is developing rapidly, and various new-age technologies continue to appear, and Digital Twins (DT) is one of them. As a brand-new interactive technology, DT technology can handle the interaction between the real world and the virtual world well. It has become a hot spot in the academic circles of all countries in the world. DT have developed rapidly in recent years result from centrality, integrity and dynamics. It is integrated with other technologies and has been applied in many fields, such as smart factory in industrial production, digital model of life in medical field, construction of smart city, security guarantee in aerospace field, immersive shopping in commercial field and so on. The introduction of DT is mostly a summary of concepts, and few practical applications of Digital Twins are introduced. The purpose of this paper is to enable people to understand the application status of DT technology. At the same time, the introduction of core technologies related to DT is interspersed in the application introduction. Finally, combined with the current development status of DT, predict the future development trend of DT and make a summary.

A neuromorphic spiking neural network detects epileptic high frequency oscillations in the scalp EEG

Interictal High Frequency Oscillations (HFO) are measurable in scalp EEG. This development has aroused interest in investigating their potential as biomarkers of epileptogenesis, seizure propensity, disease severity, and treatment response. The demand for therapy monitoring in epilepsy has kindled interest in compact wearable electronic devices for long-term EEG recording. Spiking neural networks (SNN) have emerged as optimal architectures for embedding in compact low-power signal processing hardware. We analyzed 20 scalp EEG recordings from 11 pediatric focal lesional epilepsy patients. We designed a custom SNN to detect events of interest (EoI) in the 80–250 Hz ripple band and reject artifacts in the 500–900 Hz band. We identified the optimal SNN parameters to detect EoI and reject artifacts automatically. The occurrence of HFO thus detected was associated with active epilepsy with 80% accuracy. The HFO rate mirrored the decrease in seizure frequency in 8 patients (p = 0.0047). Overall, the HFO rate correlated with seizure frequency (rho = 0.90 CI [0.75 0.96], p < 0.0001, Spearman’s correlation). The fully automated SNN detected clinically relevant HFO in the scalp EEG. This study is a further step towards non-invasive epilepsy monitoring with a low-power wearable device.

The Unrepeatable Human Mind—Challenges in the Development of Cognitive Information Systems—What Makes a Machine Human?

The understanding of human cognition has not been fully achieved; therefore, Information Systems (IS) are not yet fully synchronized with humans. By understanding the cognition process, we will be able to create a human-tailored Cognitive Information Systems (CISs). The necessity for this research is supported by the fact that present business decision makers are faced with challenges that they cannot solve in the time available without CIS. Here, the aim of the authors is to underpin the adaptability of cognitive resonance and the role of the info-communication via Human–Computer Interaction (HCI)—including linkage, relation and impacts, showing the needed direction to increase the effectiveness of the Human–Computer Interaction (HCI), that leads to an improved CIS building with a higher cognitive level. The applied research methodology consists of research analyses and an assessment of the available publications to pursue a comparative study pattern; then, a model building paradigm has been used for observing and monitoring the work with a CIS during HCI. We found a huge gap regarding information processing in the recent literature, that has been caused by the wide range of interdisciplinarity. Our research approach provides an overview of how other disciplines influence HCI and how the human mental model is supported with value added.

Automated Detection of COVID-19 Using Deep Learning Approaches with Paper-Based ECG Reports

One of the pandemics that have caused many deaths is the Coronavirus disease 2019 (COVID-19). It first appeared in late 2019, and many deaths are increasing day by day until now. Therefore, the early diagnosis of COVID-19 has become a salient issue. Additionally, the current diagnosis methods have several demerits, and a new investigation is required to enhance the diagnosis performance. In this paper, a set of phases are performed, such as collecting data, filtering and augmenting images, extracting features, and classifying ECG images. The data were obtained from two publicly available ECG image datasets, and one of them contained COVID ECG reports. A set of preprocessing methods are applied to the ECG images, and data augmentation is performed to balance the ECG images based on the classes. A deep learning approach based on a convolutional neural network (CNN) is performed for feature extraction. Four different pre-trained models are applied, such as Vgg16, Vgg19, ResNet-101, and Xception. Moreover, an ensemble of Xception and the temporary convolutional network (TCN), which is named ECGConvnet, is proposed. Finally, the results obtained from the former models are fed to four main classifiers. These classifiers are softmax, random forest (RF), multilayer perception (MLP), and support vector machine (SVM). The former classifiers are used to evaluate the diagnosis ability of the proposed methods. The classification scenario is based on fivefold cross-validation. Seven experiments are presented to evaluate the performance of the ECGConvnet. Three of them are multi-class, and the remaining are binary class diagnosing. Six out of seven experiments diagnose COVID-19 patients. The aforementioned experimental results indicated that ECGConvnet has the highest performance over other pre-trained models, and the SVM classifier showed higher accuracy in comparison with the other classifiers. The resulting accuracies from ECGConvnet based on SVM are (99.74%, 98.6%, 99.1% on the multi-class diagnosis tasks) and (99.8% on one of the binary-class diagnoses, while the remaining achieved 100%). It is possible to develop an automatic diagnosis system for COVID based on deep learning using ECG data.

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Artificial neural networks (ANNs), like convolutional neural networks (CNNs), have achieved the state-of-the-art results for many machine learning tasks. However, inference with large-scale full-precision CNNs must cause substantial energy consumption and memory occupation, which seriously hinders their deployment on mobile and embedded systems. Highly inspired from biological brain, spiking neural networks (SNNs) are emerging as new solutions because of natural superiority in brain-like learning and great energy efficiency with event-driven communication and computation. Nevertheless, training a deep SNN remains a main challenge and there is usually a big accuracy gap between ANNs and SNNs. In this paper, we introduce a hardware-friendly conversion algorithm called "scatter-and-gather" to convert quantized ANNs to lossless SNNs, where neurons are connected with ternary {-1,0,1} synaptic weights. Each spiking neuron is stateless and more like original McCulloch and Pitts model, because it fires at most one spike and need be reset at each time step. Furthermore, we develop an incremental mapping framework to demonstrate efficient network deployments on a reconfigurable neuromorphic chip. Experimental results show our spiking LeNet on MNIST and VGG-Net on CIFAR-10 datasetobtain 99.37% and 91.91% classification accuracy, respectively. Besides, the presented mapping algorithm manages network deployment on our neuromorphic chip with maximum resource efficiency and excellent flexibility. Our four-spike LeNet and VGG-Net on chip can achieve respective real-time inference speed of 0.38 ms/image, 3.24 ms/image, and an average power consumption of 0.28 mJ/image and 2.3 mJ/image at 0.9 V, 252 MHz, which is nearly two orders of magnitude more efficient than traditional GPUs.

Reconstruction of a Fully Paralleled Auditory Spiking Neural Network and FPGA Implementation

This paper presents a field-programmable gate array (FPGA) implementation of an auditory system, which is biologically inspired and has the advantages of robustness and anti-noise ability. We propose an FPGA implementation of an eleven-channel hierarchical spiking neuron network (SNN) model, which has a sparsely connected architecture with low power consumption. According to the mechanism of the auditory pathway in human brain, spiking trains generated by the cochlea are analyzed in the hierarchical SNN, and the specific word can be identified by a Bayesian classifier. Modified leaky integrate-and-fire (LIF) model is used to realize the hierarchical SNN, which achieves both high efficiency and low hardware consumption. The hierarchical SNN implemented on FPGA enables the auditory system to be operated at high speed and can be interfaced and applied with external machines and sensors. A set of speech from different speakers mixed with noise are used as input to test the performance our system, and the experimental results show that the system can classify words in a biologically plausible way with the presence of noise. The method of our system is flexible and the system can be modified into desirable scale. These confirm that the proposed biologically plausible auditory system provides a better method for on-chip speech recognition. Compare to the state-of-the-art, our auditory system achieves a higher speed with a maximum frequency of 65.03 MHz and a lower energy consumption of 276.83 μJ for a single operation. It can be applied in the field of brain-computer interface and intelligent robots.

The Role of Deep Learning-Based Echocardiography in the Diagnosis and Evaluation of the Effects of Routine Anti-Heart-Failure Western Medicines in Elderly Patients with Acute Left Heart Failure

Objective

The role of deep learning-based echocardiography in the diagnosis and evaluation of the effects of routine anti-heart-failure Western medicines was investigated in elderly patients with acute left heart failure (ALHF).

Methods

A total of 80 elderly patients with ALHF admitted to Affiliated Hangzhou First People's Hospital from August 2017 to February 2019 were selected as the research objects, and they were divided randomly into a control group and an observation group, with 40 cases in each group. Then, a deep convolutional neural network (DCNN) algorithm model was established, and image preprocessing was carried out. The binarized threshold segmentation was used for denoising, and the image was for illumination processing to balance the overall brightness of the image and increase the usable data of the model, so as to reduce the interference of subsequent feature extraction. Finally, the detailed module of deep convolutional layer network algorithm was realized. Besides, the patients from the control group were given routine echocardiography, and the observation group underwent echocardiography based on deep learning algorithm. Moreover, the hospitalization status of patients from the two groups was observed and recorded, including mortality rate, rehospitalization rate, average length of hospitalization, and hospitalization expenses. The diagnostic accuracy of the two examination methods was compared, and the electrocardiogram (ECG) and echocardiographic parameters as well as patients' quality of life were recorded in both groups at the basic state and 5 months after drug treatment.

Results

After comparison, the rehospitalization rate and mortality rate of the observation group were lower than the rates of the control group, but the diagnostic accuracy was higher than that of the control group. However, the difference between the two groups of patients was not statistically marked (P > 0.05). The length and expenses of hospitalization of the observation group were both less than those of the control group. The specificity, sensitivity, and accuracy of the examination methods in the observation group were higher than those of the control group, and the differences were statistically marked (P < 0.05). There was a statistically great difference between the interventricular delay (IVD) of the echocardiographic parameters of patients from the two groups at the basic state and the left ventricular electromechanical delay (LVEMD) parameter values after 5 months of treatment (P < 0.05), but there was no significant difference in the other parameters. After treatment, the quality of life of patients from the two groups was improved, while the observation group was more marked than the control group (P < 0.05).

Conclusion

Echocardiography based on deep learning algorithm had high diagnostic accuracy and could reduce the possibility of cardiovascular events in patients with heart failure, so as to decrease the mortality rate and diagnosis and treatment costs. Moreover, it had an obvious diagnostic effect, which was conducive to the timely detection and treatment of clinical diseases.

Thermal-based early breast cancer detection using inception V3, inception V4 and modified inception MV4

Breast cancer is one of the most significant causes of death for women around the world. Breast thermography supported by deep convolutional neural networks is expected to contribute significantly to early detection and facilitate treatment at an early stage. The goal of this study is to investigate the behavior of different recent deep learning methods for identifying breast disorders. To evaluate our proposal, we built classifiers based on deep convolutional neural networks modelling inception V3, inception V4, and a modified version of the latter called inception MV4. MV4 was introduced to maintain the computational cost across all layers by making the resultant number of features and the number of pixel positions equal. DMR database was used for these deep learning models in classifying thermal images of healthy and sick patients. A set of epochs 3–30 were used in conjunction with learning rates 1 × 10^–3, 1 × 10^–4 and 1 × 10^–5, Minibatch 10 and different optimization methods. The training results showed that inception V4 and MV4 with color images, a learning rate of 1 × 10^–4, and SGDM optimization method, reached very high accuracy, verified through several experimental repetitions. With grayscale images, inception V3 outperforms V4 and MV4 by a considerable accuracy margin, for any optimization methods. In fact, the inception V3 (grayscale) performance is almost comparable to inception V4 and MV4 (color) performance but only after 20–30 epochs. inception MV4 achieved 7% faster classification response time compared to V4. The use of MV4 model is found to contribute to saving energy consumed and fluidity in arithmetic operations for the graphic processor. The results also indicate that increasing the number of layers may not necessarily be useful in improving the performance.

Always-On Sub-Microwatt Spiking Neural Network Based on Spike-Driven Clock- and Power-Gating for an Ultra-Low-Power Intelligent Device

This paper presents a novel spiking neural network (SNN) classifier architecture for enabling always-on artificial intelligent (AI) functions, such as keyword spotting (KWS) and visual wake-up, in ultra-low-power internet-of-things (IoT) devices. Such always-on hardware tends to dominate the power efficiency of an IoT device and therefore it is paramount to minimize its power dissipation. A key observation is that the input signal to always-on hardware is typically sparse in time. This is a great opportunity that a SNN classifier can leverage because the switching activity and the power consumption of SNN hardware can scale with spike rate. To leverage this scalability, the proposed SNN classifier architecture employs event-driven architecture, especially fine-grained clock generation and gating and fine-grained power gating, to obtain very low static power dissipation. The prototype is fabricated in 65 nm CMOS and occupies an area of 1.99 mm². At 0.52 V supply voltage, it consumes 75 nW at no input activity and less than 300 nW at 100% input activity. It still maintains competitive inference accuracy for KWS and other always-on classification workloads. The prototype achieved a power consumption reduction of over three orders of magnitude compared to the state-of-the-art for SNN hardware and of about 2.3X compared to the state-of-the-art KWS hardware.

Benchmarking Highly Parallel Hardware for Spiking Neural Networks in Robotics

Animal brains still outperform even the most performant machines with significantly lower speed. Nonetheless, impressive progress has been made in robotics in the areas of vision, motion- and path planning in the last decades. Brain-inspired Spiking Neural Networks (SNN) and the parallel hardware necessary to exploit their full potential have promising features for robotic application. Besides the most obvious platform for deploying SNN, brain-inspired neuromorphic hardware, Graphical Processing Units (GPU) are well capable of parallel computing as well. Libraries for generating CUDA-optimized code, like GeNN and affordable embedded systems make them an attractive alternative due to their low price and availability. While a few performance tests exist, there has been a lack of benchmarks targeting robotic applications. We compare the performance of a neural Wavefront algorithm as a representative of use cases in robotics on different hardware suitable for running SNN simulations. The SNN used for this benchmark is modeled in the simulator-independent declarative language PyNN, which allows using the same model for different simulator backends. Our emphasis is the comparison between Nest, running on serial CPU, SpiNNaker, as a representative of neuromorphic hardware, and an implementation in GeNN. Beyond that, we also investigate the differences of GeNN deployed to different hardware. A comparison between the different simulators and hardware is performed with regard to total simulation time, average energy consumption per run, and the length of the resulting path. We hope that the insights gained about performance details of parallel hardware solutions contribute to developing more efficient SNN implementations for robotics.

An electronic neuromorphic system for real-time detection of high frequency oscillations (HFO) in intracranial EEG

The analysis of biomedical signals for clinical studies and therapeutic applications can benefit from embedded devices that can process these signals locally and in real-time. An example is the analysis of intracranial EEG (iEEG) from epilepsy patients for the detection of High Frequency Oscillations (HFO), which are a biomarker for epileptogenic brain tissue. Mixed-signal neuromorphic circuits offer the possibility of building compact and low-power neural network processing systems that can analyze data on-line in real-time. Here we present a neuromorphic system that combines a neural recording headstage with a spiking neural network (SNN) processing core on the same die for processing iEEG, and show how it can reliably detect HFO, thereby achieving state-of-the-art accuracy, sensitivity, and specificity. This is a first feasibility study towards identifying relevant features in iEEG in real-time using mixed-signal neuromorphic computing technologies.

Real‑time COVID-19 diagnosis from X-Ray images using deep CNN and extreme learning machines stabilized by chimp optimization algorithm

Real-time detection of COVID-19 using radiological images has gained priority due to the increasing demand for fast diagnosis of COVID-19 cases. This paper introduces a novel two-phase approach for classifying chest X-ray images. Deep Learning (DL) methods fail to cover these aspects since training and fine-tuning the model's parameters consume much time. In this approach, the first phase comes to train a deep CNN working as a feature extractor, and the second phase comes to use Extreme Learning Machines (ELMs) for real-time detection. The main drawback of ELMs is to meet the need of a large number of hidden-layer nodes to gain a reliable and accurate detector in applying image processing since the detective performance remarkably depends on the setting of initial weights and biases. Therefore, this paper uses Chimp Optimization Algorithm (ChOA) to improve results and increase the reliability of the network while maintaining real-time capability. The designed detector is to be benchmarked on the COVID-Xray-5k and COVIDetectioNet datasets, and the results are verified by comparing it with the classic DCNN, Genetic Algorithm optimized ELM (GA-ELM), Cuckoo Search optimized ELM (CS-ELM), and Whale Optimization Algorithm optimized ELM (WOA-ELM). The proposed approach outperforms other comparative benchmarks with 98.25 % and 99.11 % as ultimate accuracy on the COVID-Xray-5k and COVIDetectioNet datasets, respectively, and it led relative error to reduce as the amount of 1.75 % and 1.01 % as compared to a convolutional CNN. More importantly, the time needed for training deep ChOA-ELM is only 0.9474 milliseconds, and the overall testing time for 3100 images is 2.937 s.

Understanding the role of big data analytics for coordination of electronic retail service supply chain

Purpose

Based on the typical service supply chain (SSC) structure, the authors construct the model of e-tailing SSC to explore the coordination relationship in the supply chain, and big data analysis provides realistic possibilities for the creation of coordination mechanisms.

Design/methodology/approach

At the present stage, the e-commerce companies have not yet established a mature SSC system and have not achieved good synergy with other members of the supply chain, the shortage of goods and the greater pressure of express logistics companies coexist. In the case of uncertain online shopping market demand, the authors employ newsboy model, applied in the operations research, to analyze the synergistic mechanism of SSC model.

Findings

By analyzing the e-tailing SSC coordination mechanism and adjusting relevant parameters, the authors find that the synergy mechanism can be implemented and optimized. Through numerical example analysis, the authors confirmed the feasibility of the above analysis.

Originality/value

Big data analysis provides a kind of reality for the establishment of online SSC coordination mechanism. The establishment of an online supply chain coordination mechanism can effectively promote the efficient allocation of supplies and better meet consumers' needs.

Efficient Spike-Driven Learning With Dendritic Event-Based Processing

A critical challenge in neuromorphic computing is to present computationally efficient algorithms of learning. When implementing gradient-based learning, error information must be routed through the network, such that each neuron knows its contribution to output, and thus how to adjust its weight. This is known as the credit assignment problem. Exactly implementing a solution like backpropagation involves weight sharing, which requires additional bandwidth and computations in a neuromorphic system. Instead, models of learning from neuroscience can provide inspiration for how to communicate error information efficiently, without weight sharing. Here we present a novel dendritic event-based processing (DEP) algorithm, using a two-compartment leaky integrate-and-fire neuron with partially segregated dendrites that effectively solves the credit assignment problem. In order to optimize the proposed algorithm, a dynamic fixed-point representation method and piecewise linear approximation approach are presented, while the synaptic events are binarized during learning. The presented optimization makes the proposed DEP algorithm very suitable for implementation in digital or mixed-signal neuromorphic hardware. The experimental results show that spiking representations can rapidly learn, achieving high performance by using the proposed DEP algorithm. We find the learning capability is affected by the degree of dendritic segregation, and the form of synaptic feedback connections. This study provides a bridge between the biological learning and neuromorphic learning, and is meaningful for the real-time applications in the field of artificial intelligence.