Supervised learning in spiking neural networks: A review of algorithms and evaluations

NIR sensors combined with chemometric algorithms in intelligent quality evaluation of sweetpotato roots from 'Farm' to 'Table': Progresses, challenges, trends, and prospects

NIR sensors, in conjunction with advanced chemometric algorithms, have proven to be a powerful and efficient tool for intelligent quality evaluation of sweetpotato roots throughout the entire supply chain. By leveraging NIR data in different wavelength ranges, the physicochemical, nutritional and antioxidant compositions, as well as variety classification of sweetpotato roots during the different stages were adequately evaluated, and all findings involving quantitative and qualitative investigations from the beginning to the present were summarized and analyzed comprehensively. All chemometric algorithms including both linear and nonlinear employed in NIR analysis of sweetpotato roots were introduced in detail and their calibration performances in terms of regression and classification were assessed and discussed. The challenges and limitations of current NIR application in quality evaluation of sweetpotato roots are emphasized. The prospects and trends covering the ongoing advancements in software and hardware are suggested to support the sustainable and efficient sweetpotato processing and utilization.

A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

Spiking Neural Networks (SNNs) have become one of the most prominent next-generation computational models owing to their biological plausibility, low power consumption, and the potential for neuromorphic hardware implementation. Among the various methods for obtaining available SNNs, converting Artificial Neural Networks (ANNs) into SNNs is the most cost-effective approach. The early challenges in ANN-to-SNN conversion work revolved around the susceptibility of converted SNNs to conversion errors. Some recent endeavors have attempted to mitigate these conversion errors by altering the original ANNs. Despite their ability to enhance the accuracy of SNNs, these methods lack generality and cannot be directly applied to convert the majority of existing ANNs. In this paper, we present a framework named DNISNM for converting ANN to SNN, with the aim of addressing conversion errors arising from differences in the discreteness and asynchrony of network transmission between ANN and SNN. The DNISNM consists of two mechanisms, Data-based Neuronal Initialization (DNI) and Signed Neuron with Memory (SNM), designed to respectively address errors stemming from discreteness and asynchrony disparities. This framework requires no additional modifications to the original ANN and can result in SNNs with improved accuracy performance, simultaneously ensuring universality, high precision, and low inference latency. We verify it experimentally on challenging object recognition datasets, including CIFAR10, CIFAR100, and ImageNet-1k. Experimental results show that the SNN converted by our framework has very high accuracy even at extremely low latency.

Methodology based on spiking neural networks for univariate time-series forecasting

Spiking Neural Networks (SNN) are recognised as well-suited for processing spatiotemporal information with ultra-low energy consumption. However, proposals based on SNN for classification tasks are more common than for forecasting problems. In this sense, this paper presents a new general training methodology for univariate time-series forecasting based on SNN. The methodology is focused on one-step ahead forecasting problems and combines a PulseWidth Modulation based encoding-decoding algorithm with a Surrogate Gradient method as supervised training algorithm. In order to validate the generality of the presented methodology sine-wave, 3 UCI and 1 available real-world datasets are used. The results show very satisfactory forecasting results (MAE∈[0.0094,0.2891]) regardless of the characteristics of the dataset or the application field. In addition, weights can be initialised just once to achieve robust results, boosting the advantages of computational and energy cost of SNN.

Performance investigation of an active free-piston Stirling engine using artificial neural network and firefly optimization algorithm

The aim of this study is to explore the characteristics of an active Free-Piston Stirling Engine (AFPSE) through the use of machine learning methods. Due to the time-intensive nature of extracting simulation results from complex thermal equations, an Artificial Neural Network (ANN) is utilized to expedite the process. To construct a nonlinear model, 5000 samples are extracted from simulation results. Input parameters included in the model are the hot and cold source temperatures, the voltage given to the DC motor, spring stiffness, and the mass of the power piston, while output parameters are the amplitude and frequency of power piston displacement. The proposed ANN model structure comprises two hidden layer with 10 and 20 neurons, respectively, indicating the applicability of the ANN model in estimating significant parameters of AFPSE in a shorter amount of time. The firefly optimization algorithm is utilized to determine the unknown input parameters of ANN and maximize the output power. Results indicate that a maximum output power of 23.07 W can be attained by applying 8.5 V voltage on the DC motor. This study highlights the potential of machine learning techniques to explore the primary features of AFPSE.

SHIP: a computational framework for simulating and validating novel technologies in hardware spiking neural networks

Investigations in the field of spiking neural networks (SNNs) encompass diverse, yet overlapping, scientific disciplines. Examples range from purely neuroscientific investigations, researches on computational aspects of neuroscience, or applicative-oriented studies aiming to improve SNNs performance or to develop artificial hardware counterparts. However, the simulation of SNNs is a complex task that can not be adequately addressed with a single platform applicable to all scenarios. The optimization of a simulation environment to meet specific metrics often entails compromises in other aspects. This computational challenge has led to an apparent dichotomy of approaches, with model-driven algorithms dedicated to the detailed simulation of biological networks, and data-driven algorithms designed for efficient processing of large input datasets. Nevertheless, material scientists, device physicists, and neuromorphic engineers who develop new technologies for spiking neuromorphic hardware solutions would find benefit in a simulation environment that borrows aspects from both approaches, thus facilitating modeling, analysis, and training of prospective SNN systems. This manuscript explores the numerical challenges deriving from the simulation of spiking neural networks, and introduces SHIP, Spiking (neural network) Hardware In PyTorch, a numerical tool that supports the investigation and/or validation of materials, devices, small circuit blocks within SNN architectures. SHIP facilitates the algorithmic definition of the models for the components of a network, the monitoring of states and output of the modeled systems, and the training of the synaptic weights of the network, by way of user-defined unsupervised learning rules or supervised training techniques derived from conventional machine learning. SHIP offers a valuable tool for researchers and developers in the field of hardware-based spiking neural networks, enabling efficient simulation and validation of novel technologies.

Machine learning in Alzheimer's disease drug discovery and target identification

Alzheimer's disease (AD) stands as a formidable neurodegenerative ailment that poses a substantial threat to the elderly population, with no known curative or disease-slowing drugs in existence. Among the vital and time-consuming stages in the drug discovery process, disease modeling and target identification hold particular significance. Disease modeling allows for a deeper comprehension of disease progression mechanisms and potential therapeutic avenues. On the other hand, target identification serves as the foundational step in drug development, exerting a profound influence on all subsequent phases and ultimately determining the success rate of drug development endeavors. Machine learning (ML) techniques have ushered in transformative breakthroughs in the realm of target discovery. Leveraging the strengths of large dataset analysis, multifaceted data processing, and the exploration of intricate biological mechanisms, ML has become instrumental in the quest for effective AD treatments. In this comprehensive review, we offer an account of how ML methodologies are being deployed in the pursuit of drug discovery for AD. Furthermore, we provide an overview of the utilization of ML in uncovering potential intervention strategies and prospective therapeutic targets for AD. Finally, we discuss the principal challenges and limitations currently faced by these approaches. We also explore the avenues for future research that hold promise in addressing these challenges.

First-spike coding promotes accurate and efficient spiking neural networks for discrete events with rich temporal structures

Spiking neural networks (SNNs) are well-suited to process asynchronous event-based data. Most of the existing SNNs use rate-coding schemes that focus on firing rate (FR), and so they generally ignore the spike timing in events. On the contrary, methods based on temporal coding, particularly time-to-first-spike (TTFS) coding, can be accurate and efficient but they are difficult to train. Currently, there is limited research on applying TTFS coding to real events, since traditional TTFS-based methods impose one-spike constraint, which is not realistic for event-based data. In this study, we present a novel decision-making strategy based on first-spike (FS) coding that encodes FS timings of the output neurons to investigate the role of the first-spike timing in classifying real-world event sequences with complex temporal structures. To achieve FS coding, we propose a novel surrogate gradient learning method for discrete spike trains. In the forward pass, output spikes are encoded into discrete times to generate FS times. In the backpropagation, we develop an error assignment method that propagates error from FS times to spikes through a Gaussian window, and then supervised learning for spikes is implemented through a surrogate gradient approach. Additional strategies are introduced to facilitate the training of FS timings, such as adding empty sequences and employing different parameters for different layers. We make a comprehensive comparison between FS and FR coding in the experiments. Our results show that FS coding achieves comparable accuracy to FR coding while leading to superior energy efficiency and distinct neuronal dynamics on data sequences with very rich temporal structures. Additionally, a longer time delay in the first spike leads to higher accuracy, indicating important information is encoded in the timing of the first spike.

An Integrated Multi-Omics and Artificial Intelligence Framework for Advance Plant Phenotyping in Horticulture

This review discusses the transformative potential of integrating multi-omics data and artificial intelligence (AI) in advancing horticultural research, specifically plant phenotyping. The traditional methods of plant phenotyping, while valuable, are limited in their ability to capture the complexity of plant biology. The advent of (meta-)genomics, (meta-)transcriptomics, proteomics, and metabolomics has provided an opportunity for a more comprehensive analysis. AI and machine learning (ML) techniques can effectively handle the complexity and volume of multi-omics data, providing meaningful interpretations and predictions. Reflecting the multidisciplinary nature of this area of research, in this review, readers will find a collection of state-of-the-art solutions that are key to the integration of multi-omics data and AI for phenotyping experiments in horticulture, including experimental design considerations with several technical and non-technical challenges, which are discussed along with potential solutions. The future prospects of this integration include precision horticulture, predictive breeding, improved disease and stress response management, sustainable crop management, and exploration of plant biodiversity. The integration of multi-omics and AI holds immense promise for revolutionizing horticultural research and applications, heralding a new era in plant phenotyping.

Complex Oxides for Brain-Inspired Computing: A Review

The fields of brain-inspired computing, robotics, and, more broadly, artificial intelligence (AI) seek to implement knowledge gleaned from the natural world into human-designed electronics and machines. In this review, the opportunities presented by complex oxides, a class of electronic ceramic materials whose properties can be elegantly tuned by doping, electron interactions, and a variety of external stimuli near room temperature, are discussed. The review begins with a discussion of natural intelligence at the elementary level in the nervous system, followed by collective intelligence and learning at the animal colony level mediated by social interactions. An important aspect highlighted is the vast spatial and temporal scales involved in learning and memory. The focus then turns to collective phenomena, such as metal-to-insulator transitions (MITs), ferroelectricity, and related examples, to highlight recent demonstrations of artificial neurons, synapses, and circuits and their learning. First-principles theoretical treatments of the electronic structure, and in situ synchrotron spectroscopy of operating devices are then discussed. The implementation of the experimental characteristics into neural networks and algorithm design is then revewed. Finally, outstanding materials challenges that require a microscopic understanding of the physical mechanisms, which will be essential for advancing the frontiers of neuromorphic computing, are highlighted.

Applications, functions, and accuracy of artificial intelligence in restorative dentistry: A literature review

OBJECTIVE

The applications of artificial intelligence (AI) are increasing in restorative dentistry; however, the AI performance is unclear for dental professionals. The purpose of this narrative review was to evaluate the applications, functions, and accuracy of AI in diverse aspects of restorative dentistry including caries detection, tooth preparation margin detection, tooth restoration design, metal structure casting, dental restoration/implant detection, removable partial denture design, and tooth shade determination.

OVERVIEW

An electronic search was performed on Medline/PubMed, Embase, Web of Science, Cochrane, Scopus, and Google Scholar databases. English-language articles, published from January 1, 2000, to March 1, 2022, relevant to the aforementioned aspects were selected using the key terms of artificial intelligence, machine learning, deep learning, artificial neural networks, convolutional neural networks, clustering, soft computing, automated planning, computational learning, computer vision, and automated reasoning as inclusion criteria. A manual search was also performed. Therefore, 157 articles were included, reviewed, and discussed.

CONCLUSIONS

Based on the current literature, the AI models have shown promising performance in the mentioned aspects when being compared with traditional approaches in terms of accuracy; however, as these models are still in development, more studies are required to validate their accuracy and apply them to routine clinical practice.

CLINICAL SIGNIFICANCE

AI with its specific functions has shown successful applications with acceptable accuracy in diverse aspects of restorative dentistry. The understanding of these functions may lead to novel applications with optimal accuracy for AI in restorative dentistry.

Exploring spiking neural networks: a comprehensive analysis of mathematical models and applications

This article presents a comprehensive analysis of spiking neural networks (SNNs) and their mathematical models for simulating the behavior of neurons through the generation of spikes. The study explores various models, including LIF and NLIF, for constructing SNNs and investigates their potential applications in different domains. However, implementation poses several challenges, including identifying the most appropriate model for classification tasks that demand high accuracy and low-performance loss. To address this issue, this research study compares the performance, behavior, and spike generation of multiple SNN models using consistent inputs and neurons. The findings of the study provide valuable insights into the benefits and challenges of SNNs and their models, emphasizing the significance of comparing multiple models to identify the most effective one. Moreover, the study quantifies the number of spiking operations required by each model to process the same inputs and produce equivalent outputs, enabling a thorough assessment of computational efficiency. The findings provide valuable insights into the benefits and limitations of SNNs and their models. The research underscores the significance of comparing different models to make informed decisions in practical applications. Additionally, the results reveal essential variations in biological plausibility and computational efficiency among the models, further emphasizing the importance of selecting the most suitable model for a given task. Overall, this study contributes to a deeper understanding of SNNs and offers practical guidelines for using their potential in real-world scenarios.

ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network

Recent developments in artificial neural networks and their learning algorithms have enabled new research directions in computer vision, language modeling, and neuroscience. Among various neural network algorithms, spiking neural networks (SNNs) are well-suited for understanding the behavior of biological neural circuits. In this work, we propose to guide the training of a sparse SNN in order to replace a sub-region of a cultured hippocampal network with limited hardware resources. To verify our approach with a realistic experimental setup, we record spikes of cultured hippocampal neurons with a microelectrode array (in vitro). The main focus of this work is to dynamically cut unimportant synapses during SNN training on the fly so that the model can be realized on resource-constrained hardware, e.g., implantable devices. To do so, we adopt a simple STDP learning rule to easily select important synapses that impact the quality of spike timing learning. By combining the STDP rule with online supervised learning, we can precisely predict the spike pattern of the cultured network in real-time. The reduction in the model complexity, i.e., the reduced number of connections, significantly reduces the required hardware resources, which is crucial in developing an implantable chip for the treatment of neurological disorders. In addition to the new learning algorithm, we prototype a sparse SNN hardware on a small FPGA with pipelined execution and parallel computing to verify the possibility of real-time replacement. As a result, we can replace a sub-region of the biological neural circuit within 22 μs using 2.5 × fewer hardware resources, i.e., by allowing 80% sparsity in the SNN model, compared to the fully-connected SNN model. With energy-efficient algorithms and hardware, this work presents an essential step toward real-time neuroprosthetic computation.

Incorporating structural plasticity into self-organization recurrent networks for sequence learning

Introduction

Spiking neural networks (SNNs), inspired by biological neural networks, have received a surge of interest due to its temporal encoding. Biological neural networks are driven by multiple plasticities, including spike timing-dependent plasticity (STDP), structural plasticity, and homeostatic plasticity, making network connection patterns and weights to change continuously during the lifecycle. However, it is unclear how these plasticities interact to shape neural networks and affect neural signal processing.

Method

Here, we propose a reward-modulated self-organization recurrent network with structural plasticity (RSRN-SP) to investigate this issue. Specifically, RSRN-SP uses spikes to encode information, and incorporate multiple plasticities including reward-modulated spike timing-dependent plasticity (R-STDP), homeostatic plasticity, and structural plasticity. On the one hand, combined with homeostatic plasticity, R-STDP is presented to guide the updating of synaptic weights. On the other hand, structural plasticity is utilized to simulate the growth and pruning of synaptic connections.

Results and discussion

Extensive experiments for sequential learning tasks are conducted to demonstrate the representational ability of the RSRN-SP, including counting task, motion prediction, and motion generation. Furthermore, the simulations also indicate that the characteristics arose from the RSRN-SP are consistent with biological observations.

A regularization perspective based theoretical analysis for adversarial robustness of deep spiking neural networks

Spiking Neural Network (SNN) has been recognized as the third generation of neural networks. Conventionally, a SNN can be converted from a pre-trained Artificial Neural Network (ANN) with less computation and memory than training from scratch. But, these converted SNNs are vulnerable to adversarial attacks. Numerical experiments demonstrate that the SNN trained by optimizing the loss function will be more adversarial robust, but the theoretical analysis for the mechanism of robustness is lacking. In this paper, we provide a theoretical explanation by analyzing the expected risk function. Starting by modeling the stochastic process introduced by the Poisson encoder, we prove that there is a positive semidefinite regularizer. Perhaps surprisingly, this regularizer can make the gradients of the output with respect to input closer to zero, thus resulting in inherent robustness against adversarial attacks. Extensive experiments on the CIFAR10 and CIFAR100 datasets support our point of view. For example, we find that the sum of squares of the gradients of the converted SNNs is 13∼160 times that of the trained SNNs. And, the smaller the sum of the squares of the gradients, the smaller the degradation of accuracy under adversarial attack.

All-optical spiking neural network and optical spike-time-dependent plasticity based on the self-pulsing effect within a micro-ring resonator

In this paper, we proposed an all-optical version of photonic spiking neurons and spike-time-dependent plasticity (STDP) based on the nonlinear optical effects within a micro-ring resonator. In this system, the self-pulsing effect was exploited to implement threshold control, and the equivalent pulse energy required for spiking, calculated by multiplying the input pulse power amplitude with its duration, was about 14.1 pJ. The positive performance of the neurons in the excitability and cascadability tests validated the feasibility of this scheme. Furthermore, two simulations were performed to demonstrate that such an all-optical spiking neural network incorporated with STDP could run stably on a stochastic topology. The essence of such an all-optical spiking neural network is a nonlinear spiking dynamical system that combines the advantages of photonics and spiking neural networks (SNNs), promising access to the high speed and lower consumption inherent to optical systems.

Direct learning-based deep spiking neural networks: a review

The spiking neural network (SNN), as a promising brain-inspired computational model with binary spike information transmission mechanism, rich spatially-temporal dynamics, and event-driven characteristics, has received extensive attention. However, its intricately discontinuous spike mechanism brings difficulty to the optimization of the deep SNN. Since the surrogate gradient method can greatly mitigate the optimization difficulty and shows great potential in directly training deep SNNs, a variety of direct learning-based deep SNN works have been proposed and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these direct learning-based deep SNN works, mainly categorized into accuracy improvement methods, efficiency improvement methods, and temporal dynamics utilization methods. In addition, we also divide these categorizations into finer granularities further to better organize and introduce them. Finally, the challenges and trends that may be faced in future research are prospected.

Multitask computation through dynamics in recurrent spiking neural networks

In this work, inspired by cognitive neuroscience experiments, we propose recurrent spiking neural networks trained to perform multiple target tasks. These models are designed by considering neurocognitive activity as computational processes through dynamics. Trained by input-output examples, these spiking neural networks are reverse engineered to find the dynamic mechanisms that are fundamental to their performance. We show that considering multitasking and spiking within one system provides insightful ideas on the principles of neural computation.

Origin of the efficiency of spike timing-based neural computation for processing temporal information

Although the advantage of spike timing-based over rate-based network computation has been recognized, the underlying mechanism remains unclear. Using Tempotron and Perceptron as elementary neural models, we examined the intrinsic difference between spike timing-based and rate-based computations. For more direct comparison, we modified Tempotron computation into rate-based computation with the retention of some temporal information. Previous studies have shown that spike timing-based computation are computationally more powerful than rate-based computation in terms of the number of computational units required and the capability in classifying random patterns. Our study showed that spike timing-based and rate-based Tempotron computations provided similar capability in classifying random spike patterns, as well as in text sentiment classification and spam text detection. However, spike timing-based computation is superior in performing a task involving discriminating forward vs. reverse sequence of events, i.e., information mainly temporal in nature. Further studies revealed that this superiority required the asymmetry in the profile of the postsynaptic potential (PSP), and that temporal sequence information was converted to biased spatial distribution of synaptic weight modifications during learning. Thus, the intrinsic PSP asymmetry is a mechanistic basis for the high efficiency of spike timing-based computation for processing temporal information.

Single dendritic neural classification with an effective spherical search-based whale learning algorithm

McCulloch-Pitts neuron-based neural networks have been the mainstream deep learning methods, achieving breakthrough in various real-world applications. However, McCulloch-Pitts neuron is also under longtime criticism of being overly simplistic. To alleviate this issue, the dendritic neuron model (DNM), which employs non-linear information processing capabilities of dendrites, has been widely used for prediction and classification tasks. In this study, we innovatively propose a hybrid approach to co-evolve DNM in contrast to back propagation (BP) techniques, which are sensitive to initial circumstances and readily fall into local minima. The whale optimization algorithm is improved by spherical search learning to perform co-evolution through dynamic hybridizing. Eleven classification datasets were selected from the well-known UCI Machine Learning Repository. Its efficiency in our model was verified by statistical analysis of convergence speed and Wilcoxon sign-rank tests, with receiver operating characteristic curves and the calculation of area under the curve. In terms of classification accuracy, the proposed co-evolution method beats 10 existing cutting-edge non-BP methods and BP, suggesting that well-learned DNMs are computationally significantly more potent than conventional McCulloch-Pitts types and can be employed as the building blocks for the next-generation deep learning methods.

S(3)NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks

As the scales of neural networks increase, techniques that enable them to run with low computational cost and energy efficiency are required. From such demands, various efficient neural network paradigms, such as spiking neural networks (SNNs) or binary neural networks (BNNs), have been proposed. However, they have sticky drawbacks, such as degraded inference accuracy and latency. To solve these problems, we propose a single-step spiking neural network (S³NN), an energy-efficient neural network with low computational cost and high precision. The proposed S³NN processes the information between hidden layers by spikes as SNNs. Nevertheless, it has no temporal dimension so that there is no latency within training and inference phases as BNNs. Thus, the proposed S³NN has a lower computational cost than SNNs that require time-series processing. However, S³NN cannot adopt naïve backpropagation algorithms due to the non-differentiability nature of spikes. We deduce a suitable neuron model by reducing the surrogate gradient for multi-time step SNNs to a single-time step. We experimentally demonstrated that the obtained surrogate gradient allows S³NN to be trained appropriately. We also showed that the proposed S³NN could achieve comparable accuracy to full-precision networks while being highly energy-efficient.

Machine learning-augmented surface-enhanced spectroscopy toward next-generation molecular diagnostics

The world today is witnessing the significant role and huge demand for molecular detection and screening in healthcare and medical diagnosis, especially during the outbreak of COVID-19. Surface-enhanced spectroscopy techniques, including Surface-Enhanced Raman Scattering (SERS) and Infrared Absorption (SEIRA), provide lattice and molecular vibrational fingerprint information which is directly linked to the molecular constituents, chemical bonds, and configuration. These properties make them an unambiguous, nondestructive, and label-free toolkit for molecular diagnostics and screening. However, new issues in molecular diagnostics, such as increasing molecular species, faster spread of viruses, and higher requirements for detection accuracy and sensitivity, have brought great challenges to detection technology. Advancements in artificial intelligence and machine learning (ML) techniques show promising potential in empowering SERS and SEIRA with rapid analysis and automatic data processing to jointly tackle the challenge. This review introduces the combination of ML and SERS/SEIRA by investigating how ML algorithms can be beneficial to SERS/SEIRA, discussing the general process of combining ML and SEIRA/SERS, highlighting the molecular diagnostics and screening applications based on ML-combined SEIRA/SERS, and providing perspectives on the future development of ML-integrated SEIRA/SERS. In general, this review offers comprehensive knowledge about the recent advances and the future outlook regarding ML-integrated SEIRA/SERS for molecular diagnostics and screening.

Supervised Learning Algorithm Based on Spike Train Inner Product for Deep Spiking Neural Networks

By mimicking the hierarchical structure of human brain, deep spiking neural networks (DSNNs) can extract features from a lower level to a higher level gradually, and improve the performance for the processing of spatio-temporal information. Due to the complex hierarchical structure and implicit nonlinear mechanism, the formulation of spike train level supervised learning methods for DSNNs remains an important problem in this research area. Based on the definition of kernel function and spike trains inner product (STIP) as well as the idea of error backpropagation (BP), this paper firstly proposes a deep supervised learning algorithm for DSNNs named BP-STIP. Furthermore, in order to alleviate the intrinsic weight transport problem of the BP mechanism, feedback alignment (FA) and broadcast alignment (BA) mechanisms are utilized to optimize the error feedback mode of BP-STIP, and two deep supervised learning algorithms named FA-STIP and BA-STIP are also proposed. In the experiments, the effectiveness of the proposed three DSNN algorithms is verified on the MNIST digital image benchmark dataset, and the influence of different kernel functions on the learning performance of DSNNs with different network scales is analyzed. Experimental results show that the FA-STIP and BP-STIP algorithms can achieve 94.73% and 95.65% classification accuracy, which apparently possess better learning performance and stability compared with the benchmark algorithm BP-STIP.

Introducing the Dendrify framework for incorporating dendrites to spiking neural networks

Computational modeling has been indispensable for understanding how subcellular neuronal features influence circuit processing. However, the role of dendritic computations in network-level operations remains largely unexplored. This is partly because existing tools do not allow the development of realistic and efficient network models that account for dendrites. Current spiking neural networks, although efficient, are usually quite simplistic, overlooking essential dendritic properties. Conversely, circuit models with morphologically detailed neuron models are computationally costly, thus impractical for large-network simulations. To bridge the gap between these two extremes and facilitate the adoption of dendritic features in spiking neural networks, we introduce Dendrify, an open-source Python package based on Brian 2. Dendrify, through simple commands, automatically generates reduced compartmental neuron models with simplified yet biologically relevant dendritic and synaptic integrative properties. Such models strike a good balance between flexibility, performance, and biological accuracy, allowing us to explore dendritic contributions to network-level functions while paving the way for developing more powerful neuromorphic systems.

A Review on Machine Learning Applications for Solar Plants

A solar plant system has complex nonlinear dynamics with uncertainties due to variations in system parameters and insolation. Thereby, it is difficult to approximate these complex dynamics with conventional algorithms whereas Machine Learning (ML) methods yield the essential performance required. ML models are key units in recent sensor systems for solar plant design, forecasting, maintenance, and control to provide the best safety, reliability, robustness, and performance as compared to classical methods which are usually employed in the hardware and software of solar plants. Considering this, the goal of our paper is to explore and analyze ML technologies and their advantages and shortcomings as compared to classical methods for the design, forecasting, maintenance, and control of solar plants. In contrast with other review articles, our research briefly summarizes our intelligent, self-adaptive models for sizing, forecasting, maintenance, and control of a solar plant; sets benchmarks for performance comparison of the reviewed ML models for a solar plant's system; proposes a simple but effective integration scheme of an ML sensor solar plant system's implementation and outlines its future digital transformation into a smart solar plant based on the integrated cutting-edge technologies; and estimates the impact of ML technologies based on the proposed scheme on a solar plant value chain.

BSNN: Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons

The spiking neural network (SNN) computes and communicates information through discrete binary events. Recent work has achieved essential progress on an excellent performance by converting ANN to SNN. Due to the difference in information processing, the converted deep SNN usually suffers serious performance loss and large time delay. In this paper, we analyze the reasons for the performance loss and propose a novel bistable spiking neural network (BSNN) that addresses the problem of the phase lead and phase lag. Also, we design synchronous neurons (SN) to help efficiently improve performance when ResNet structure-based ANNs are converted. BSNN significantly improves the performance of the converted SNN by enabling more accurate delivery of information to the next layer after one cycle. Experimental results show that the proposed method only needs 1/4-1/10 of the time steps compared to previous work to achieve nearly lossless conversion. We demonstrate better ANN-SNN conversion for VGG16, ResNet20, and ResNet34 on challenging datasets including CIFAR-10 (95.16% top-1), CIFAR-100 (78.12% top-1), and ImageNet (72.64% top-1).

Linear leaky-integrate-and-fire neuron model based spiking neural networks and its mapping relationship to deep neural networks

Spiking neural networks (SNNs) are brain-inspired machine learning algorithms with merits such as biological plausibility and unsupervised learning capability. Previous works have shown that converting Artificial Neural Networks (ANNs) into SNNs is a practical and efficient approach for implementing an SNN. However, the basic principle and theoretical groundwork are lacking for training a non-accuracy-loss SNN. This paper establishes a precise mathematical mapping between the biological parameters of the Linear Leaky-Integrate-and-Fire model (LIF)/SNNs and the parameters of ReLU-AN/Deep Neural Networks (DNNs). Such mapping relationship is analytically proven under certain conditions and demonstrated by simulation and real data experiments. It can serve as the theoretical basis for the potential combination of the respective merits of the two categories of neural networks.

Spiking VGG7: Deep Convolutional Spiking Neural Network with Direct Training for Object Recognition

We propose a deep convolutional spiking neural network (DCSNN) with direct training to classify concrete bridge damage in a real engineering environment. The leaky-integrate-and-fire (LIF) neuron model is employed in our DCSNN that is similar to VGG. Poisson encoding and convolution encoding strategies are considered. The gradient surrogate method is introduced to realize the supervised training for the DCSNN. In addition, we have examined the effect of observation time step on the network performance. The testing performance for two different spike encoding strategies are compared. The results show that the DCSNN using gradient surrogate method can achieve a performance of 97.83%, which is comparable to traditional CNN. We also present a comparison with STDP-based unsupervised learning and a converted algorithm, and the proposed DCSNN is proved to have the best performance. To demonstrate the generalization performance of the model, we also use a public dataset for comparison. This work paves the way for the practical engineering applications of the deep SNNs.

A Spike Neural Network Model for Lateral Suppression of Spike-Timing-Dependent Plasticity with Adaptive Threshold

Aiming at the practical constraints of high resource occupancy and complex calculations in the existing Spike Neural Network (SNN) image classification model, in order to seek a more lightweight and efficient machine vision solution, this paper proposes an adaptive threshold Spike Neural Network (SNN) model of lateral inhibition of Spike-Timing-Dependent Plasticity (STDP). The conversion from grayscale image to pulse sequence is completed by convolution normalization and first pulse time coding. The network self-classification is realized by combining the classical Spike-Timing-Dependent Plasticity algorithm (STDP) and lateral suppression algorithm. The occurrence of overfitting is effectively suppressed by introducing an adaptive threshold. The experimental results on the MNIST data set show that compared with the traditional SNN classification model, the complexity of the weight update algorithm is reduced from O(n2) to O(1), and the accuracy rate can still remain stable at about 96%. The provided model is conducive to the migration of software algorithms to the bottom layer of the hardware platform, and can provide a reference for the realization of edge computing solutions for small intelligent hardware terminals with high efficiency and low power consumption.

Optimizing the Neural Structure and Hyperparameters of Liquid State Machines Based on Evolutionary Membrane Algorithm

As one of the important artificial intelligence fields, brain-like computing attempts to give machines a higher intelligence level by studying and simulating the cognitive principles of the human brain. A spiking neural network (SNN) is one of the research directions of brain-like computing, characterized by better biogenesis and stronger computing power than the traditional neural network. A liquid state machine (LSM) is a neural computing model with a recurrent network structure based on SNN. In this paper, a learning algorithm based on an evolutionary membrane algorithm is proposed to optimize the neural structure and hyperparameters of an LSM. First, the object of the proposed algorithm is designed according to the neural structure and hyperparameters of the LSM. Second, the reaction rules of the proposed algorithm are employed to discover the best neural structure and hyperparameters of the LSM. Third, the membrane structure is that the skin membrane contains several elementary membranes to speed up the search of the proposed algorithm. In the simulation experiment, effectiveness verification is carried out on the MNIST and KTH datasets. In terms of the MNIST datasets, the best test results of the proposed algorithm with 500, 1000 and 2000 spiking neurons are 86.8%, 90.6% and 90.8%, respectively. The best test results of the proposed algorithm on KTH with 500, 1000 and 2000 spiking neurons are 82.9%, 85.3% and 86.3%, respectively. The simulation results show that the proposed algorithm has a more competitive advantage than other experimental algorithms.

Automotive Vulnerability Analysis for Deep Learning Blockchain Consensus Algorithm

In this study, future cars are attempting self-driving around the world. However, hacking, such as ECUs in automobiles, creates problems that are directly connected to human life. Therefore, this study wrote a paper that detects anomalies in such cars by field. As a related study, the study investigated the vulnerabilities of the automobile security committee and automobile security standards and investigated the detection of abnormalities in the hacking of geo-train cars using artificial intelligence’s LSTM and blockchain consensus algorithm. In addition, in automobile security, an algorithm was studied to predict normal and abnormal values using LSTM-based anomaly detection techniques on the premise that automobile communication networks are largely divided into internal and external networks. In the methodology, LSTM’s pure propagation malicious code detection technique was used, and it worked with an artificial intelligence consensus algorithm to increase security. In addition, Unity ML conducted an experiment by constructing a virtual environment using the Beta version. The LSTM blockchain consensus node network was composed of 50,000 processes to compare performance. For the first time, 100 Grouped Tx, 500 Channels were tested for performance. For the first time, the malicious code detection rate of the existing system was verified. Accelerator, Multichannel, Sharding, Raiden, Plasma, and Trubit values were verified, and values of approximately 15,000 to 50,000 were obtained. In this paper, we studied to become a paper of great significance on hacking that threatens human life with the development of self-driving cars in the future.

Supervised Learning Algorithm for Multilayer Spiking Neural Networks with Long-Term Memory Spike Response Model

As a new brain-inspired computational model of artificial neural networks, spiking neural networks transmit and process information via precisely timed spike trains. Constructing efficient learning methods is a significant research field in spiking neural networks. In this paper, we present a supervised learning algorithm for multilayer feedforward spiking neural networks; all neurons can fire multiple spikes in all layers. The feedforward network consists of spiking neurons governed by biologically plausible long-term memory spike response model, in which the effect of earlier spikes on the refractoriness is not neglected to incorporate adaptation effects. The gradient descent method is employed to derive synaptic weight updating rule for learning spike trains. The proposed algorithm is tested and verified on spatiotemporal pattern learning problems, including a set of spike train learning tasks and nonlinear pattern classification problems on four UCI datasets. Simulation results indicate that the proposed algorithm can improve learning accuracy in comparison with other supervised learning algorithms.

Artificial Intelligence Methodologies for Data Management

This study analyses the main challenges, trends, technological approaches, and artificial intelligence methods developed by new researchers and professionals in the field of machine learning, with an emphasis on the most outstanding and relevant works to date. This literature review evaluates the main methodological contributions of artificial intelligence through machine learning. The methodology used to study the documents was content analysis; the basic terminology of the study corresponds to machine learning, artificial intelligence, and big data between the years 2017 and 2021. For this study, we selected 181 references, of which 120 are part of the literature review. The conceptual framework includes 12 categories, four groups, and eight subgroups. The study of data management using AI methodologies presents symmetry in the four machine learning groups: supervised learning, unsupervised learning, semi-supervised learning, and reinforced learning. Furthermore, the artificial intelligence methods with more symmetry in all groups are artificial neural networks, Support Vector Machines, K-means, and Bayesian Methods. Finally, five research avenues are presented to improve the prediction of machine learning.

Adaptive SNN for Anthropomorphic Finger Control

Anthropomorphic hands that mimic the smoothness of human hand motions should be controlled by artificial units of high biological plausibility. Adaptability is among the characteristics of such control units, which provides the anthropomorphic hand with the ability to learn motions. This paper presents a simple structure of an adaptive spiking neural network implemented in analogue hardware that can be trained using Hebbian learning mechanisms to rotate the metacarpophalangeal joint of a robotic finger towards targeted angle intervals. Being bioinspired, the spiking neural network drives actuators made of shape memory alloy and receives feedback from neuromorphic sensors that convert the joint rotation angle and compression force into the spiking frequency. The adaptive SNN activates independent neural paths that correspond to angle intervals and learns in which of these intervals the rotation the finger rotation is stopped by an external force. Learning occurs when angle-specific neural paths are stimulated concurrently with the supraliminar stimulus that activates all the neurons that inhibit the SNN output stopping the finger. The results showed that after learning, the finger stopped in the angle interval in which the angle-specific neural path was active, without the activation of the supraliminar stimulus. The proposed concept can be used to implement control units for anthropomorphic robots that are able to learn motions unsupervised, based on principles of high biological plausibility.