All Spin Artificial Neural Networks Based on Compound Spintronic Synapse and Neuron

Bimodal alteration of cognitive accuracy for spintronic artificial neural networks

Spintronics-based artificial neural networks (ANNs) exhibiting nonvolatile, fast, and energy-efficient computing capabilities are promising neuromorphic hardware for performing complex cognitive tasks of artificial intelligence and machine learning. Early experimental efforts focused on multistate device concepts to enhance synaptic weight precisions, albeit compromising on cognitive accuracy due to their low magnetoresistance. Here, we propose a hybrid approach based on the tuning of tunnel magnetoresistance (TMR) and the number of states in the compound magnetic tunnel junctions (MTJs) to improve the cognitive performance of an all-spin ANN. A TMR variation of 33-78% is controlled by the free layer (FL) thickness wedge (1.6-2.6 nm) across the wafer. Meanwhile, the number of resistance states in the compound MTJ is manipulated by varying the number of constituent MTJ cells (n = 1-3), generating n + 1 states with a TMR difference between consecutive states of at least 21%. Using MNIST handwritten digit and fashion object databases, the test accuracy of the compound MTJ ANN is observed to increase with the number of intermediate states for a fixed FL thickness or TMR. Meanwhile, the test accuracy for a 1-cell MTJ increases linearly by 8.3% and 7.4% for handwritten digits and fashion objects, respectively, with increasing TMR. Interestingly, a multifarious TMR dependence of test accuracy is observed with the increasing synaptic complexity in the 2- and 3-cell MTJs. By leveraging on the bimodal tuning of multilevel and TMR, we establish viable paths for enhancing the cognitive performance of spintronic ANN for in-memory and neuromorphic computing.

Analyzing the effects of streetscape and land use on urban accidents and predicting future accidents by using machine learning algorithms (case study: Mashhad)

In general, land use and layout of streets can have a significant impact on the behavior of drivers and pedestrians. In particular, streetscape has often been overlooked that recognizing the role of streetscape on street accident in urban areas is important. The aim of this research is to investigate the influence of streetscape and land use on urban accidents that occurred in Mashhad between the years 2017 and 2021. To achieve this objective, the study focused on analyzing accidents in three different urban zones. It also considered the land use types adjacent to both closed and open streets, including residential, commercial, and mixed land uses. The research employed various surveys to gather the necessary data and insights related to the targeted areas. Statistics on accident in three zones show that among the mentioned land uses, commercial areas have experienced the highest number of accidents, with their share being approximately three times that of accidents in residential areas. Additionally, 75 % of all accidents took place in areas with open streetscape, whereas accidents in areas with enclosed view accounted for one third of the number of accidents in open streetscape areas. In this research, analysis and modeling were conducted using machine learning algorithms implemented in the Python programming language. Several models were employed, and the best models were selected based on their performance and accuracy, which include Random Forest Regression (RFR), Multilayer Neural Network Perceptron Regression (MLP) and Extreme Boost Gradient Regression (XGBoost). The accuracy of the machine learning models which successfully predicted future outcomes was as follows: Random Forest Regression (RFR) achieved 85 % accuracy, Extreme Boost Gradient Regression (XGBoost) achieved 81 % accuracy, and finally, Neural Network Multilayer Perceptron Regression (MLP) achieved 75 % accuracy.

Strain-mediated multistate skyrmion for neuron devices

Magnetic skyrmions are potential candidates for neuromorphic computing because of their inherent topological stability, low drive current density and nanoscale size. However, an artificial neuron device based on current-driven skyrmion motion cannot satisfy the requirement of energy efficiency and integration density due to hundreds of millions of interconnected neurons and synapses present in the deep networks. Here, we present a compact and energy efficient skyrmion-based artificial neuron consisting of ferromagnetic/heavy metal/ferroelectric layers which uses strain-mediated voltage manipulation of skyrmion states to mimic the Integrate-and-Fire (IF) function of biological neurons. By implementation of a spiking neural network (SNN) based on the proposed skyrmionic neuronal devices, it can achieve a high accuracy of 95.08% on a modified National Institute of Standards and Technology (MNIST) handwritten digit dataset, as well as a low power consumption of ∼46.8 fJ per epoch per neuron. The present work suggests a novel way to realize energy-efficient and high-density neuromorphic computing.

Domain wall magnetic tunnel junction-based artificial synapses and neurons for all-spin neuromorphic hardware

We report a breakthrough in the hardware implementation of energy-efficient all-spin synapse and neuron devices for highly scalable integrated neuromorphic circuits. Our work demonstrates the successful execution of all-spin synapse and activation function generator using domain wall-magnetic tunnel junctions. By harnessing the synergistic effects of spin-orbit torque and interfacial Dzyaloshinskii-Moriya interaction in selectively etched spin-orbit coupling layers, we achieve a programmable multi-state synaptic device with high reliability. Our first-principles calculations confirm that the reduced atomic distance between 5d and 3d atoms enhances Dzyaloshinskii-Moriya interaction, leading to stable domain wall pinning. Our experimental results, supported by visualizing energy landscapes and theoretical simulations, validate the proposed mechanism. Furthermore, we demonstrate a spin-neuron with a sigmoidal activation function, enabling high operation frequency up to 20 MHz and low energy consumption of 508 fJ/operation. A neuron circuit design with a compact sigmoidal cell area and low power consumption is also presented, along with corroborated experimental implementation. Our findings highlight the great potential of domain wall-magnetic tunnel junctions in the development of all-spin neuromorphic computing hardware, offering exciting possibilities for energy-efficient and scalable neural network architectures.

Bi-sigmoid spike-timing dependent plasticity learning rule for magnetic tunnel junction-based SNN

In this study, we explore spintronic synapses composed of several Magnetic Tunnel Junctions (MTJs), leveraging their attractive characteristics such as endurance, nonvolatility, stochasticity, and energy efficiency for hardware implementation of unsupervised neuromorphic systems. Spiking Neural Networks (SNNs) running on dedicated hardware are suitable for edge computing and IoT devices where continuous online learning and energy efficiency are important characteristics. We focus in this work on synaptic plasticity by conducting comprehensive electrical simulations to optimize the MTJ-based synapse design and find the accurate neuronal pulses that are responsible for the Spike Timing Dependent Plasticity (STDP) behavior. Most proposals in the literature are based on hardware-independent algorithms that require the network to store the spiking history to be able to update the weights accordingly. In this work, we developed a new learning rule, the Bi-Sigmoid STDP (B2STDP), which originates from the physical properties of MTJs. This rule enables immediate synaptic plasticity based on neuronal activity, leveraging in-memory computing. Finally, the integration of this learning approach within an SNN framework leads to a 91.71% accuracy in unsupervised image classification, demonstrating the potential of MTJ-based synapses for effective online learning in hardware-implemented SNNs.

Multistate Compound Magnetic Tunnel Junction Synapses for Digital Recognition

The quest to mimic the multistate synapses for bioinspired computing has triggered nascent research that leverages the well-established magnetic tunnel junction (MTJ) technology. Early works on the spin transfer torque MTJ-based artificial neural network (ANN) are susceptible to poor thermal reliability, high latency, and high critical current densities. Meanwhile, work on spin-orbit torque (SOT) MTJ-based ANN mainly utilized domain wall motion, which yields negligibly small readout signals differentiating consecutive states and has designs that are incompatible with technological scale-up. Here, we propose a multistate device concept built upon a compound MTJ consisting of multiple SOT-MTJs (number of MTJs, n = 1-4) on a shared write channel, mimicking the spin-based ANN. The n + 1 resistance states representing varying synaptic weights can be tuned by varying the voltage pulses (±1.5-1.8 V), pulse duration (100-300 ns), and applied in-plane fields (5.5-10.5 mT). A large TMR difference of more than 13.6% is observed between two consecutive states for the 4-cell compound MTJ, a 4-fold improvement from reported state-of-the-art spin-based synaptic devices. The ANN built upon the compound MTJ shows high learning accuracy for digital recognition tasks with incremental states and retraining, achieving test accuracy as high as 95.75% in the 4-cell compound MTJ. These results provide an industry-compatible platform to integrate these multistate SOT-MTJ synapses directly into neuromorphic architecture for in-memory and unconventional computing applications.

Electrochemical-Memristor-Based Artificial Neurons and Synapses-Fundamentals, Applications, and Challenges

Artificial neurons and synapses are considered essential for the progress of the future brain-inspired computing, based on beyond von Neumann architectures. Here, a discussion on the common electrochemical fundamentals of biological and artificial cells is provided, focusing on their similarities with the redox-based memristive devices. The driving forces behind the functionalities and the ways to control them by an electrochemical-materials approach are presented. Factors such as the chemical symmetry of the electrodes, doping of the solid electrolyte, concentration gradients, and excess surface energy are discussed as essential to understand, predict, and design artificial neurons and synapses. A variety of two- and three-terminal memristive devices and memristive architectures are presented and their application for solving various problems is shown. The work provides an overview of the current understandings on the complex processes of neural signal generation and transmission in both biological and artificial cells and presents the state-of-the-art applications, including signal transmission between biological and artificial cells. This example is showcasing the possibility for creating bioelectronic interfaces and integrating artificial circuits in biological systems. Prospectives and challenges of the modern technology toward low-power, high-information-density circuits are highlighted.

Multi-state MRAM cells for hardware neuromorphic computing

Magnetic tunnel junctions (MTJ) have been successfully applied in various sensing application and digital information storage technologies. Currently, a number of new potential applications of MTJs are being actively studied, including high-frequency electronics, energy harvesting or random number generators. Recently, MTJs have been also proposed in designs of new platforms for unconventional or bio-inspired computing. In the current work, we present a complete hardware implementation design of a neural computing device that incorporates serially connected MTJs forming a multi-state memory cell can be used in a hardware implementation of a neural computing device. The main purpose of the multi-cell is the formation of quantized weights in the network, which can be programmed using the proposed electronic circuit. Multi-cells are connected to a CMOS-based summing amplifier and a sigmoid function generator, forming an artificial neuron. The operation of the designed network is tested using a recognition of hand-written digits in 20 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document}× 20 pixels matrix and shows detection ratio comparable to the software algorithm, using weights stored in a multi-cell consisting of four MTJs or more. Moreover, the presented solution has better energy efficiency in terms of energy consumed per single image processing, as compared to a similar design.

High On/Off Ratio Spintronic Multi-Level Memory Unit for Deep Neural Network

Spintronic devices are considered as one of the most promising technologies for non-volatile memory and computing. However, two crucial drawbacks, that is, lack of intrinsic multi-level operation and low on/off ratio, greatly hinder their further application for advanced computing concepts, such as deep neural network (DNN) accelerator. In this paper, a spintronic multi-level memory unit with high on/off ratio is proposed by integrating several series-connected magnetic tunnel junctions (MTJs) with perpendicular magnetic anisotropy (PMA) and a Schottky diode in parallel. Due to the rectification effect on the PMA MTJ, an on/off ratio over 100, two orders of magnitude higher than intrinsic values, is obtained under proper proportion of alternating current and direct current. Multiple resistance states are stably achieved and can be reconfigured by spin transfer torque effect. A computing-in-memory architecture based DNN accelerator for image classification with the experimental parameters of this proposal to evidence its application potential is also evaluated. This work can satisfy the rigorous requirements of DNN for memory unit and promote the development of high-accuracy and robust artificial intelligence applications.

Pure voltage-driven spintronic neuron based on stochastic magnetization switching behaviour

Voltage-driven stochastic magnetization switching in a nanomagnet has attracted more attention recently with its superiority in achieving energy-efficient artificial neuron. Here, a novel pure voltage-driven scheme with ∼27.66 aJ energy dissipation is proposed, which could rotate magnetization vector randomly using only a pair of electrodes covered on the multiferroic nanomagnet. Results show that the probability of 180° magnetization switching is examined as a sigmoid-like function of the voltage pulse width and magnitude, which can be utilized as the activation function of designed neuron. Considering the size errors of designed neuron in fabrication, it's found that reasonable thickness and width variations cause little effect on recognition accuracy for MNIST hand-written dataset. In other words, the designed pure voltage-driven spintronic neuron could tolerate size errors. These results open a new way toward the realization of artificial neural network with low power consumption and high reliability.

Stimuli-Responsive Memristive Materials for Artificial Synapses and Neuromorphic Computing

Neuromorphic computing holds promise for building next-generation intelligent systems in a more energy-efficient way than the conventional von Neumann computing architecture. Memristive hardware, which mimics biological neurons and synapses, offers high-speed operation and low power consumption, enabling energy- and area-efficient, brain-inspired computing. Here, recent advances in memristive materials and strategies that emulate synaptic functions for neuromorphic computing are highlighted. The working principles and characteristics of biological neurons and synapses, which can be mimicked by memristive devices, are presented. Besides device structures and operation with different external stimuli such as electric, magnetic, and optical fields, how memristive materials with a rich variety of underlying physical mechanisms can allow fast, reliable, and low-power neuromorphic applications is also discussed. Finally, device requirements are examined and a perspective on challenges in developing memristive materials for device engineering and computing science is given.

Neuromorphic Devices for Bionic Sensing and Perception

Neuromorphic devices that can emulate the bionic sensory and perceptual functions of neural systems have great applications in personal healthcare monitoring, neuro-prosthetics, and human-machine interfaces. In order to realize bionic sensing and perception, it's crucial to prepare neuromorphic devices with the function of perceiving environment in real-time. Up to now, lots of efforts have been made in the incorporation of the bio-inspired sensing and neuromorphic engineering in the booming artificial intelligence industry. In this review, we first introduce neuromorphic devices based on diverse materials and mechanisms. Then we summarize the progress made in the emulation of biological sensing and perception systems. Finally, the challenges and opportunities in these fields are also discussed.

Implementation of an efficient magnetic tunnel junction-based stochastic neural network with application to iris data classification

Stochastic neuromorphic computation (SNC) has the potential to enable a low power, error tolerant and scalable computing platform in comparison to its deterministic counterparts. However, the hardware implementation of complementary metal oxide semiconductor (CMOS)-based stochastic circuits involves conversion blocks that cost more than the actual processing circuits. The realization of the activation function for SNCs also requires a complicated circuit that results in a significant amount of power dissipation and area overhead. The inherent probabilistic switching behavior of nanomagnets provides an advantage to overcome these complexity issues for the realization of low power and area efficient SNC systems. This paper presents magnetic tunnel junction (MTJ)-based stochastic computing methodology for the implementation of a neural network. The stochastic switching behavior of the MTJ has been exploited to design a binary to stochastic converter to mitigate the complexity of the CMOS-based design. The paper also presents the technique for realizing stochastic sigmoid activation function using an MTJ. Such circuits are simpler than existing ones and use considerably less power. An image classification system employing the proposed circuits has been implemented to verify the effectiveness of the technique. The MTJ-based SNC system shows area and energy reduction by a factor of 13.5 and 2.5, respectively, while the prediction accuracy is 86.66%. Furthermore, this paper investigates how crucial parameters, such as stochastic bitstream length, number of hidden layers and number of nodes in a hidden layer, need to be set precisely to realize an efficient MTJ-based stochastic neural network (SNN). The proposed methodology can prove a promising alternative for highly efficient digital stochastic computing applications.

Room-Temperature Manipulation of Magnetization Angle, Achieved with an All-Solid-State Redox Device

An all-solid-state redox device, composed of magnetite (Fe₃O₄) thin film and Li⁺ conducting electrolyte thin film, was fabricated for the manipulation of a magnetization angle at room temperature (RT). This is a key technology for the creation of efficient spintronics devices, but has not yet been achieved at RT by other carrier doping methods. Variations in magnetization angle and magnetic stability were precisely tracked through the use of planar Hall measurements at RT. The magnetization angle was reversibly manipulated at 10° by maintaining magnetic stability. Meanwhile, the manipulatable angle reached 56°, although the manipulation became irreversible when the magnetic stability was reduced. This large manipulation of magnetic angle was achieved through tuning of the 3d electron number and modulation of the internal strain in the Fe₃O₄ due to the insertion of high-density Li⁺ (approximately 10²¹ cm^-3). This RT manipulation is applicable to highly integrated spintronics devices due to its simple structure and low electric power consumption.

Antiferromagnet-Based Neuromorphics Using Dynamics of Topological Charges

We propose a spintronics-based hardware implementation of neuromorphic computing, specifically, the spiking neural network, using topological winding textures in one-dimensional antiferromagnets. The consistency of such a network is emphasized in light of the conservation of topological charges, and the natural spatiotemporal interconversions of magnetic winding. We discuss the realization of the leaky integrate-and-fire behavior of neurons and the spike-timing-dependent plasticity of synapses. Our proposal opens the possibility for an all-spin neuromorphic platform based on antiferromagnetic insulators.

Nonvolatile Multistates Memories for High-Density Data Storage

In the current information age, the realization of memory devices with energy efficient design, high storage density, nonvolatility, fast access, and low cost is still a great challenge. As a promising technology to meet these stringent requirements, nonvolatile multistates memory (NMSM) has attracted lots of attention over the past years. Owing to the capability to store data in more than a single bit (0 or 1), the storage density is dramatically enhanced without scaling down the memory cell, making memory devices more efficient and less expensive. Multistates in a single cell also provide an unconventional in-memory computing platform beyond the Von Neumann architecture and enable neuromorphic computing with low power consumption. In this review, an in-depth perspective is presented on the recent progress and challenges on the device architectures, material innovation, working mechanisms of various types of NMSMs, including flash, magnetic random-access memory (MRAM), resistive random-access memory (RRAM), ferroelectric random-access memory (FeRAM), and phase-change memory (PCM). The intriguing properties and performance of these NMSMs, which are the key to realizing highly integrated memory hierarchy, are discussed and compared.

Improvement of Conductance Modulation Linearity in a Cu2+-Doped KNbO3 Memristor through the Increase of the Number of Oxygen Vacancies

The Pt/KNbO₃/TiN/Si (KN) memristor exhibits various biological synaptic properties. However, it also displays nonlinear conductance modulation with the application of identical pulses, indicating that it should be improved for neuromorphic applications. The abrupt change of the conductance originates from the inhomogeneous growth/dissolution of oxygen vacancy filaments in the KN film. The change of the filaments in a KN film is controlled by two mechanisms with different growth/dissolution rates: a redox process with a fast rate and an oxygen vacancy diffusion process with a slow rate. Therefore, the conductance modulation linearity can be improved if the growth/dissolution of the filaments is controlled by only one mechanism. When the number of oxygen vacancies in the KN film was increased through doping of Cu²⁺ ions, the growth/dissolution of the filaments in the Cu²⁺-doped KN (CKN) film was mainly influenced by the redox process of oxygen vacancies. Therefore, the CKN film exhibited improved conductance modulation linearity, confirming that the linearity of conductance modulation can be improved by increasing the number of oxygen vacancies in the memristor. This method can be applied to other memristors to improve the linearity of conductance modulation. The CKN memristor also provides excellent biological synaptic characteristics for neuromorphic computing systems.

Unsupervised online learning of temporal information in spiking neural network using thin-film transistor-type NOR flash memory devices

Brain-inspired analog neuromorphic systems based on the synaptic arrays have attracted large attention due to low-power computing. Spike-timing-dependent plasticity (STDP) algorithm is considered as one of the appropriate neuro-inspired techniques to be applied for on-chip learning. The aim of this study is to investigate the methodology of unsupervised STDP based learning in temporal encoding systems. The system-level simulation was performed based on the measurement results of thin-film transistor-type asymmetric floating-gate NOR flash memory. With proposed learning methods, 91.53% of recognition accuracy is obtained in inferencing MNIST standard dataset with 200 output neurons. Moreover, temporal encoding rules showed that the number of input pulses and the computing power can be compressed without significant loss of recognition accuracy compared to the conventional rate encoding scheme. In addition, temporal computing in a multi-layer network is suitable for learning data sequences, suggesting the possibility of applying to real-world tasks such as classifying direction of moving objects.

Emerging memory technologies for neuromorphic computing

In this paper, we reviewed the recent trends on neuromorphic computing using emerging memory technologies. Two representative learning algorithms used to implement a hardware-based neural network are described as a bio-inspired learning algorithm and software-based learning algorithm, in particular back-propagation. The requirements of the synaptic device to apply each algorithm were analyzed. Then, we reviewed the research trends of synaptic devices to implement an artificial neural network.

A Split-Gate Positive Feedback Device With an Integrate-and-Fire Capability for a High-Density Low-Power Neuron Circuit

Hardware-based spiking neural networks (SNNs) to mimic biological neurons have been reported. However, conventional neuron circuits in SNNs have a large area and high power consumption. In this work, a split-gate floating-body positive feedback (PF) device with a charge trapping capability is proposed as a new neuron device that imitates the integrate-and-fire function. Because of the PF characteristic, the subthreshold swing (SS) of the device is less than 0.04 mV/dec. The super-steep SS of the device leads to a low energy consumption of ∼0.25 pJ/spike for a neuron circuit (PF neuron) with the PF device, which is ∼100 times smaller than that of a conventional neuron circuit. The charge storage properties of the device mimic the integrate function of biological neurons without a large membrane capacitor, reducing the PF neuron area by about 17 times compared to that of a conventional neuron. We demonstrate the successful operation of a dense multiple PF neuron system with reset and lateral inhibition using a common self-controller in a neuron layer through simulation. With the multiple PF neuron system and the synapse array, on-line unsupervised pattern learning and recognition are successfully performed to demonstrate the feasibility of our PF device in a neural network.

Skyrmions in Magnetic Tunnel Junctions

In this work, we demonstrate that skyrmions can be nucleated in the free layer of a magnetic tunnel junction (MTJ) with Dzyaloshinskii-Moriya interactions (DMIs) by a spin-polarized current with the assistance of stray fields from the pinned layer. The size, stability, and number of created skyrmions can be tuned by either the DMI strength or the stray field distribution. The interaction between the stray field and the DMI effective field is discussed. A device with multilevel tunneling magnetoresistance is proposed, which could pave the ways for skyrmion-MTJ-based multibit storage and artificial neural network computation. Our results may facilitate the efficient nucleation and electrical detection of skyrmions.

Recent Advances on Neuromorphic Systems Using Phase-Change Materials

Realization of brain-like computer has always been human's ultimate dream. Today, the possibility of having this dream come true has been significantly boosted due to the advent of several emerging non-volatile memory devices. Within these innovative technologies, phase-change memory device has been commonly regarded as the most promising candidate to imitate the biological brain, owing to its excellent scalability, fast switching speed, and low energy consumption. In this context, a detailed review concerning the physical principles of the neuromorphic circuit using phase-change materials as well as a comprehensive introduction of the currently available phase-change neuromorphic prototypes becomes imperative for scientists to continuously progress the technology of artificial neural networks. In this paper, we first present the biological mechanism of human brain, followed by a brief discussion about physical properties of phase-change materials that recently receive a widespread application on non-volatile memory field. We then survey recent research on different types of neuromorphic circuits using phase-change materials in terms of their respective geometrical architecture and physical schemes to reproduce the biological events of human brain, in particular for spike-time-dependent plasticity. The relevant virtues and limitations of these devices are also evaluated. Finally, the future prospect of the neuromorphic circuit based on phase-change technologies is envisioned.