Memristive stochastic plasticity enables mimicking of neural synchrony: Memristive circuit emulates an optical illusion

Blooming and pruning: learning from mistakes with memristive synapses

Blooming and pruning is one of the most important developmental mechanisms of the biological brain in the first years of life, enabling it to adapt its network structure to the demands of the environment. The mechanism is thought to be fundamental for the development of cognitive skills. Inspired by this, Chialvo and Bak proposed in 1999 a learning scheme that learns from mistakes by eliminating from the initial surplus of synaptic connections those that lead to an undesirable outcome. Here, this idea is implemented in a neuromorphic circuit scheme using CMOS integrated HfO2-based memristive devices. The implemented two-layer neural network learns in a self-organized manner without positive reinforcement and exploits the inherent variability of the memristive devices. This approach provides hardware, local, and energy-efficient learning. A combined experimental and simulation-based parameter study is presented to find the relevant system and device parameters leading to a compact and robust memristive neuromorphic circuit that can handle association tasks.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

Adaptive cognition implemented with a context-aware and flexible neuron for next-generation artificial intelligence

Neuromorphic computing mimics the organizational principles of the brain in its quest to replicate the brain's intellectual abilities. An impressive ability of the brain is its adaptive intelligence, which allows the brain to regulate its functions "on the fly" to cope with myriad and ever-changing situations. In particular, the brain displays three adaptive and advanced intelligence abilities of context-awareness, cross frequency coupling, and feature binding. To mimic these adaptive cognitive abilities, we design and simulate a novel, hardware-based adaptive oscillatory neuron using a lattice of magnetic skyrmions. Charge current fed to the neuron reconfigures the skyrmion lattice, thereby modulating the neuron's state, its dynamics and its transfer function "on the fly." This adaptive neuron is used to demonstrate the three cognitive abilities, of which context-awareness and cross-frequency coupling have not been previously realized in hardware neurons. Additionally, the neuron is used to construct an adaptive artificial neural network (ANN) and perform context-aware diagnosis of breast cancer. Simulations show that the adaptive ANN diagnoses cancer with higher accuracy while learning faster and using a more compact and energy-efficient network than a nonadaptive ANN. The work further describes how hardware-based adaptive neurons can mitigate several critical challenges facing contemporary ANNs. Modern ANNs require large amounts of training data, energy, and chip area, and are highly task-specific; conversely, hardware-based ANNs built with adaptive neurons show faster learning, compact architectures, energy-efficiency, fault-tolerance, and can lead to the realization of broader artificial intelligence.

Toward Reflective Spiking Neural Networks Exploiting Memristive Devices

The design of modern convolutional artificial neural networks (ANNs) composed of formal neurons copies the architecture of the visual cortex. Signals proceed through a hierarchy, where receptive fields become increasingly more complex and coding sparse. Nowadays, ANNs outperform humans in controlled pattern recognition tasks yet remain far behind in cognition. In part, it happens due to limited knowledge about the higher echelons of the brain hierarchy, where neurons actively generate predictions about what will happen next, i.e., the information processing jumps from reflex to reflection. In this study, we forecast that spiking neural networks (SNNs) can achieve the next qualitative leap. Reflective SNNs may take advantage of their intrinsic dynamics and mimic complex, not reflex-based, brain actions. They also enable a significant reduction in energy consumption. However, the training of SNNs is a challenging problem, strongly limiting their deployment. We then briefly overview new insights provided by the concept of a high-dimensional brain, which has been put forward to explain the potential power of single neurons in higher brain stations and deep SNN layers. Finally, we discuss the prospect of implementing neural networks in memristive systems. Such systems can densely pack on a chip 2D or 3D arrays of plastic synaptic contacts directly processing analog information. Thus, memristive devices are a good candidate for implementing in-memory and in-sensor computing. Then, memristive SNNs can diverge from the development of ANNs and build their niche, cognitive, or reflective computations.

Binding events through the mutual synchronization of spintronic nano-neurons

The brain naturally binds events from different sources in unique concepts. It is hypothesized that this process occurs through the transient mutual synchronization of neurons located in different regions of the brain when the stimulus is presented. This mechanism of 'binding through synchronization' can be directly implemented in neural networks composed of coupled oscillators. To do so, the oscillators must be able to mutually synchronize for the range of inputs corresponding to a single class, and otherwise remain desynchronized. Here we show that the outstanding ability of spintronic nano-oscillators to mutually synchronize and the possibility to precisely control the occurrence of mutual synchronization by tuning the oscillator frequencies over wide ranges allows pattern recognition. We demonstrate experimentally on a simple task that three spintronic nano-oscillators can bind consecutive events and thus recognize and distinguish temporal sequences. This work is a step forward in the construction of neural networks that exploit the non-linear dynamic properties of their components to perform brain-inspired computations.

Memristive control of mutual spin Hall nano-oscillator synchronization for neuromorphic computing

Synchronization of large spin Hall nano-oscillator (SHNO) arrays is an appealing approach toward ultrafast non-conventional computing. However, interfacing to the array, tuning its individual oscillators and providing built-in memory units remain substantial challenges. Here, we address these challenges using memristive gating of W/CoFeB/MgO/AlO_x-based SHNOs. In its high resistance state, the memristor modulates the perpendicular magnetic anisotropy at the CoFeB/MgO interface by the applied electric field. In its low resistance state the memristor adds or subtracts current to the SHNO drive. Both electric field and current control affect the SHNO auto-oscillation mode and frequency, allowing us to reversibly turn on/off mutual synchronization in chains of four SHNOs. We also demonstrate that two individually controlled memristors can be used to tune a four-SHNO chain into differently synchronized states. Memristor gating is therefore an efficient approach to input, tune and store the state of SHNO arrays for non-conventional computing models.

Emulation of Astrocyte Induced Neural Phase Synchrony in Spin-Orbit Torque Oscillator Neurons

Astrocytes play a central role in inducing concerted phase synchronized neural-wave patterns inside the brain. In this article, we demonstrate that injected radio-frequency signal in underlying heavy metal layer of spin-orbit torque oscillator neurons mimic the neuron phase synchronization effect realized by glial cells. Potential application of such phase coupling effects is illustrated in the context of a temporal "binding problem." We also present the design of a coupled neuron-synapse-astrocyte network enabled by compact neuromimetic devices by combining the concepts of local spike-timing dependent plasticity and astrocyte induced neural phase synchrony.

Stochastic Memristive Interface for Neural Signal Processing

We propose a memristive interface consisting of two FitzHugh–Nagumo electronic neurons connected via a metal–oxide (Au/Zr/ZrO₂(Y)/TiN/Ti) memristive synaptic device. We create a hardware–software complex based on a commercial data acquisition system, which records a signal generated by a presynaptic electronic neuron and transmits it to a postsynaptic neuron through the memristive device. We demonstrate, numerically and experimentally, complex dynamics, including chaos and different types of neural synchronization. The main advantages of our system over similar devices are its simplicity and real-time performance. A change in the amplitude of the presynaptic neurogenerator leads to the potentiation of the memristive device due to the self-tuning of its parameters. This provides an adaptive modulation of the postsynaptic neuron output. The developed memristive interface, due to its stochastic nature, simulates a real synaptic connection, which is very promising for neuroprosthetic applications.

Adaptive Extreme Edge Computing for Wearable Devices

Wearable devices are a fast-growing technology with impact on personal healthcare for both society and economy. Due to the widespread of sensors in pervasive and distributed networks, power consumption, processing speed, and system adaptation are vital in future smart wearable devices. The visioning and forecasting of how to bring computation to the edge in smart sensors have already begun, with an aspiration to provide adaptive extreme edge computing. Here, we provide a holistic view of hardware and theoretical solutions toward smart wearable devices that can provide guidance to research in this pervasive computing era. We propose various solutions for biologically plausible models for continual learning in neuromorphic computing technologies for wearable sensors. To envision this concept, we provide a systematic outline in which prospective low power and low latency scenarios of wearable sensors in neuromorphic platforms are expected. We successively describe vital potential landscapes of neuromorphic processors exploiting complementary metal-oxide semiconductors (CMOS) and emerging memory technologies (e.g., memristive devices). Furthermore, we evaluate the requirements for edge computing within wearable devices in terms of footprint, power consumption, latency, and data size. We additionally investigate the challenges beyond neuromorphic computing hardware, algorithms and devices that could impede enhancement of adaptive edge computing in smart wearable devices.

Master-Slave Synchronization of Delayed Neural Networks With Time-Varying Control

This brief investigates the master-slave synchronization problem of delayed neural networks with general time-varying control. Assuming a linear feedback controller with time-varying control gain, the synchronization problem is recast into the stability problem of a delayed system with a time-varying coefficient. The main theorem is established in terms of the time average of the control gain by using the Lyapunov-Razumikhin theorem. Moreover, the proposed framework encompasses some general intermittent control schemes, such as the switched control gain with external disturbance and intermittent control with pulse-modulated gain function, while some useful corollaries are consequently deduced. Interestingly, our theorem also provides a solution for regaining stability under control failure. The validity of the theorem and corollaries is further demonstrated with numerical examples.

Engineering Method for Tailoring Electrical Characteristics in TiN/TiOx/HfOx/Au Bi-Layer Oxide Memristive Devices

Memristive devices have led to an increased interest in neuromorphic systems. However, different device requirements are needed for the multitude of computation schemes used there. While linear and time-independent conductance modulation is required for machine learning, non-linear and time-dependent properties are necessary for neurobiologically realistic learning schemes. In this context, an adaptation of the resistance switching characteristic is necessary with regard to the desired application. Recently, bi-layer oxide memristive systems have proven to be a suitable device structure for this purpose, as they combine the possibility of a tailored memristive characteristic with low power consumption and uniformity of the device performance. However, this requires technological solutions that allow for precise adjustment of layer thicknesses, defect densities in the oxide layers, and suitable area sizes of the active part of the devices. For this purpose, we have investigated the bi-layer oxide system TiN/TiOx/HfOx/Au with respect to tailored I-V non-linearity, the number of resistance states, electroforming, and operating voltages. Therefore, a 4-inch full device wafer process was used. This process allows a systematic investigation, i.e., the variation of physical device parameters across the wafer as well as a statistical evaluation of the electrical properties with regard to the variability from device to device and from cycle to cycle. For the investigation, the thickness of the HfOx layer was varied between 2 and 8 nm, and the size of the active area of devices was changed between 100 and 2,500 µm2. Furthermore, the influence of the HfOx deposition condition was investigated, which influences the conduction mechanisms from a volume-based, filamentary to an interface-based resistive switching mechanism. Our experimental results are supported by numerical simulations that show the contribution of the HfOx film in the bi-layer memristive system and guide the development of a targeting device.

Integration and Co-design of Memristive Devices and Algorithms for Artificial Intelligence

Memristive devices share remarkable similarities to biological synapses, dendrites, and neurons at both the physical mechanism level and unit functionality level, making the memristive approach to neuromorphic computing a promising technology for future artificial intelligence. However, these similarities do not directly transfer to the success of efficient computation without device and algorithm co-designs and optimizations. Contemporary deep learning algorithms demand the memristive artificial synapses to ideally possess analog weighting and linear weight-update behavior, requiring substantial device-level and circuit-level optimization. Such co-design and optimization have been the main focus of memristive neuromorphic engineering, which often abandons the “non-ideal” behaviors of memristive devices, although many of them resemble what have been observed in biological components. Novel brain-inspired algorithms are being proposed to utilize such behaviors as unique features to further enhance the efficiency and intelligence of neuromorphic computing, which calls for collaborations among electrical engineers, computing scientists, and neuroscientists.

Prospect of Spin-Orbitronic Devices and Their Applications

Science, engineering, and medicine ultimately demand fast information processing with ultra-low power consumption. The recently developed spin-orbit torque (SOT)-induced magnetization switching paradigm has been fueling opportunities for spin-orbitronic devices, i.e., enabling SOT memory and logic devices at sub-nano second and sub-picojoule regimes. Importantly, spin-orbitronic devices are intrinsic of nonvolatility, anti-radiation, unlimited endurance, excellent stability, and CMOS compatibility, toward emerging applications, e.g., processing in-memory, neuromorphic computing, probabilistic computing, and 3D magnetic random access memory. Nevertheless, the cutting-edge SOT-based devices and application remain at a premature stage owing to the lack of scalable methodology on the field-free SOT switching. Moreover, spin-orbitronics poises as an interdisciplinary field to be driven by goals of both fundamental discoveries and application innovations, to open fascinating new paths for basic research and new line of technologies. In this perspective, the specific challenges and opportunities are summarized to exert momentum on both research and eventual applications of spin-orbitronic devices.

Analogue pattern recognition with stochastic switching binary CMOS-integrated memristive devices

Biological neural networks outperform current computer technology in terms of power consumption and computing speed while performing associative tasks, such as pattern recognition. The analogue and massive parallel in-memory computing in biology differs strongly from conventional transistor electronics that rely on the von Neumann architecture. Therefore, novel bio-inspired computing architectures have been attracting a lot of attention in the field of neuromorphic computing. Here, memristive devices, which serve as non-volatile resistive memory, are employed to emulate the plastic behaviour of biological synapses. In particular, CMOS integrated resistive random access memory (RRAM) devices are promising candidates to extend conventional CMOS technology to neuromorphic systems. However, dealing with the inherent stochasticity of resistive switching can be challenging for network performance. In this work, the probabilistic switching is exploited to emulate stochastic plasticity with fully CMOS integrated binary RRAM devices. Two different RRAM technologies with different device variabilities are investigated in detail, and their potential applications in stochastic artificial neural networks (StochANNs) capable of solving MNIST pattern recognition tasks is examined. A mixed-signal implementation with hardware synapses and software neurons combined with numerical simulations shows that the proposed concept of stochastic computing is able to process analogue data with binary memory cells.

Dynamics and coupling of fractional-order models of the motor cortex and central pattern generators

OBJECTIVE

Fractional calculus plays a key role in the analysis of neural dynamics. In particular, fractional calculus has been recently exploited for analyzing complex biological systems and capturing intrinsic phenomena. Also, artificial neural networks have been shown to have complex neuronal dynamics and characteristics that can be modeled by fractional calculus. Moreover, for a neural microcircuit placed on the spinal cord, fractional calculus can be employed to model the central pattern generator (CPG). However, the relation between the CPG and the motor cortex is still unclear.

APPROACH

In this paper, fractional-order models of the CPG and the motor cortex are built on the Van der Pol oscillator and the neural mass model (NMM), respectively. A self-consistent mean field approximation is used to construct the potential landscape of the Van der Pol oscillator. This landscape provides a useful tool to observe the 3D dynamics of the oscillator. To infer the relation of the motor cortex and CPG, the coupling model between the fractional-order Van der Pol oscillator and the NMM is built. As well, the influence of the coupling parameters on the CPG and the motor cortex is assessed.

MAIN RESULTS

Fractional-order NMM and coupling model of the motor cortex and the CPG are first established. The potential landscape is used to show 3D probabilistic evolution of the Van der Pol oscillator states. Detailed observations of the evolution of the system states can be made with fractional calculus. In particular, fractional calculus enables the observation of the creation of stable modes and switching between them.

SIGNIFICANCE

The results confirm that the motor cortex and CPG have associated modes or states that can be switched based on changes in the fractional order and the time delay. Fractional calculus and the potential landscape are helpful methods for better understanding of the working principles of locomotion systems.

Two-dimensional mutually synchronized spin Hall nano-oscillator arrays for neuromorphic computing

In spin Hall nano-oscillators (SHNOs), pure spin currents drive local regions of magnetic films and nanostructures into auto-oscillating precession. If such regions are placed in close proximity to each other they can interact and may mutually synchronize. Here, we demonstrate robust mutual synchronization of two-dimensional SHNO arrays ranging from 2 × 2 to 8 × 8 nano-constrictions, observed both electrically and using micro-Brillouin light scattering microscopy. On short time scales, where the auto-oscillation linewidth [Formula: see text] is governed by white noise, the signal quality factor, [Formula: see text], increases linearly with the number of mutually synchronized nano-constrictions (N), reaching 170,000 in the largest arrays. We also show that SHNO arrays exposed to two independently tuned microwave frequencies exhibit the same synchronization maps as can be used for neuromorphic vowel recognition. Our demonstrations may hence enable the use of SHNO arrays in two-dimensional oscillator networks for high-quality microwave signal generation and ultra-fast neuromorphic computing.

Bridging Biological and Artificial Neural Networks with Emerging Neuromorphic Devices: Fundamentals, Progress, and Challenges

As the research on artificial intelligence booms, there is broad interest in brain-inspired computing using novel neuromorphic devices. The potential of various emerging materials and devices for neuromorphic computing has attracted extensive research efforts, leading to a large number of publications. Going forward, in order to better emulate the brain's functions, its relevant fundamentals, working mechanisms, and resultant behaviors need to be re-visited, better understood, and connected to electronics. A systematic overview of biological and artificial neural systems is given, along with their related critical mechanisms. Recent progress in neuromorphic devices is reviewed and, more importantly, the existing challenges are highlighted to hopefully shed light on future research directions.