CrossNets: High-Performance Neuromorphic Architectures for CMOL Circuits

Mosaic: in-memory computing and routing for small-world spike-based neuromorphic systems

The brain's connectivity is locally dense and globally sparse, forming a small-world graph-a principle prevalent in the evolution of various species, suggesting a universal solution for efficient information routing. However, current artificial neural network circuit architectures do not fully embrace small-world neural network models. Here, we present the neuromorphic Mosaic: a non-von Neumann systolic architecture employing distributed memristors for in-memory computing and in-memory routing, efficiently implementing small-world graph topologies for Spiking Neural Networks (SNNs). We've designed, fabricated, and experimentally demonstrated the Mosaic's building blocks, using integrated memristors with 130 nm CMOS technology. We show that thanks to enforcing locality in the connectivity, routing efficiency of Mosaic is at least one order of magnitude higher than other SNN hardware platforms. This is while Mosaic achieves a competitive accuracy in a variety of edge benchmarks. Mosaic offers a scalable approach for edge systems based on distributed spike-based computing and in-memory routing.

Back-End, CMOS-Compatible Ferroelectric Field-Effect Transistor for Synaptic Weights

Neuromorphic computing architectures enable the dense colocation of memory and processing elements within a single circuit. This colocation removes the communication bottleneck of transferring data between separate memory and computing units as in standard von Neuman architectures for data-critical applications including machine learning. The essential building blocks of neuromorphic systems are nonvolatile synaptic elements such as memristors. Key memristor properties include a suitable nonvolatile resistance range, continuous linear resistance modulation, and symmetric switching. In this work, we demonstrate voltage-controlled, symmetric and analog potentiation and depression of a ferroelectric Hf_0.57Zr_0.43O₂ (HZO) field-effect transistor (FeFET) with good linearity. Our FeFET operates with low writing energy (fJ) and fast programming time (40 ns). Retention measurements have been performed over 4 bit depth with low noise (1%) in the tungsten oxide (WO_x) readout channel. By adjusting the channel thickness from 15 to 8 nm, the on/off ratio of the FeFET can be engineered from 1 to 200% with an on-resistance ideally >100 kΩ, depending on the channel geometry. The device concept is using earth-abundant materials and is compatible with a back end of line (BEOL) integration into complementary metal-oxide-semiconductor (CMOS) processes. It has therefore a great potential for the fabrication of high-density, large-scale integrated arrays of artificial analog synapses.

A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: from mitigation to exploitation

Memristive devices represent a promising technology for building neuromorphic electronic systems. In addition to their compactness and non-volatility, they are characterized by their computationally relevant physical properties, such as their state-dependence, non-linear conductance changes, and intrinsic variability in both their switching threshold and conductance values, that make them ideal devices for emulating the bio-physics of real synapses. In this paper we present a spiking neural network architecture that supports the use of memristive devices as synaptic elements and propose mixed-signal analog-digital interfacing circuits that mitigate the effect of variability in their conductance values and exploit their variability in the switching threshold for implementing stochastic learning. The effect of device variability is mitigated using pairs of memristive devices configured in a complementary push-pull mechanism and interfaced to a current-mode normalizer circuit. The stochastic learning mechanism is obtained by mapping the desired change in synaptic weight into a corresponding switching probability that is derived from the intrinsic stochastic behavior of memristive devices. We demonstrate the features of the CMOS circuits and apply the architecture proposed to a standard neural network hand-written digit classification benchmark based on the MNIST data-set. We evaluate the performance of the approach proposed in this benchmark using behavioral-level spiking neural network simulation, showing both the effect of the reduction in conductance variability produced by the current-mode normalizer circuit and the increase in performance as a function of the number of memristive devices used in each synapse.

Mimicking biological neurons with a nanoscale ferroelectric transistor

Neuron is the basic computing unit in brain-inspired neural networks. Although a multitude of excellent artificial neurons realized with conventional transistors have been proposed, they might not be energy and area efficient in large-scale networks. The recent discovery of ferroelectricity in hafnium oxide (HfO2) and the related switching phenomena at the nanoscale might provide a solution. This study employs the newly reported accumulative polarization reversal in nanoscale HfO2-based ferroelectric field-effect transistors (FeFETs) to implement two key neuronal dynamics: the integration of action potentials and the subsequent firing according to the biologically plausible all-or-nothing law. We show that by carefully shaping electrical excitations based on the particular nucleation-limited switching kinetics of the ferroelectric layer further neuronal behaviors can be emulated, such as firing activity tuning, arbitrary refractory period and the leaky effect. Finally, we discuss the advantages of an FeFET-based neuron, highlighting its transferability to advanced scaling technologies and the beneficial impact it may have in reducing the complexity of neuromorphic circuits.

A 2-transistor/1-resistor artificial synapse capable of communication and stochastic learning in neuromorphic systems

Resistive (or memristive) switching devices based on metal oxides find applications in memory, logic and neuromorphic computing systems. Their small area, low power operation, and high functionality meet the challenges of brain-inspired computing aiming at achieving a huge density of active connections (synapses) with low operation power. This work presents a new artificial synapse scheme, consisting of a memristive switch connected to 2 transistors responsible for gating the communication and learning operations. Spike timing dependent plasticity (STDP) is achieved through appropriate shaping of the pre-synaptic and the post synaptic spikes. Experiments with integrated artificial synapses demonstrate STDP with stochastic behavior due to (i) the natural variability of set/reset processes in the nanoscale switch, and (ii) the different response of the switch to a given stimulus depending on the initial state. Experimental results are confirmed by model-based simulations of the memristive switching. Finally, system-level simulations of a 2-layer neural network and a simplified STDP model show random learning and recognition of patterns.

Pattern classification by memristive crossbar circuits using ex situ and in situ training

Memristors are memory resistors that promise the efficient implementation of synaptic weights in artificial neural networks. Whereas demonstrations of the synaptic operation of memristors already exist, the implementation of even simple networks is more challenging and has yet to be reported. Here we demonstrate pattern classification using a single-layer perceptron network implemented with a memrisitive crossbar circuit and trained using the perceptron learning rule by ex situ and in situ methods. In the first case, synaptic weights, which are realized as conductances of titanium dioxide memristors, are calculated on a precursor software-based network and then imported sequentially into the crossbar circuit. In the second case, training is implemented in situ, so the weights are adjusted in parallel. Both methods work satisfactorily despite significant variations in the switching behaviour of the memristors. These results give hope for the anticipated efficient implementation of artificial neuromorphic networks and pave the way for dense, high-performance information processing systems.

Integration of nanoscale memristor synapses in neuromorphic computing architectures

Conventional neuro-computing architectures and artificial neural networks have often been developed with no or loose connections to neuroscience. As a consequence, they have largely ignored key features of biological neural processing systems, such as their extremely low-power consumption features or their ability to carry out robust and efficient computation using massively parallel arrays of limited precision, highly variable, and unreliable components. Recent developments in nano-technologies are making available extremely compact and low power, but also variable and unreliable solid-state devices that can potentially extend the offerings of availing CMOS technologies. In particular, memristors are regarded as a promising solution for modeling key features of biological synapses due to their nanoscale dimensions, their capacity to store multiple bits of information per element and the low energy required to write distinct states. In this paper, we first review the neuro- and neuromorphic computing approaches that can best exploit the properties of memristor and scale devices, and then propose a novel hybrid memristor-CMOS neuromorphic circuit which represents a radical departure from conventional neuro-computing approaches, as it uses memristors to directly emulate the biophysics and temporal dynamics of real synapses. We point out the differences between the use of memristors in conventional neuro-computing architectures and the hybrid memristor-CMOS circuit proposed, and argue how this circuit represents an ideal building block for implementing brain-inspired probabilistic computing paradigms that are robust to variability and fault tolerant by design.

Synaptic electronics: materials, devices and applications

In this paper, the recent progress of synaptic electronics is reviewed. The basics of biological synaptic plasticity and learning are described. The material properties and electrical switching characteristics of a variety of synaptic devices are discussed, with a focus on the use of synaptic devices for neuromorphic or brain-inspired computing. Performance metrics desirable for large-scale implementations of synaptic devices are illustrated. A review of recent work on targeted computing applications with synaptic devices is presented.

Nanopsychiatry--the potential role of nanotechnologies in the future of psychiatry: a systematic review

Nanomedicine is defined as the area using nanotechnology's concepts for the benefit of human beings' health and well being. In this article, we aimed to provide an overview of areas where nanotechnology is applied and how they could be extended to care for psychiatric illnesses. The main applications of nanotechnology in psychiatry are (i) pharmacology. There are two main difficulties in neuropharmacology: drugs have to pass the blood-brain barrier and then to be internalized by targeted cells. Nanoparticles could increase drugs bioavailability and pharmacokinetics, especially improving safety and efficacy of psychotropic drugs. Liposomes, nanosomes, nanoparticle polymers, nanobubbles are some examples of this targeted drug delivery. Nanotechnologies could also add new pharmacological properties, like nanoshells and dendrimers (ii) living analysis. Nanotechnology provides technical assistance to in vivo imaging or metabolome analysis (iii) central nervous system modeling. Research teams have succeeded to modelize inorganic synapses and mimick synaptic behavior, a step essential for further creation of artificial neural systems. Some nanoparticle assemblies present the same small worlds and free-scale networks architecture as cortical neural networks. Nanotechnologies and quantum physics could be used to create models of artificial intelligence and mental illnesses. We are not about to see a concrete application of nanomedicine in daily psychiatric practice. Even if nanotechnologies are promising, their safety is still inconsistent and this must be kept in mind. However, it seems essential that psychiatrists do not forsake this area of research the perspectives of which could be decisive in the field of mental illness.

Stochastic memristive devices for computing and neuromorphic applications

Nanoscale resistive switching devices (memristive devices or memristors) have been studied for a number of applications ranging from non-volatile memory, logic to neuromorphic systems. However a major challenge is to address the potentially large variations in space and time in these nanoscale devices. Here we show that in metal-filament based memristive devices the switching can be fully stochastic. While individual switching events are random, the distribution and probability of switching can be well predicted and controlled. Rather than trying to force high switching probabilities using excess voltage or time, the inherent stochastic nature of resistive switching allows these binary devices to be used as building blocks for novel error-tolerant computing schemes such as stochastic computing and provides the needed "analog" feature for neuromorphic applications. To verify such potential, we demonstrated memristor-based stochastic bitstreams in both time and space domains, and show that an array of binary memristors can act as a multi-level "analog" device for neuromorphic applications.

[Nanopsychiatry. The potential role of nanotechnologies in the future of psychiatry. A systematic review]

INTRODUCTION

Nanomedicine is defined as the area using nanotechnology's concepts for the benefit of human beings, their health and well being. The field of nanotechnology opened new unsuspected fields of research a few years ago.

AIM OF THE STUDY

To provide an overview of nanotechnology application areas that could affect care for psychiatric illnesses.

METHODS

We conducted a systematic review using the PRISMA criteria (preferred reporting items for systematic reviews and meta-analysis). Inclusion criteria were specified in advance: all studies describing the development of nanotechnology in psychiatry. The research paradigm was: "(nanotechnology OR nanoparticles OR nanomedicine) AND (central nervous system)" Articles were identified in three research bases, Medline (1966-present), Web of Science (1975-present) and Cochrane (all articles). The last search was carried out on April 2, 2012. Seventy-six items were included in this qualitative review.

RESULTS

The main applications of nanotechnology in psychiatry are (i) pharmacology. There are two main difficulties in neuropharmacology. Drugs have to pass the blood brain barrier and then to be internalized by targeted cells. Nanoparticles could increase drugs' bioavailability and pharmacokinetics, especially improving safety and efficacy of psychotropic drugs. Liposomes, nanosomes, nanoparticle polymers, nanobubbles are some examples of this targeted drug delivery. Nanotechnologies could also add new pharmacological properties, like nanohells and dendrimers; (ii) living analysis. Nanotechnology provides technical assistance to in vivo imaging or metabolome analysis; (iii) central nervous system modeling. Research teams have modelized inorganic synapses and mimicked synaptic behavior, essential for further creation of artificial neural systems. Some nanoparticle assemblies present the same small world and free-scale network architecture as cortical neural networks. Nanotechnologies and quantum physics could be used to create models of artificial intelligence and mental illnesses.

DISCUSSION

Even if nanotechnologies are promising, their safety is still tricky and this must be kept in mind.

CONCLUSION

We are not about to see a concrete application of nanomedicine in daily psychiatric practice. However, it seems essential that psychiatrists do not forsake this area of research the perspectives of which could be decisive in the field of mental illness.

Memristive devices for computing

Memristive devices are electrical resistance switches that can retain a state of internal resistance based on the history of applied voltage and current. These devices can store and process information, and offer several key performance characteristics that exceed conventional integrated circuit technology. An important class of memristive devices are two-terminal resistance switches based on ionic motion, which are built from a simple conductor/insulator/conductor thin-film stack. These devices were originally conceived in the late 1960s and recent progress has led to fast, low-energy, high-endurance devices that can be scaled down to less than 10 nm and stacked in three dimensions. However, the underlying device mechanisms remain unclear, which is a significant barrier to their widespread application. Here, we review recent progress in the development and understanding of memristive devices. We also examine the performance requirements for computing with memristive devices and detail how the outstanding challenges could be met.

Neuromorphic atomic switch networks

Efforts to emulate the formidable information processing capabilities of the brain through neuromorphic engineering have been bolstered by recent progress in the fabrication of nonlinear, nanoscale circuit elements that exhibit synapse-like operational characteristics. However, conventional fabrication techniques are unable to efficiently generate structures with the highly complex interconnectivity found in biological neuronal networks. Here we demonstrate the physical realization of a self-assembled neuromorphic device which implements basic concepts of systems neuroscience through a hardware-based platform comprised of over a billion interconnected atomic-switch inorganic synapses embedded in a complex network of silver nanowires. Observations of network activation and passive harmonic generation demonstrate a collective response to input stimulus in agreement with recent theoretical predictions. Further, emergent behaviors unique to the complex network of atomic switches and akin to brain function are observed, namely spatially distributed memory, recurrent dynamics and the activation of feedforward subnetworks. These devices display the functional characteristics required for implementing unconventional, biologically and neurally inspired computational methodologies in a synthetic experimental system.

Biomimetic model of the outer plexiform layer by incorporating memristive devices

In this paper we present a biorealistic model for the first part of the early vision of processing by incorporating memristive nanodevices. The architecture of the proposed network is based on the organization and functioning of the outer plexiform layer (OPL) in the vertebrate retina. We demonstrate that memristive devices are indeed a valuable building block for neuromorphic architectures, as their highly nonlinear and adaptive response could be exploited for establishing ultradense networks with dynamics similar to that of their biological counterparts. We particularly show that hexagonal memristive grids can be employed for faithfully emulating the smoothing effect occurring in the OPL to enhance the dynamic range of the system. In addition, we employ a memristor-based thresholding scheme for detecting the edges of grayscale images, while the proposed system is also evaluated for its adaptation and fault tolerance capacity against different light or noise conditions as well as its distinct device yields.

Emergent criticality in complex turing B-type atomic switch networks

Recent advances in the neuromorphic operation of atomic switches as individual synapse-like devices demonstrate the ability to process information with both short-term and long-term memorization in a single two terminal junction. Here it is shown that atomic switches can be self-assembled within a highly interconnected network of silver nanowires similar in structure to Turing’s “B-Type unorganized machine”, originally proposed as a randomly connected network of NAND logic gates. In these experimental embodiments,complex networks of coupled atomic switches exhibit emergent criticality similar in nature to previously reported electrical activity of biological brains and neuron assemblies. Rapid fluctuations in electrical conductance display metastability and power law scaling of temporal correlation lengths that are attributed to dynamic reorganization of the interconnected electro-ionic network resulting from induced non-equilibrium thermodynamic instabilities. These collective properties indicate a potential utility for realtime,multi-input processing of distributed sensory data through reservoir computation. We propose these highly coupled, nonlinear electronic networks as an implementable hardware-based platform toward the creation of physically intelligent machines.

Four-dimensional address topology for circuits with stacked multilayer crossbar arrays

We present a topological framework that provides a simple yet powerful electronic circuit architecture for constructing and using multilayer crossbar arrays, allowing a significantly increased integration density of memristive crosspoint devices beyond the scaling limits of lateral feature sizes. The truly remarkable feature of such circuits, which is an extension of the CMOL (Cmos + MOLecular-scale devices) concept for an area-like interface to a three-dimensional system, is that a large-feature-size complimentary metal-oxide-semiconductor (CMOS) substrate can provide high-density interconnects to multiple crossbar layers through a single set of vertical vias. The physical locations of the memristive devices are mapped to a four-dimensional logical address space such that unique access from the CMOS substrate is provided to every device in a stacked array of crossbars. This hybrid architecture is compatible with digital memories, field-programmable gate arrays, and biologically inspired adaptive networks and with state-of-the-art integrated circuit foundries.

A hybrid nanomemristor/transistor logic circuit capable of self-programming

Memristor crossbars were fabricated at 40 nm half-pitch, using nanoimprint lithography on the same substrate with Si metal-oxide-semiconductor field effect transistor (MOS FET) arrays to form fully integrated hybrid memory resistor (memristor)/transistor circuits. The digitally configured memristor crossbars were used to perform logic functions, to serve as a routing fabric for interconnecting the FETs and as the target for storing information. As an illustrative demonstration, the compound Boolean logic operation (A AND B) OR (C AND D) was performed with kilohertz frequency inputs, using resistor-based logic in a memristor crossbar with FET inverter/amplifier outputs. By routing the output signal of a logic operation back onto a target memristor inside the array, the crossbar was conditionally configured by setting the state of a nonvolatile switch. Such conditional programming illuminates the way for a variety of self-programmed logic arrays, and for electronic synaptic computing.

Negative differential resistance at sequential single-electron tunnelling through atoms and molecules

We have carried out calculations of electron transport in single-electron transistors using single atoms or small molecules as single-electron islands. The theory is based on a combination of (i) the general theory of the sequential single-electron transport through objects with a quantized energy spectrum, developed by Averin and Korotkov, (ii) the ab initio calculation of molecular orbitals and energy spectra within the density functional theory framework (using the NRLMOL software package), and (iii) Bardeen's approximation for the rate of tunnelling due to wavefunction overlap. The results show, in particular, that dc I-V curves of molecular-scale single-electron transistors typically have extended branches with negative differential resistance. This effect is due to the enhancement of one of the two tunnelling barriers of the transistor by the source-drain electric field, and apparently has already been observed experimentally by at least two groups. In conclusion, the possibility of using this effect for increasing the density and performance of hybrid semiconductor/nanodevice integrated circuits is discussed in brief.

Self-assembled molecular magnets on patterned silicon substrates: Bridging bio-molecules with nanoelectronics

The paper reports the methods of preparing molecular magnets and patterning of the molecules on a semiconductor surface. A highly magnetically aligned metallothionein containing Mn and Cd (Mn,Cd-MT-2) is first synthesized, and the molecules are then placed into nanopores prepared on silicon (001) surfaces using electron beam lithography and reactive ion-etching techniques. We have observed the self-assemble growth of the MT molecules on the patterned Si surface such that the MT molecules have grown into rod or ring type three-dimensional nanostructures, depending on the patterned nanostructures on the surface. We also provide scanning electron microscopy, atomic force microscopy, and magnetic force microscope studies of the molecular nanostructures. This engineered molecule shows molecular magnetization and is biocompatible with conventional semiconductors. These features make Mn,Cd-MT-2 a good candidate for biological applications and sensing sources of new nanodevices. Using molecular self-assembly and topographical patterning of the semiconductor substrate, we can close the gap between bio-molecules and nanoelectronics built into the semiconductor chip.