Phase Oscillatory Network and Visual Pattern Recognition

Experimental demonstration of coupled differential oscillator networks for versatile applications

Oscillatory neural networks (ONNs) exhibit a high potential for energy-efficient computing. In ONNs, neurons are implemented with oscillators and synapses with resistive and/or capacitive coupling between pairs of oscillators. Computing is carried out on the basis of the rich, complex, non-linear synchronization dynamics of a system of coupled oscillators. The exploited synchronization phenomena in ONNs are an example of fully parallel collective computing. A fast system's convergence to stable states, which correspond to the desired processed information, enables an energy-efficient solution if small area and low-power oscillators are used, specifically when they are built on the basis of the hysteresis exhibited by phase-transition materials such as VO2. In recent years, there have been numerous studies on ONNs using VO2. Most of them report simulation results. Although in some cases experimental results are also shown, they do not implement the design techniques that other works on electrical simulations report that allow to improve the behavior of the ONNs. Experimental validation of these approaches is necessary. Therefore, in this study, we describe an ONN realized in a commercial CMOS technology in which the oscillators are built using a circuit that we have developed to emulate the VO2 device. The purpose is to be able to study in-depth the synchronization dynamics of relaxation oscillators similar to those that can be performed with VO2 devices. The fabricated circuit is very flexible. It allows programming the synapses to implement different ONNs, calibrating the frequency of the oscillators, or controlling their initialization. It uses differential oscillators and resistive synapses, equivalent to the use of memristors. In this article, the designed and fabricated circuits are described in detail, and experimental results are shown. Specifically, its satisfactory operation as an associative memory is demonstrated. The experiments carried out allow us to conclude that the ONN must be operated according to the type of computational task to be solved, and guidelines are extracted in this regard.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

Unified approach for applications of oscillatory associative-memory networks with error-free retrieval

Given a set of standard binary patterns and a defective pattern, the binary pattern retrieval task is to find the closest pattern to the defective one among these standard patterns. The associative-memory network of Kuramoto oscillators consisting of a Hebbian coupling term and a second-order Fourier term can be applied to this task. When the memorized patterns stored in the Hebbian coupling are mutually orthogonal, recent studies show that the network is capable of distinguishing the memorized patterns from most other patterns. However, the orthogonality usually fails in real situations. In this paper, we present a unified approach for the application of this model in pattern retrieval problems with any general set of standard patterns. By subgrouping the standard patterns and employing an orthogonal lift of each subgroup, this approach makes use of the theory in the case of mutually orthogonal memorized patterns. In particular, the error-free retrieval can be guaranteed, which requires that the retrieved pattern must coincide with one of the standard patterns. As illustrative simulations, pattern retrieval tests for partly sheltered Arabic number symbols are presented.

Memristive Izhikevich Spiking Neuron Model and Its Application in Oscillatory Associative Memory

The Izhikevich (IZH) spiking neuron model can display spiking and bursting behaviors of neurons. Based on the switching property and bio-plausibility of the memristor, the memristive Izhikevich (MIZH) spiking neuron model is built. Firstly, the MIZH spiking model is introduced and used to generate 23 spiking patterns. We compare the 23 spiking patterns produced by the IZH and MIZH spiking models. Secondly, the MIZH spiking model actively reproduces various neuronal behaviors, including the excitatory cortical neurons, the inhibitory cortical neurons, and other cortical neurons. Finally, the collective dynamic activities of the MIZH neuronal network are performed, and the MIZH oscillatory network is constructed. Experimental results illustrate that the constructed MIZH spiking neuron model performs high firing frequency and good frequency adaptation. The model can easily simulate various spiking and bursting patterns of distinct neurons in the brain. The MIZH neuronal network realizes the synchronous and asynchronous collective behaviors. The MIZH oscillatory network can memorize and retrieve the information patterns correctly and efficiently with high retrieval accuracy.

Artificial Intelligence for Biology

Despite efforts to integrate research across different subdisciplines of biology, the scale of integration remains limited. We hypothesize that future generations of Artificial Intelligence (AI) technologies specifically adapted for biological sciences will help enable the reintegration of biology. AI technologies will allow us not only to collect, connect and analyze data at unprecedented scales, but also to build comprehensive predictive models that span various subdisciplines. They will make possible both targeted (testing specific hypotheses) and untargeted discoveries. AI for biology will be the cross-cutting technology that will enhance our ability to do biological research at every scale. We expect AI to revolutionize biology in the 21st century much like statistics transformed biology in the 20th century. The difficulties, however, are many, including data curation and assembly, development of new science in the form of theories that connect the subdisciplines, and new predictive and interpretable AI models that are more suited to biology than existing machine learning and AI techniques. Development efforts will require strong collaborations between biological and computational scientists. This white paper provides a vision for AI for Biology and highlights some challenges.

Oscillatory Neural Networks Using VO2 Based Phase Encoded Logic

Nano-oscillators based on phase-transition materials are being explored for the implementation of different non-conventional computing paradigms. In particular, vanadium dioxide (VO₂) devices are used to design autonomous non-linear oscillators from which oscillatory neural networks (ONNs) can be developed. In this work, we propose a new architecture for ONNs in which sub-harmonic injection locking (SHIL) is exploited to ensure that the phase information encoded in each neuron can only take two values. In this sense, the implementation of ONNs from neurons that inherently encode information with two-phase values has advantages in terms of robustness and tolerance to variability present in VO₂ devices. Unlike conventional interconnection schemes, in which the sign of the weights is coded in the value of the resistances, in our proposal the negative (positive) weights are coded using static inverting (non-inverting) logic at the output of the oscillator. The operation of the proposed architecture is shown for pattern recognition applications.

Coupled VO2 Oscillators Circuit as Analog First Layer Filter in Convolutional Neural Networks

In this work we present an in-memory computing platform based on coupled VO₂ oscillators fabricated in a crossbar configuration on silicon. Compared to existing platforms, the crossbar configuration promises significant improvements in terms of area density and oscillation frequency. Further, the crossbar devices exhibit low variability and extended reliability, hence, enabling experiments on 4-coupled oscillator. We demonstrate the neuromorphic computing capabilities using the phase relation of the oscillators. As an application, we propose to replace digital filtering operation in a convolutional neural network with oscillating circuits. The concept is tested with a VGG13 architecture on the MNIST dataset, achieving performances of 95% in the recognition task.

Cooperative maintenance of cellular identity in systems with intercellular communication defects

The cooperative dynamics of cellular populations emerging from the underlying interactions determines cellular functions and thereby their identity in tissues. Global deviations from this dynamics, on the other hand, reflect pathological conditions. However, how these conditions are stabilized from dysregulation on the level of the single entities is still unclear. Here, we tackle this question using the generic Hodgkin-Huxley type of models that describe physiological bursting dynamics of pancreatic β-cells and introduce channel dysfunction to mimic pathological silent dynamics. The probability for pathological behavior in β-cell populations is ∼100% when all cells have these defects, despite the negligible size of the silent state basin of attraction for single cells. In stark contrast, in a more realistic scenario for a mixed population, stabilization of the pathological state depends on the size of the subpopulation which acquired the defects. However, the probability to exhibit stable pathological dynamics in this case is less than 10%. These results, therefore, suggest that the physiological bursting dynamics of a population of β-cells is cooperatively maintained, even under intercellular communication defects induced by dysfunctional channels of single cells.

Predicting slow and fast neuronal dynamics with machine learning

In this work, we employ reservoir computing, a recently developed machine learning technique, to predict the time evolution of neuronal activity produced by the Hindmarsh-Rose neuronal model. Our results show accurate short- and long-term predictions for periodic (tonic and bursting) neuronal behaviors, but only short-term accurate predictions for chaotic neuronal states. However, after the accuracy of the short-term predictability deteriorates in the chaotic regime, the predicted output continues to display similarities with the actual neuronal behavior. This is reinforced by a striking resemblance between the bifurcation diagrams of the actual and of the predicted outputs. Error analyses of the reservoir's performance are consistent with standard results previously obtained.

DEGRADAÇÃO DE SACOLAS PLÁSTICAS CONVENCIONAIS E OXI-BIODEGRADÁVEIS SUBMETIDAS AOS PROCESSOS DE HIDRÓLISE ÁCIDA E BÁSICA

O aumento da utilização de sacolas plásticas gerou um grande problema em seu descarte, que muitas vezes é feito em locais inapropriados. Tem sido bastante discutido como alternativa para esse problema a reciclagem química de materiais poliméricos via despolimerização por hidrólise, que consiste na reação do polímero com excesso de água na presença de um catalisador e aquecimento, resultando na recuperação dos monômeros de partida. O presente estudo teve como principal objetivo analisar a degradação das sacolas plásticas convencionais e oxi-biodegradáveis pós-consumo através da reação de hidrólise em meio ácido e básico. O experimento foi composto por um esquema fatorial 2 x 4, onde respectivamente corresponde ao tipo de sacola versus diferentes reagentes, e 4 x 3, onde respectivamente corresponde a diferentes reagentes versus diferentes concentrações, totalizando 24 amostras. Na primeira semana as soluções contendo as amostras ficaram acondicionadas em temperatura ambiente, já na segunda semana as mesmas foram acondicionadas em uma estufa e mantidas a 70ºC, após esse período as amostras foram retiradas da estufa e lavadas com água destilada, para que fosse feita a secagem dos reagentes. O melhor resultado foi observado no ensaio N13, onde o percentual de perda de massa foi de 27,08%. Já o pior resultado ocorreu na amostra B18, que obteve o ganho de massa de 110,14%. Com esse trabalho conclui-se que a degradação foi significativa apenas nas sacolas convencionais e que é possível a utilização dessa técnica na criação de alternativas tecnológicas que combatam o crescente descaso com o meio ambiente.

Theoretical Study of Oscillator Neurons in Recurrent Neural Networks

Neurons in a network can be both active or inactive. Given a subset of neurons in a network, is it possible for the subset of neurons to evolve to form an active oscillator by applying some external periodic stimulus? Furthermore, can these oscillator neurons be observable, that is, is it a stable oscillator? This paper explores such possibility, finding that an important property: any subset of neurons can be intermittently co-activated to form a stable oscillator by applying some external periodic input without any condition. Thus, the existing of intermittently active oscillator neurons is an essential property possessed by the networks. Moreover, this paper shows that, under some conditions, a subset of neurons can be fully co-activated to form a stable oscillator. Such neurons are called selectable oscillator neurons. Necessary and sufficient conditions are established for a subset of neurons to be selectable oscillator neurons in linear threshold recurrent neuron networks. It is proved that a subset of neurons forms selectable oscillator neurons if and only if the real part of each eigenvalue of the associated synaptic connection weight submatrix of the network is not larger than one. This simple condition makes the concept of selectable oscillator neurons tractable. The selectable oscillator neurons can be regarded as memories stored in the synaptic connections of networks, which enables to find a new perspective of memories in neural networks, different from the equilibrium-type attractors.

Quantum-Inspired Multidirectional Associative Memory With a Self-Convergent Iterative Learning

Quantum-inspired computing is an emerging research area, which has significantly improved the capabilities of conventional algorithms. In general, quantum-inspired hopfield associative memory (QHAM) has demonstrated quantum information processing in neural structures. This has resulted in an exponential increase in storage capacity while explaining the extensive memory, and it has the potential to illustrate the dynamics of neurons in the human brain when viewed from quantum mechanics perspective although the application of QHAM is limited as an autoassociation. We introduce a quantum-inspired multidirectional associative memory (QMAM) with a one-shot learning model, and QMAM with a self-convergent iterative learning model (IQMAM) based on QHAM in this paper. The self-convergent iterative learning enables the network to progressively develop a resonance state, from inputs to outputs. The simulation experiments demonstrate the advantages of QMAM and IQMAM, especially the stability to recall reliability.

Bifurcation transitions in gap-junction-coupled neurons

Here we investigate transitions occurring in the dynamical states of pairs of distinct neurons electrically coupled, with one neuron tonic and the other bursting. Depending on the dynamics of the individual neurons, and for strong enough coupling, they synchronize either in a tonic or a bursting regime, or initially tonic transitioning to bursting via a period doubling cascade. Certain intrinsic properties of the individual neurons such as minimum firing rates are carried over into the dynamics of the coupled neurons affecting their ultimate synchronous state.

Robust autoassociative memory with coupled networks of Kuramoto-type oscillators

Uncertain recognition success, unfavorable scaling of connection complexity, or dependence on complex external input impair the usefulness of current oscillatory neural networks for pattern recognition or restrict technical realizations to small networks. We propose a network architecture of coupled oscillators for pattern recognition which shows none of the mentioned flaws. Furthermore we illustrate the recognition process with simulation results and analyze the dynamics analytically: Possible output patterns are isolated attractors of the system. Additionally, simple criteria for recognition success are derived from a lower bound on the basins of attraction.

Synchronization versus neighborhood similarity in complex networks of nonidentical oscillators

Does the assignment order of a fixed collection of slightly distinct subsystems into given communication channels influence the overall ensemble behavior? We discuss this question in the context of complex networks of nonidentical interacting oscillators. Three types of connection configurations are considered: Similar, Dissimilar, and Neutral patterns. These different groups correspond, respectively, to oscillators alike, distinct, and indifferent relative to their neighbors. To construct such scenarios we define a vertex-weighted graph measure, the total dissonance, which comprises the sum of the dissonances between all neighbor oscillators in the network. Our numerical simulations show that the more homogeneous a network, the higher tend to be both the coupling strength required for phase locking and the associated final phase configuration spread over the circle. On the other hand, the initial spread of partial synchronization occurs faster for Similar patterns in comparison to Dissimilar ones, while neutral patterns are an intermediate situation between both extremes.

Partial synchronization in networks of non-linearly coupled oscillators: The Deserter Hubs Model

We study the Deserter Hubs Model: a Kuramoto-like model of coupled identical phase oscillators on a network, where attractive and repulsive couplings are balanced dynamically due to nonlinearity of interactions. Under weak force, an oscillator tends to follow the phase of its neighbors, but if an oscillator is compelled to follow its peers by a sufficient large number of cohesive neighbors, then it actually starts to act in the opposite manner, i.e., in anti-phase with the majority. Analytic results yield that if the repulsion parameter is small enough in comparison with the degree of the maximum hub, then the full synchronization state is locally stable. Numerical experiments are performed to explore the model beyond this threshold, where the overall cohesion is lost. We report in detail partially synchronous dynamical regimes, like stationary phase-locking, multistability, periodic and chaotic states. Via statistical analysis of different network organizations like tree, scale-free, and random ones, we found a measure allowing one to predict relative abundance of partially synchronous stationary states in comparison to time-dependent ones.