A spiking neuron model: applications and learning

Synchrony measure for a neuron driven by excitatory and inhibitory inputs and its adaptation to experimentally-recorded data

The aim of the current work is twofold: firstly to adapt an existing method measuring the input synchrony of a neuron driven only by excitatory inputs in such a way so as to account for inhibitory inputs as well and secondly to further appropriately adapt this measure so as to be correctly utilised on experimentally-recorded data. The existing method uses the normalized pre-spike slope (NPSS) of the membrane potential, resulting from observing the slope of depolarization of the membrane potential of a neuron prior to the moment of crossing the threshold within a short period of time, to identify the response-relevant input synchrony and through it to infer the operational mode of a neuron. The first adaptation of NPSS is made such that its upper bound calculation accommodates for the higher possible slope values caused by the lower average and minimum membrane potential values due to inhibitory inputs. Results indicate that when the input spike trains arrive randomly, the modified NPSS works as expected inferring that the neuron is operating as a temporal integrator. When the input spike trains arrive in perfect synchrony though, the modified NPSS works as expected only when the level of inhibition is much higher than the level of excitation. This suggests that calculation of the upper bound of the NPSS should be a function of the ratio between excitatory and inhibitory inputs in order to be able to correctly capture perfect synchrony at a neuron's input. In addition, we effectively demonstrate a process which has to be followed when aiming to use the NPSS on real neuron recordings. This process, which relies on empirical observations of the slope of depolarisation for estimating the bounds for the range of observed interspike interval lengths, is successfully applied to experimentally-recorded data showing that through it both a real neuron's operational mode and the amount of input synchrony that caused its firing can be inferred.

Distributed storage and recall of sentences

The human brain is able to learn language by processing written or spoken language. Recently, several deep neural networks have been successfully used for natural language generation. Although it is possible to train such networks, it remains unknown how these networks (or the brain) actually process language. A scalable method for distributed storage and recall of sentences within a neural network is presented. A corpus of 59 million words was used for training. A system using this method can efficiently identify sentences that can be considered reasonable replies to an input sentence. The system first selects a small number of seeds words which occur with low frequency in the corpus. These seed words are then used to generate answer sentences. Possible answers are scored using statistical data also obtained from the corpus. A number of sample answers generated by the system are shown to illustrate how the method works.

Learning optimisation by high firing irregularity

In a network of leaky integrate-and-fire (LIF) neurons, we investigate the functional role of irregular spiking at high rates. Irregular spiking is produced by either employing the partial somatic reset mechanism on every LIF neuron of the network or by using temporally correlated inputs. In both the benchmark problem of XOR (exclusive-OR) and in a general-sum game, it is shown that irrespective of the mechanism that is used to produce it, high firing irregularity enhances the learning capability of the spiking neural network trained with reward-modulated spike-timing-dependent plasticity. These results suggest that the brain may be utilising high firing irregularity for the purposes of learning optimisation.

Multiagent reinforcement learning: spiking and nonspiking agents in the iterated Prisoner's Dilemma

This paper investigates multiagent reinforcement learning (MARL) in a general-sum game where the payoffs' structure is such that the agents are required to exploit each other in a way that benefits all agents. The contradictory nature of these games makes their study in multiagent systems quite challenging. In particular, we investigate MARL with spiking and nonspiking agents in the Iterated Prisoner's Dilemma by exploring the conditions required to enhance its cooperative outcome. The spiking agents are neural networks with leaky integrate-and-fire neurons trained with two different learning algorithms: 1) reinforcement of stochastic synaptic transmission, or 2) reward-modulated spike-timing-dependent plasticity with eligibility trace. The nonspiking agents use a tabular representation and are trained with Q- and SARSA learning algorithms, with a novel reward transformation process also being applied to the Q-learning agents. According to the results, the cooperative outcome is enhanced by: 1) transformed internal reinforcement signals and a combination of a high learning rate and a low discount factor with an appropriate exploration schedule in the case of non-spiking agents, and 2) having longer eligibility trace time constant in the case of spiking agents. Moreover, it is shown that spiking and nonspiking agents have similar behavior and therefore they can equally well be used in a multiagent interaction setting. For training the spiking agents in the case where more than one output neuron competes for reinforcement, a novel and necessary modification that enhances competition is applied to the two learning algorithms utilized, in order to avoid a possible synaptic saturation. This is done by administering to the networks additional global reinforcement signals for every spike of the output neurons that were not "responsible" for the preceding decision.