Toward the Optimal Design and FPGA Implementation of Spiking Neural Networks

The performance of a biologically plausible spiking neural network (SNN) largely depends on the model parameters and neural dynamics. This article proposes a parameter optimization scheme for improving the performance of a biologically plausible SNN and a parallel on-field-programmable gate array (FPGA) online learning neuromorphic platform for the digital implementation based on two numerical methods, namely, the Euler and third-order Runge-Kutta (RK3) methods. The optimization scheme explores the impact of biological time constants on information transmission in the SNN and improves the convergence rate of the SNN on digit recognition with a suitable choice of the time constants. The parallel digital implementation leads to a significant speedup over software simulation on a general-purpose CPU. The parallel implementation with the Euler method enables around 180× ( 20× ) training (inference) speedup over a Pytorch-based SNN simulation on CPU. Moreover, compared with previous work, our parallel implementation shows more than 300× ( 240× ) improvement on speed and 180× ( 250× ) reduction in energy consumption for training (inference). In addition, due to the high-order accuracy, the RK3 method is demonstrated to gain 2× training speedup over the Euler method, which makes it suitable for online training in real-time applications.

From End to End: Gaining, Sorting, and Employing High-Density Neural Single Unit Recordings

The meaning behind neural single unit activity has constantly been a challenge, so it will persist in the foreseeable future. As one of the most sourced strategies, detecting neural activity in high-resolution neural sensor recordings and then attributing them to their corresponding source neurons correctly, namely the process of spike sorting, has been prevailing so far. Support from ever-improving recording techniques and sophisticated algorithms for extracting worthwhile information and abundance in clustering procedures turned spike sorting into an indispensable tool in electrophysiological analysis. This review attempts to illustrate that in all stages of spike sorting algorithms, the past 5 years innovations' brought about concepts, results, and questions worth sharing with even the non-expert user community. By thoroughly inspecting latest innovations in the field of neural sensors, recording procedures, and various spike sorting strategies, a skeletonization of relevant knowledge lays here, with an initiative to get one step closer to the original objective: deciphering and building in the sense of neural transcript.

Duplicate Detection of Spike Events: A Relevant Problem in Human Single-Unit Recordings

Single-unit recordings in the brain of behaving human subjects provide a unique opportunity to advance our understanding of neural mechanisms of cognition. These recordings are exclusively performed in medical centers during diagnostic or therapeutic procedures. The presence of medical instruments along with other aspects of the hospital environment limit the control of electrical noise compared to animal laboratory environments. Here, we highlight the problem of an increased occurrence of simultaneous spike events on different recording channels in human single-unit recordings. Most of these simultaneous events were detected in clusters previously labeled as artifacts and showed similar waveforms. These events may result from common external noise sources or from different micro-electrodes recording activity from the same neuron. To address the problem of duplicate recorded events, we introduce an open-source algorithm to identify these artificial spike events based on their synchronicity and waveform similarity. Applying our method to a comprehensive dataset of human single-unit recordings, we demonstrate that our algorithm can substantially increase the data quality of these recordings. Given our findings, we argue that future studies of single-unit activity recorded under noisy conditions should employ algorithms of this kind to improve data quality.

Spiking Neural P Systems with Delay on Synapses

Based on the feature and communication of neurons in animal neural systems, spiking neural P systems (SN P systems) were proposed as a kind of powerful computing model. Considering the length of axons and the information transmission speed on synapses, SN P systems with delay on synapses (SNP-DS systems) are proposed in this work. Unlike the traditional SN P systems, where all the postsynaptic neurons receive spikes at the same instant from their presynaptic neuron, the postsynaptic neurons in SNP-DS systems would receive spikes at different instants, depending on the delay time on the synapses connecting them. It is proved that the SNP-DS systems are universal as number generators. Two small universal SNP-DS systems, with standard or extended rules, are constructed to compute functions, using 56 and 36 neurons, respectively. Moreover, a simulator has been provided, in order to check the correctness of these two SNP-DS systems, thus providing an experimental validation of the universality of the systems designed.

Monitor-Based Spiking Recurrent Network for the Representation of Complex Dynamic Patterns

Neural networks are powerful computation tools for mimicking the human brain to solve realistic problems. Since spiking neural networks are a type of brain-inspired network, called the novel spiking system, Monitor-based Spiking Recurrent network (MbSRN), is derived to learn and represent patterns in this paper. This network provides a computational framework for memorizing the targets using a simple dynamic model that maintains biological plasticity. Based on a recurrent reservoir, the MbSRN presents a mechanism called a ‘monitor’ to track the components of the state space in the training stage online and to self-sustain the complex dynamics in the testing stage. The network firing spikes are optimized to represent the target dynamics according to the accumulation of the membrane potentials of the units. Stability analysis of the monitor conducted by limiting the coefficient penalty in the loss function verifies that our network has good anti-interference performance under neuron loss and noise. The results of solving some realistic tasks show that the MbSRN not only achieves a high goodness-of-fit of the target patterns but also maintains good spiking efficiency and storage capacity.

Evaluation of Spike Sorting Algorithms: Application to Human Subthalamic Nucleus Recordings and Simulations

An important prerequisite for the analysis of spike synchrony in extracellular recordings is the extraction of single-unit activity from the multi-unit signal. To identify single units, potential spikes are separated with respect to their potential neuronal origins ('spike sorting'). However, different sorting algorithms yield inconsistent unit assignments, which seriously influences subsequent spike train analyses. We aim to identify the best sorting algorithm for subthalamic nucleus recordings of patients with Parkinson's disease (experimental data ED). Therefore, we apply various prevalent algorithms offered by the 'Plexon Offline Sorter' and evaluate the sorting results. Since this evaluation leaves us unsure about the best algorithm, we apply all methods again to artificial data (AD) with known ground truth. AD consists of pairs of single units with different shape similarity embedded in the background noise of the ED. The sorting evaluation depicts a significant influence of the respective methods on the single unit assignments. We find a high variability in the sortings obtained by different algorithms that increases with single units shape similarity. We also find significant differences in the resulting firing characteristics. We conclude that Valley-Seeking algorithms produce the most accurate result if the exclusion of artifacts as unsorted events is important. If the latter is less important ('clean' data) the K-Means algorithm is a better option. Our results strongly argue for the need of standardized validation procedures based on ground truth data. The recipe suggested here is simple enough to become a standard procedure.

Directional communication during movement execution interferes with tremor in Parkinson's disease

BACKGROUND

Both the cerebello-thalamo-cortical circuit and the basal ganglia/cortical motor loop have been postulated to be generators of tremor in PD. The recent suggestion that the basal ganglia trigger tremor episodes and the cerebello-thalamo-cortical circuitry modulates tremor amplitude combines both competing hypotheses. However, the role of the STN in tremor generation and the impact of proprioceptive feedback on tremor suppression during voluntary movements have not been considered in this model yet.

OBJECTIVES

The objective of this study was to evaluate the role of the STN and proprioceptive feedback in PD tremor generation during movement execution.

METHODS

Local-field potentials of the STN as well as electromyographical and electroencephalographical rhythms were recorded in tremor-dominant and nontremor PD patients while performing voluntary movements of the contralateral hand during DBS surgery. Effective connectivity between these electrophysiological signals were analyzed and compared to electromyographical tremor activity.

RESULTS

There was an intensified information flow between the STN and the muscle in the tremor frequencies (5-8 Hz) for tremor-dominant, in comparison to nontremor, patients. In both subtypes, active movement was associated with an increase of afferent interaction between the muscle and the cortex in the β- and γ-frequencies. The γ-frequency (30-40 Hz) of this communication between muscle and cortex correlated inversely with electromyographical tremor activity.

CONCLUSIONS

Our results indicate an involvement of the STN in propagation of tremor-related activity to the muscle. Furthermore, we provide evidence that increased proprioceptive information flow during voluntary movement interferes with central tremor generation. © 2018 International Parkinson and Movement Disorder Society.

Intraoperative localization of spatially and spectrally distinct resting-state networks in Parkinson's disease

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an effective treatment for symptomatic Parkinson's disease (PD); the clinical benefit may not only mirror modulation of local STN activity but also reflect consecutive network effects on cortical oscillatory activity. Moreover, STN-DBS selectively suppresses spatially and spectrally distinct patterns of synchronous oscillatory activity within cortical-subcortical loops. These STN-cortical circuits have been described in PD patients using magnetoencephalography after surgery. This network information, however, is currently not available during surgery to inform the implantation strategy.The authors recorded spontaneous brain activity in 3 awake patients with PD (mean age 67 ± 14 years; mean disease duration 13 ± 7 years) during implantation of DBS electrodes into the STN after overnight withdrawal of dopaminergic medication. Intraoperative propofol was discontinued at least 30 minutes prior to the electrophysiological recordings. The authors used a novel approach for performing simultaneous recordings of STN local field potentials (LFPs) and multichannel electroencephalography (EEG) at rest. Coherent oscillations between LFP and EEG sensors were computed, and subsequent dynamic imaging of coherent sources was performed.The authors identified coherent activity in the upper beta range (21-35 Hz) between the STN and the ipsilateral mesial (pre)motor area. Coherence in the theta range (4-6 Hz) was detected in the ipsilateral prefrontal area.These findings demonstrate the feasibility of detecting frequency-specific and spatially distinct synchronization between the STN and cortex during DBS surgery. Mapping the STN with this technique may disentangle different functional loops relevant for refined targeting during DBS implantation.

Spike sorting based on shape, phase, and distribution features, and K-TOPS clustering with validity and error indices

Spike sorting is one of the most important data analysis problems in neurophysiology. The precision in all steps of the spike-sorting procedure critically affects the accuracy of all subsequent analyses. After data preprocessing and spike detection have been carried out properly, both feature extraction and spike clustering are the most critical subsequent steps of the spike-sorting procedure. The proposed spike sorting approach comprised a new feature extraction method based on shape, phase, and distribution features of each spike (hereinafter SS-SPDF method), which reveal significant information of the neural events under study. In addition, we applied an efficient clustering algorithm based on K-means and template optimization in phase space (hereinafter K-TOPS) that included two integrative clustering measures (validity and error indices) to verify the cohesion-dispersion among spike events during classification and the misclassification of clustering, respectively. The proposed method/algorithm was tested on both simulated data and real neural recordings. The results obtained for these datasets suggest that our spike sorting approach provides an efficient way for sorting both single-unit spikes and overlapping waveforms. By analyzing raw extracellular recordings collected from the rostral-medial prefrontal cortex (rmPFC) of behaving rabbits during classical eyeblink conditioning, we have demonstrated that the present method/algorithm performs better at classifying spikes and neurons and at assessing their modulating properties than other methods currently used in neurophysiology.

Identification of Retinal Ganglion Cell Firing Patterns Using Clustering Analysis Supplied with Failure Diagnosis

An important goal in visual neuroscience is to understand how neuronal population coding in vertebrate retina mediates the broad range of visual functions. Microelectrode arrays interface on isolated retina registers a collective measure of the spiking dynamics of retinal ganglion cells (RGCs) by probing them simultaneously and in large numbers. The recorded data stream is then processed to identify spike trains of individual RGCs by efficient and scalable spike detection and sorting routines. Most spike sorting software packages, available either commercially or as freeware, combine automated steps with judgment calls by the investigator to verify the quality of sorted spikes. This work focused on sorting spikes of RGCs into clusters using an integrated analytical platform for the data recorded during visual stimulation of wild-type mice retinas with whole field stimuli. After spike train detection, we projected each spike onto two feature spaces: a parametric space and a principal components space. We then applied clustering algorithms to sort spikes into separate clusters. To eliminate the need for human intervention, the initial clustering results were submitted to diagnostic tests that evaluated the results to detect the sources of failure in cluster assignment. This failure diagnosis formed a decision logic for diagnosable electrodes to enhance the clustering quality iteratively through rerunning the clustering algorithms. The new clustering results showed that the spike sorting accuracy was improved. Subsequently, the number of active RGCs during each whole field stimulation was found, and the light responsiveness of each RGC was identified. Our approach led to error-resilient spike sorting in both feature extraction methods; however, using parametric features led to less erroneous spike sorting compared to principal components, particularly for low signal-to-noise ratios. As our approach is reliable for retinal signal processing in response to simple visual stimuli, it could be applied to the evaluation of disrupted physiological signaling in retinal neurodegenerative diseases.

Simplified and Yet Turing Universal Spiking Neural P Systems with Communication on Request

Spiking neural P systems are a class of third generation neural networks belonging to the framework of membrane computing. Spiking neural P systems with communication on request (SNQ P systems) are a type of spiking neural P system where the spikes are requested from neighboring neurons. SNQ P systems have previously been proved to be universal (computationally equivalent to Turing machines) when two types of spikes are considered. This paper studies a simplified version of SNQ P systems, i.e. SNQ P systems with one type of spike. It is proved that one type of spike is enough to guarantee the Turing universality of SNQ P systems. Theoretical results are shown in the cases of the SNQ P system used in both generating and accepting modes. Furthermore, the influence of the number of unbounded neurons (the number of spikes in a neuron is not bounded) on the computation power of SNQ P systems with one type of spike is investigated. It is found that SNQ P systems functioning as number generating devices with one type of spike and four unbounded neurons are Turing universal.

Dynamic Redistribution of Plasticity in a Cerebellar Spiking Neural Network Reproducing an Associative Learning Task Perturbed by TMS

During natural learning, synaptic plasticity is thought to evolve dynamically and redistribute within and among subcircuits. This process should emerge in plastic neural networks evolving under behavioral feedback and should involve changes distributed across multiple synaptic sites. In eyeblink classical conditioning (EBCC), the cerebellum learns to predict the precise timing between two stimuli, hence EBCC represents an elementary yet meaningful paradigm to investigate the cerebellar network functioning. We have simulated EBCC mechanisms by reconstructing a realistic cerebellar microcircuit model and embedding multiple plasticity rules imitating those revealed experimentally. The model was tuned to fit experimental EBCC human data, estimating the underlying learning time-constants. Learning started rapidly with plastic changes in the cerebellar cortex followed by slower changes in the deep cerebellar nuclei. This process was characterized by differential development of long-term potentiation and depression at individual synapses, with a progressive accumulation of plasticity distributed over the whole network. The experimental data included two EBCC sessions interleaved by a trans-cranial magnetic stimulation (TMS). The experimental and the model response data were not significantly different in each learning phase, and the model goodness-of-fit was [Formula: see text] for all the experimental conditions. The models fitted on TMS data revealed a slowed down re-acquisition (sessions-2) compared to the control condition ([Formula: see text]). The plasticity parameters characterizing each model significantly differ among conditions, and thus mechanistically explain these response changes. Importantly, the model was able to capture the alteration in EBCC consolidation caused by TMS and showed that TMS affected plasticity at cortical synapses thereby altering the fast learning phase. This, secondarily, also affected plasticity in deep cerebellar nuclei altering learning dynamics in the entire sensory-motor loop. This observation reveals dynamic redistribution of changes over the entire network and suggests how TMS affects local circuit computation and memory processing in the cerebellum.

Spiking Neural P Systems with Communication on Request

Spiking Neural [Formula: see text] Systems are Neural System models characterized by the fact that each neuron mimics a biological cell and the communication between neurons is based on spikes. In the Spiking Neural [Formula: see text] systems investigated so far, the application of evolution rules depends on the contents of a neuron (checked by means of a regular expression). In these [Formula: see text] systems, a specified number of spikes are consumed and a specified number of spikes are produced, and then sent to each of the neurons linked by a synapse to the evolving neuron.

[Formula: see text]In the present work, a novel communication strategy among neurons of Spiking Neural [Formula: see text] Systems is proposed. In the resulting models, called Spiking Neural [Formula: see text] Systems with Communication on Request, the spikes are requested from neighboring neurons, depending on the contents of the neuron (still checked by means of a regular expression). Unlike the traditional Spiking Neural [Formula: see text] systems, no spikes are consumed or created: the spikes are only moved along synapses and replicated (when two or more neurons request the contents of the same neuron).

[Formula: see text]The Spiking Neural [Formula: see text] Systems with Communication on Request are proved to be computationally universal, that is, equivalent with Turing machines as long as two types of spikes are used. Following this work, further research questions are listed to be open problems.

Reliable Analysis of Single-Unit Recordings from the Human Brain under Noisy Conditions: Tracking Neurons over Hours

Recording extracellulary from neurons in the brains of animals in vivo is among the most established experimental techniques in neuroscience, and has recently become feasible in humans. Many interesting scientific questions can be addressed only when extracellular recordings last several hours, and when individual neurons are tracked throughout the entire recording. Such questions regard, for example, neuronal mechanisms of learning and memory consolidation, and the generation of epileptic seizures. Several difficulties have so far limited the use of extracellular multi-hour recordings in neuroscience: Datasets become huge, and data are necessarily noisy in clinical recording environments. No methods for spike sorting of such recordings have been available. Spike sorting refers to the process of identifying the contributions of several neurons to the signal recorded in one electrode. To overcome these difficulties, we developed Combinato: a complete data-analysis framework for spike sorting in noisy recordings lasting twelve hours or more. Our framework includes software for artifact rejection, automatic spike sorting, manual optimization, and efficient visualization of results. Our completely automatic framework excels at two tasks: It outperforms existing methods when tested on simulated and real data, and it enables researchers to analyze multi-hour recordings. We evaluated our methods on both short and multi-hour simulated datasets. To evaluate the performance of our methods in an actual neuroscientific experiment, we used data from from neurosurgical patients, recorded in order to identify visually responsive neurons in the medial temporal lobe. These neurons responded to the semantic content, rather than to visual features, of a given stimulus. To test our methods with multi-hour recordings, we made use of neurons in the human medial temporal lobe that respond selectively to the same stimulus in the evening and next morning.