Low-latency single channel real-time neural spike sorting system based on template matching

A machine learning approach for real-time cortical state estimation

Objective.Cortical function is under constant modulation by internally-driven, latent variables that regulate excitability, collectively known as 'cortical state'. Despite a vast literature in this area, the estimation of cortical state remains relatively ad hoc, and not amenable to real-time implementation. Here, we implement robust, data-driven, and fast algorithms that address several technical challenges for online cortical state estimation.Approach. We use unsupervised Gaussian mixture models to identify discrete, emergent clusters in spontaneous local field potential signals in cortex. We then extend our approach to a temporally-informed hidden semi-Markov model (HSMM) with Gaussian observations to better model and infer cortical state transitions. Finally, we implement our HSMM cortical state inference algorithms in a real-time system, evaluating their performance in emulation experiments.Main results. Unsupervised clustering approaches reveal emergent state-like structure in spontaneous electrophysiological data that recapitulate arousal-related cortical states as indexed by behavioral indicators. HSMMs enable cortical state inferences in a real-time context by modeling the temporal dynamics of cortical state switching. Using HSMMs provides robustness to state estimates arising from noisy, sequential electrophysiological data.Significance. To our knowledge, this work represents the first implementation of a real-time software tool for continuously decoding cortical states with high temporal resolution (40 ms). The software tools that we provide can facilitate our understanding of how cortical states dynamically modulate cortical function on a moment-by-moment basis and provide a basis for state-aware brain machine interfaces across health and disease.

An unsupervised real-time spike sorting system based on optimized OSort

Objective. The OSort algorithm, a pivotal unsupervised spike sorting method, has been implemented in dedicated hardware devices for real-time spike sorting. However, due to the inherent complexity of neural recording environments, OSort still grapples with numerous transient cluster occurrences during the practical sorting process. This leads to substantial memory usage, heavy computational load, and complex hardware architectures, especially in noisy recordings and multi-channel systems.Approach. This study introduces an optimized OSort algorithm (opt-OSort) which utilizes correlation coefficient (CC), instead of Euclidean distance as classification criterion. TheCCmethod not only bolsters the robustness of spike classification amidst the diverse and ever-changing conditions of physiological and recording noise environments, but also can finish the entire sorting procedure within a fixed number of cluster slots, thus preventing a large number of transient clusters. Moreover, the opt-OSort incorporates two configurable validation loops to efficiently reject cluster outliers and track recording variations caused by electrode drifting in real-time.Main results. The opt-OSort significantly reduces transient cluster occurrences by two orders of magnitude and decreases memory usage by 2.5-80 times in the number of pre-allocated transient clusters compared with other hardware implementations of OSort. The opt-OSort maintains an accuracy comparable to offline OSort and other commonly-used algorithms, with a sorting time of 0.68µs as measured by the hardware-implemented system in both simulated datasets and experimental data. The opt-OSort's ability to handle variations in neural activity caused by electrode drifting is also demonstrated.Significance. These results present a rapid, precise, and robust spike sorting solution suitable for integration into low-power, portable, closed-loop neural control systems and brain-computer interfaces.

Functional clustering of neuronal signals with FMM mixture models

The identification of unlabeled neuronal electric signals is one of the most challenging open problems in neuroscience, widely known as Spike Sorting. Motivated to solve this problem, we propose a model-based approach within the mixture modeling framework for clustering oscillatory functional data called MixFMM. The core of the approach is the FMM (Frequency Modulated Möbius) waves, which are non-linear parametric time functions, flexible enough to describe different oscillatory patterns and simple enough to be estimated efficiently. In particular, specific model parameters describe the phase, amplitude and shape of the waveforms. A mixture model is defined using FMM waves as basic functions and gaussian errors, and an EM algorithm is proposed for estimating the parameters. Spike Sorting (SS) has received considerable attention in the literature, and different functional clustering approaches have been considered. We have conducted a fair comparative analysis of the MixFMM with three competitors. Two of them are traditional methods in functional clustering and widely used in Spike Sorting. The third is an approach that has proven superior to many others solving Spike Sorting problems. The datasets used for validation include benchmarking simulated and real cases. The internal and external validation indexes confirm a better performance of the MixFMM on real data sets against the three competitors and an outstanding performance in simulated data against traditional approaches.

Edge computing on TPU for brain implant signal analysis

The ever-increasing number of recording sites of silicon-based probes imposes a great challenge for detecting and evaluating single-unit activities in an accurate and efficient manner. Currently separate solutions are available for high precision offline evaluation and separate solutions for embedded systems where computational resources are more limited. We propose a deep learning-based spike sorting system, that utilizes both unsupervised and supervised paradigms to learn a general feature embedding space and detect neural activity in raw data as well as predict the feature vectors for sorting. The unsupervised component uses contrastive learning to extract features from individual waveforms, while the supervised component is based on the MobileNetV2 architecture. One of the key advantages of our system is that it can be trained on multiple, diverse datasets simultaneously, resulting in greater generalizability than previous deep learning-based models. We demonstrate that the proposed model does not only reaches the accuracy of current state-of-art offline spike sorting methods but has the unique potential to run on edge Tensor Processing Units (TPUs), specialized chips designed for artificial intelligence and edge computing. We compare our model performance with state of art solutions on paired datasets as well as on hybrid recordings as well. The herein demonstrated system paves the way to the integration of deep learning-based spike sorting algorithms into wearable electronic devices, which will be a crucial element of high-end brain-computer interfaces.

A Neuromorphic Brain Interface based on RRAM Crossbar Arrays for High Throughput Real-time Spike Sorting

Real-time spike sorting and processing are crucial for closed-loop brain-machine interfaces and neural prosthetics. Recent developments in high-density multi-electrode arrays with hundreds of electrodes have enabled simultaneous recordings of spikes from a large number of neurons. However, the high channel count imposes stringent demands on real-time spike sorting hardware regarding data transmission bandwidth and computation complexity. Thus, it is necessary to develop a specialized real-time hardware that can sort neural spikes on the fly with high throughputs while consuming minimal power. Here, we present a real-time, low latency spike sorting processor that utilizes high-density CuO_x resistive crossbars to implement in-memory spike sorting in a massively parallel manner. We developed a fabrication process which is compatible with CMOS BEOL integration. We extensively characterized switching characteristics and statistical variations of the CuO_x memory devices. In order to implement spike sorting with crossbar arrays, we developed a template matching-based spike sorting algorithm that can be directly mapped onto RRAM crossbars. By using synthetic and in vivo recordings of extracellular spikes, we experimentally demonstrated energy efficient spike sorting with high accuracy. Our neuromorphic interface offers substantial improvements in area (~1000× less area), power (~200× less power), and latency (4.8μs latency for sorting 100 channels) for real-time spike sorting compared to other hardware implementations based on FPGAs and microcontrollers.

A Low Power 1024-Channels Spike Detector Using Latch-Based RAM for Real-Time Brain Silicon Interfaces

High-density microelectrode arrays allow the neuroscientist to study a wider neurons population, however, this causes an increase of communication bandwidth. Given the limited resources available for an implantable silicon interface, an on-fly data reduction is mandatory to stay within the power/area constraints. This can be accomplished by implementing a spike detector aiming at sending only the useful information about spikes. We show that the novel non-linear energy operator called ASO in combination with a simple but robust noise estimate, achieves a good trade-off between performance and consumption. The features of the investigated technique make it a good candidate for implantable BMIs. Our proposal is tested both on synthetic and real datasets providing a good sensibility at low SNR. We also provide a 1024-channels VLSI implementation using a Random-Access Memory composed by latches to reduce as much as possible the power consumptions. The final architecture occupies an area of 2.3 mm2, dissipating 3.6 µW per channels. The comparison with the state of art shows that our proposal finds a place among other methods presented in literature, certifying its suitability for BMIs.

Investigating well potential parameters on neural spike enhancement in a stochastic-resonance pre-emphasis algorithm

Objective.Background noise experienced during extracellular neural recording limits the number of spikes that can be reliably detected, which ultimately limits the performance of next-generation neuroscientific work. In this study, we aim to utilize stochastic resonance (SR), a technique that can help identify weak signals in noisy environments, to enhance spike detectability.Approach.Previously, an SR-based pre-emphasis algorithm was proposed, where a particle inside a 1D potential well is exerted by a force defined by the extracellular recording, and the output is obtained as the displacement of the particle. In this study, we investigate how the well shape and damping status impact the output signal-to-noise ratio (SNR). We compare the overdamped and underdamped solutions of shallow- and steep-wall monostable wells and bistable wells in terms of SNR improvement using two synthetic datasets. Then, we assess the spike detection performance when thresholding is applied on the output of the well shape-damping status configuration giving the best SNR enhancement.Main results.The SNR depends on the well-shape and damping-status type as well as the input noise level. The underdamped solution of the shallow-wall monostable well can yield to more than four orders of magnitude greater SNR improvement compared to other configurations for low noise intensities. Using this configuration also results in better spike detection sensitivity and positive predictivity than the state-of-the-art spike detection algorithms for a public synthetic dataset. For larger noise intensities, the overdamped solution of the steep-wall monostable well provides better spike enhancement than the others.Significance.The dependence of SNR improvement on the input signal noise level can be used to design a detector with multiple outputs, each more sensitive to a certain distance from the electrode. Such a detector can potentially enhance the performance of a successive spike sorting stage.

Computational approaches to decode grasping force and velocity level in upper-limb amputee from intraneural peripheral signals

Objective. Recent results have shown the potentials of neural interfaces to provide sensory feedback to subjects with limb amputation increasing prosthesis usability. However, their advantages for decoding motor control signals over current methods based on electromyography (EMG) are still debated. In this study we compared a standard EMG-based method with approaches that use peripheral intraneural data to infer distinct levels of grasping force and velocity in a trans-radial amputee.Approach. Surface EMG (three channels) and intraneural signals (collected with transverse intrafascicular multichannel electrodes, TIMEs, 56 channels) were simultaneously recorded during the amputee's intended grasping movements. We sorted single unit activity (SUA) from each neural signal and then we identified the most informative units. EMG envelopes were extracted from the recorded EMG signals. A reference support vector machine (SVM) classifier was used to map EMG envelopes into desired force and velocity levels. Two decoding approaches using SUA were then tested and compared to the EMG-based reference classifier: (a) SVM classification of firing rates into desired force and velocity levels; (b) reconstruction of covariates (the grasp cue level or EMG envelopes) from neural data and use of covariates for classification into desired force and velocity levels.Main results.Using EMG envelopes as reconstructed covariates from SUA yielded significantly better results than the other approaches tested, with performance similar to that of the EMG-based reference classifier, and stable over three different recording days. Of the two reconstruction algorithms used in this approach, a linear Kalman filter and a nonlinear point process adaptive filter, the nonlinear filter gave better results.Significance.This study presented a new effective approach for decoding grasping force and velocity from peripheral intraneural signals in a trans-radial amputee, which relies on using SUA to reconstruct EMG envelopes. Being dependent on EMG recordings only for the training phase, this approach can fully exploit the advantages of implanted neural interfaces and potentially overcome, in the medium to long term, current state-of-the-art methods. (Clinical trial's registration number: NCT02848846).