Massively parallel recordings in macaque motor cortex during an instructed delayed reach-to-grasp task

Spiking neural networks on FPGA: A survey of methodologies and recent advancements

The mimicry of the biological brain's structure in information processing enables spiking neural networks (SNNs) to exhibit significantly reduced power consumption compared to conventional systems. Consequently, these networks have garnered heightened attention and spurred extensive research endeavors in recent years, proposing various structures to achieve low power consumption, high speed, and improved recognition ability. However, researchers are still in the early stages of developing more efficient neural networks that more closely resemble the biological brain. This development and research require suitable hardware for execution with appropriate capabilities, and field-programmable gate array (FPGA) serves as a highly qualified candidate compared to existing hardware such as central processing unit (CPU) and graphics processing unit (GPU). FPGA, with parallel processing capabilities similar to the brain, lower latency and power consumption, and higher throughput, is highly eligible hardware for assisting in the development of spiking neural networks. In this review, an attempt has been made to facilitate researchers' path to further develop this field by collecting and examining recent works and the challenges that hinder the implementation of these networks on FPGA.

Time Varying Encoding of Grasping Type and Force in the Primate Motor Cortex

The primary motor cortex (M1) is strongly engaged by movement planning and execution. However, the role of M1 activity in voluntary grasping is still not completely understood. Here we analyze recordings of M1 neurons during the execution of a delayed reach-to-grasp task, where monkeys had to actively grasp an object with either a side or a precision grip, and then pull it with a low or high amount of force. Single cell and neural populations analyses showed that grip type was robustly and specifically encoded by a large population of neurons, while force level was weakly and transiently encoded within mixed-selective neurons that also encoded grip type. Notably, the grip type was stably decoded from motor cortical populations during the preparation and execution epochs of the task. Our results are consistent with the idea that planning and performing specific grasping movements are high-level skills that strongly engage M1 neurons, while the execution of pulling force might be prominently encoded at lower stages of the motor system.

A method for unsupervised learning of coherent spatiotemporal patterns in multiscale data

The unsupervised and principled diagnosis of multiscale data is a fundamental obstacle in modern scientific problems from, for instance, weather and climate prediction, neurology, epidemiology, and turbulence. Multiscale data are characterized by a combination of processes acting along multiple dimensions simultaneously, spatiotemporal scales across orders of magnitude, nonstationarity, and/or invariances such as translation and rotation. Existing methods are not well-suited to multiscale data, usually requiring supervised strategies such as human intervention, extensive tuning, or selection of ideal time periods. We present the multiresolution coherent spatio-temporal scale separation (mrCOSTS), a hierarchical and automated algorithm for the diagnosis of coherent patterns or modes in multiscale data. mrCOSTS is a variant of dynamic mode decomposition which decomposes data into bands of spatial patterns with shared time dynamics, thereby providing a robust method for analyzing multiscale data. It requires no training but instead takes advantage of the hierarchical nature of multiscale systems. We demonstrate mrCOSTS using complex multiscale datasets that are canonically difficult to analyze: 1) climate patterns of sea surface temperature, 2) electrophysiological observations of neural signals of the motor cortex, and 3) horizontal wind in the mountain boundary layer. With mrCOSTS, we trivially retrieve complex dynamics that were previously difficult to resolve while additionally extracting hitherto unknown patterns of activity embedded in the dynamics, allowing for advancing the understanding of these fields of study. This method is an important advancement for addressing the multiscale data which characterize many of the grand challenges in science and engineering.

An Efficient Brain-Switch for Asynchronous Brain-Computer Interfaces

Intracortical brain computer interfaces (iBCIs) utilizing extracellular recordings mainly employ in vivo signal processing application-specific integrated circuits (ASICs) to detect action potentials (spikes). Conventionally, "brain-switches" based on spiking activity have been employed to realize asynchronous (self-paced) iBCIs, estimating when the user involves in the underlying BCI task. Several studies have demonstrated that local field potentials (LFPs) can effectively replace action potentials, drastically reducing the power consumption and processing requirements of in vivo ASICs. This article presents the first LFP-based brain-switch design and implementation using gated recurrent neural networks (RNNs). Compared to the previously reported brain-switches, our design requires no exhaustive learning phase for the estimation of optimal recording channels or frequency band selection, making it more applicable to practical asynchronous iBCIs. The synthesized ASIC of the designed in vivo LFP-based feature extraction unit, in a standard 180-nm CMOS process, occupies only 0.09 mm of silicon area, and the post place-and-route synthesis results indicate that it consumes 91.87 nW of power while operating at 2 kHz. Compared to the previously published ASICs, the proposed LFP-based brain-switch consumes the least power for in vivo digital signal processing and achieves comparable state estimation performance to that of spike-based brain-switches.

A Power-and-Area-Efficient Channel-Interleaved Neural Signal Processor for Wireless Brain-Computer Interfaces With Unsupervised Spike Sorting

Next generation of wireless brain-computer-interface (BCI) devices require dedicated neural signal processors (NSPs) to extract key neurological information while operating within given power consumption and transmission bandwidth limits. Spike detection and clustering are important signal processing steps in neuroscience research and clinical applications. Computational-friendly spike detection and feature extraction algorithms are first systematically evaluated in this work. The nonlinear energy operator (NEO) and the first-and-second-derivative (FSDE) together with the 'perturbed' K-mean clustering achieve the highest accuracy performance. An NSP ASIC is implemented in a channel-interleaved architecture and the folding ratio of 16 leads to the minimum power-and-area product. As the result, the NSP consumes 2-µW power consumption and occupies 0.0057 mm2 for each channel in a 65-nm CMOS technology. The proposed system achieves the unsupervised spike classification accuracy of 92% and a data-rate reduction of 98.3%, showing the promise for realizing high-channel-count wireless BCIs.

Efficient in Vivo Neural Signal Compression Using an Autoencoder-Based Neural Network

Conventional in vivo neural signal processing involves extracting spiking activity within the recorded signals from an ensemble of neurons and transmitting only spike counts over an adequate interval. However, for brain-computer interface (BCI) applications utilizing continuous local field potentials (LFPs) for cognitive decoding, the volume of neural data to be transmitted to a computer imposes relatively high data rate requirements. This is particularly true for BCIs employing high-density intracortical recordings with hundreds or thousands of electrodes. This article introduces the first autoencoder-based compression digital circuit for the efficient transmission of LFP neural signals. Various algorithmic and architectural-level optimizations are implemented to significantly reduce the computational complexity and memory requirements of the designed in vivo compression circuit. This circuit employs an autoencoder-based neural network, providing a robust signal reconstruction. The application-specific integrated circuit (ASIC) of the in vivo compression logic occupies the smallest silicon area and consumes the lowest power among the reported state-of-the-art compression ASICs. Additionally, it offers a higher compression rate and a superior signal-to-noise and distortion ratio.

Facilitating the Sharing of Electrophysiology Data Analysis Results Through In-Depth Provenance Capture

Scientific research demands reproducibility and transparency, particularly in data-intensive fields like electrophysiology. Electrophysiology data are typically analyzed using scripts that generate output files, including figures. Handling these results poses several challenges due to the complexity and iterative nature of the analysis process. These stem from the difficulty to discern the analysis steps, parameters, and data flow from the results, making knowledge transfer and findability challenging in collaborative settings. Provenance information tracks data lineage and processes applied to it, and provenance capture during the execution of an analysis script can address those challenges. We present Alpaca (Automated Lightweight Provenance Capture), a tool that captures fine-grained provenance information with minimal user intervention when running data analysis pipelines implemented in Python scripts. Alpaca records inputs, outputs, and function parameters and structures information according to the W3C PROV standard. We demonstrate the tool using a realistic use case involving multichannel local field potential recordings of a neurophysiological experiment, highlighting how the tool makes result details known in a standardized manner in order to address the challenges of the analysis process. Ultimately, using Alpaca will help to represent results according to the FAIR principles, which will improve research reproducibility and facilitate sharing the results of data analyses.

Brain Feature Extraction With an Artifact-Tolerant Multiplexed Time-Encoding Neural Frontend for True Real-Time Closed-Loop Neuromodulation

Closed-loop neuromodulation is emerging as a more effective and targeted solution for the treatment of neurological symptoms compared to traditional open-loop stimulation. The majority of the present designs lack the ability to continuously record brain activity during electrical stimulation; hence they cannot fully monitor the treatment's effectiveness. This is due to the large stimulation artifacts that can saturate the sensitive readout circuits. To overcome this challenge, this work presents a rapid-artifact-recovery time-multiplexed neural readout frontend in combination with backend linear interpolation to reconstruct the artifact-corrupted local field potentials' (LFP) features. Our hybrid technique is an alternative approach to avoid power-hungry large-dynamic-range readout architectures or large and complex artifact template subtraction circuits. We discuss the design and measurements of a prototype implementation of the proposed readout frontend in 180-nm CMOS. It combines time multiplexing and time-domain conversion in a novel 13-bit incremental ADC, requiring only 0.0018 mm2/channel of readout area despite the large 180-nm CMOS process used, while consuming only 4.51 μW/channel. This is the smallest reported area for stimulation-voltage-compatible technologies (i.e. ≥ 65 nm). The frontend also yields a best-in-class peak total harmonic distortion of -72.6 dB @2.5-mVpp input, thanks to its implicit DAC mismatch-error shaping property. We employ the chip to measure brain LFP signals corrupted with artifacts, then perform linear interpolation and feature extraction on the measured signals and evaluate the reconstruction quality, using a set of sixteen commonly used features and three stimulation scenarios. The results show relative accuracies above 95% with respect to the situation without artifacts. This work is an ideal candidate for integration in high-channel-count true closed-loop neuromodulation systems.

Neural Decoding for Intracortical Brain-Computer Interfaces

Brain-computer interfaces have revolutionized the field of neuroscience by providing a solution for paralyzed patients to control external devices and improve the quality of daily life. To accurately and stably control effectors, it is important for decoders to recognize an individual's motor intention from neural activity either by noninvasive or intracortical neural recording. Intracortical recording is an invasive way of measuring neural electrical activity with high temporal and spatial resolution. Herein, we review recent developments in neural signal decoding methods for intracortical brain-computer interfaces. These methods have achieved good performance in analyzing neural activity and controlling robots and prostheses in nonhuman primates and humans. For more complex paradigms in motor rehabilitation or other clinical applications, there remains more space for further improvements of decoders.

Spatiotemporal spike-centered averaging reveals symmetry of temporal and spatial components of the spike-LFP relationship during human focal seizures

The electrographic manifestation of neural activity can reflect the relationship between the faster action potentials of individual neurons and the slower fluctuations of the local field potential (LFP). This relationship is typically examined in the temporal domain using the spike-triggered average. In this study, we add a spatial component to this relationship. Here we first derive a theoretical model of the spike-LFP relationship across a macroelectrode. This mathematical derivation showed a special symmetry in the spike-LFP relationship wherein a sinc function in the temporal domain predicts a sinc function in the spatial domain. We show that this theoretical result is observed in a real-world system by characterizing the spike-LFP relationship using microelectrode array (MEA) recordings of human focal seizures. To do this, we present a approach, termed the spatiotemporal spike-centered average (st-SCA), that allows for visualization of the spike-LFP relationship in both the temporal and spatial domains. We applied this method to 25 MEA recordings obtained from seven patients with pharmacoresistant focal epilepsy. Of the five patients with MEAs implanted in recruited territory, three exhibited spatiotemporal patterns consistent with a sinc function, and two exhibited spatiotemporal patterns resembling deep wells of excitation. These results suggest that in some cases characterization of the spike-LFP relationship in the temporal domain is sufficient to predict the underlying spatial pattern. Finally, we discuss the biological interpretation of these findings and propose that the sinc function may reflect the role of mid-range excitatory connections during seizure activity.

Beta oscillations and waves in motor cortex can be accounted for by the interplay of spatially structured connectivity and fluctuating inputs

The beta rhythm (13-30 Hz) is a prominent brain rhythm. Recordings in primates during instructed-delay reaching tasks have shown that different types of traveling waves of oscillatory activity are associated with episodes of beta oscillations in motor cortex during movement preparation. We propose here a simple model of motor cortex based on local excitatory-inhibitory neuronal populations coupled by long-range excitation, where additionally inputs to the motor cortex from other neural structures are represented by stochastic inputs on the different model populations. We show that the model accurately reproduces the statistics of recording data when these external inputs are correlated on a short time scale (25 ms) and have two different components, one that targets the motor cortex locally and another one that targets it in a global and synchronized way. The model reproduces the distribution of beta burst durations, the proportion of the different observed wave types, and wave speeds, which we show not to be linked to axonal propagation speed. When the long-range connectivity or the local input targets are anisotropic, traveling waves are found to preferentially propagate along the axis where connectivity decays the fastest. Different from previously proposed mechanistic explanations, the model suggests that traveling waves in motor cortex are the reflection of the dephasing by external inputs, putatively of thalamic origin, of an oscillatory activity that would otherwise be spatially synchronized by recurrent connectivity.

Parallel movement planning is achieved via an optimal preparatory state in motor cortex

How do patterns of neural activity in the motor cortex contribute to the planning of a movement? A recent theory developed for single movements proposes that the motor cortex acts as a dynamical system whose initial state is optimized during the preparatory phase of the movement. This theory makes important yet untested predictions about preparatory dynamics in more complex behavioral settings. Here, we analyze preparatory activity in non-human primates planning not one but two movements simultaneously. As predicted by the theory, we find that parallel planning is achieved by adjusting preparatory activity within an optimal subspace to an intermediate state reflecting a trade-off between the two movements. The theory quantitatively accounts for the relationship between this intermediate state and fluctuations in the animals' behavior down at the trial level. These results uncover a simple mechanism for planning multiple movements in parallel and further point to motor planning as a controlled dynamical process.

Improving the accuracy of decoding monkey brain-machine interface data by estimating the state of unobserved cell assemblies

BACKGROUND

The brain-machine interface is a technology that has been used for improving the quality of life of individuals with physical disabilities and also healthy individuals. It is important to improve the methods used for decoding the brain-machine interface data as the accuracy and speed of movements achieved using the existing technology are not comparable to the normal body.

COMPARISON WITH THE EXISTING METHOD

Decoding of brain-machine interface data using the proposed method resulted in improved decoding accuracy compared to the existing method.

CONCLUSIONS

The results demonstrated the usefulness of cell assembly state estimation method for decoding the brain-machine interface data.

NEW METHOD

We incorporated a novel method of estimating cell assembly states using spike trains with the existing decoding method that used only firing rate data. Synaptic connectivity pattern was used as feature values in addition to firing rate. Publicly available monkey brain-machine interface datasets were used in the study.

RESULTS

As long as the decoding was successful, the root mean square error of the proposed method was significantly smaller than the existing method. Artificial neural netowork-based decoding method resulted in more stable decoding, and also improved the decoding accuracy due to incorporation of synaptic connectivity pattern.

Power-efficientin vivobrain-machine interfaces via brain-state estimation

Objective.Advances in brain-machine interfaces (BMIs) can potentially improve the quality of life of millions of users with spinal cord injury or other neurological disorders by allowing them to interact with the physical environment at their will.Approach.To reduce the power consumption of the brain-implanted interface, this article presents the first hardware realization of anin vivointention-aware interface via brain-state estimation.Main Results.It is shown that incorporating brain-state estimation reduces thein vivopower consumption and reduces total energy dissipation by over 1.8× compared to those of the current systems, enabling longer better life for implanted circuits. The synthesized application-specific integrated circuit (ASIC) of the designed intention-aware multi-unit spike detection system in a standard 180 nm CMOS process occupies 0.03 mm²of silicon area and consumes 0.63 µW of power per channel, which is the least power consumption among the currentin vivoASIC realizations.Significance.The proposed interface is the first practical approach towards realizing asynchronous BMIs while reducing the power consumption of the BMI interface and enhancing neural decoding performance compared to those of the conventional synchronous BMIs.

Evaluating the statistical similarity of neural network activity and connectivity via eigenvector angles

Neural systems are networks, and strategic comparisons between multiple networks are a prevalent task in many research scenarios. In this study, we construct a statistical test for the comparison of matrices representing pairwise aspects of neural networks, in particular, the correlation between spiking activity and connectivity. The "eigenangle test" quantifies the similarity of two matrices by the angles between their ranked eigenvectors. We calibrate the behavior of the test for use with correlation matrices using stochastic models of correlated spiking activity and demonstrate how it compares to classical two-sample tests, such as the Kolmogorov-Smirnov distance, in the sense that it is able to evaluate also structural aspects of pairwise measures. Furthermore, the principle of the eigenangle test can be applied to compare the similarity of adjacency matrices of certain types of networks. Thus, the approach can be used to quantitatively explore the relationship between connectivity and activity with the same metric. By applying the eigenangle test to the comparison of connectivity matrices and correlation matrices of a random balanced network model before and after a specific synaptic rewiring intervention, we gauge the influence of connectivity features on the correlated activity. Potential applications of the eigenangle test include simulation experiments, model validation, and data analysis.

Thermal Probing Techniques for a Single Live Cell

Temperature is a significant factor in determining and characterizing cellular metabolism and other biochemical activities. In this study, we provide a brief overview of two important technologies used to monitor the local temperatures of individual living cells: fluorescence nano-thermometry and an array of micro-/nano-sized thin-film thermocouples. We explain some key technical issues that must be addressed and optimised for further practical applications, such as in cell biology, drug selection, and novel antitumor therapy. We also offer a method for combining them into a hybrid measuring system.

From End to End: Gaining, Sorting, and Employing High-Density Neural Single Unit Recordings

The meaning behind neural single unit activity has constantly been a challenge, so it will persist in the foreseeable future. As one of the most sourced strategies, detecting neural activity in high-resolution neural sensor recordings and then attributing them to their corresponding source neurons correctly, namely the process of spike sorting, has been prevailing so far. Support from ever-improving recording techniques and sophisticated algorithms for extracting worthwhile information and abundance in clustering procedures turned spike sorting into an indispensable tool in electrophysiological analysis. This review attempts to illustrate that in all stages of spike sorting algorithms, the past 5 years innovations' brought about concepts, results, and questions worth sharing with even the non-expert user community. By thoroughly inspecting latest innovations in the field of neural sensors, recording procedures, and various spike sorting strategies, a skeletonization of relevant knowledge lays here, with an initiative to get one step closer to the original objective: deciphering and building in the sense of neural transcript.

Group-level inference of information-based measures for the analyses of cognitive brain networks from neurophysiological data

The reproducibility crisis in neuroimaging and in particular in the case of underpowered studies has introduced doubts on our ability to reproduce, replicate and generalize findings. As a response, we have seen the emergence of suggested guidelines and principles for neuroscientists known as Good Scientific Practice for conducting more reliable research. Still, every study remains almost unique in its combination of analytical and statistical approaches. While it is understandable considering the diversity of designs and brain data recording, it also represents a striking point against reproducibility. Here, we propose a non-parametric permutation-based statistical framework, primarily designed for neurophysiological data, in order to perform group-level inferences on non-negative measures of information encompassing metrics from information-theory, machine-learning or measures of distances. The framework supports both fixed- and random-effect models to adapt to inter-individuals and inter-sessions variability. Using numerical simulations, we compared the accuracy in ground-truth retrieving of both group models, such as test- and cluster-wise corrections for multiple comparisons. We then reproduced and extended existing results using both spatially uniform MEG and non-uniform intracranial neurophysiological data. We showed how the framework can be used to extract stereotypical task- and behavior-related effects across the population covering scales from the local level of brain regions, inter-areal functional connectivity to measures summarizing network properties. We also present an open-source Python toolbox called Frites¹ that includes the proposed statistical pipeline using information-theoretic metrics such as single-trial functional connectivity estimations for the extraction of cognitive brain networks. Taken together, we believe that this framework deserves careful attention as its robustness and flexibility could be the starting point toward the uniformization of statistical approaches.

Electrophysiological dataset from macaque visual cortical area MST in response to a novel motion stimulus

Establishing the cortical neural representation of visual stimuli is a central challenge of systems neuroscience. Publicly available data would allow a broad range of scientific analyses and hypothesis testing, but are rare and largely focused on the early visual system. To address the shortage of open data from higher visual areas, we provide a comprehensive dataset from a neurophysiology study in macaque monkey visual cortex that includes a complete record of extracellular action potential recordings from the extrastriate medial superior temporal (MST) area, behavioral data, and detailed stimulus records. It includes spiking activity of 172 single neurons recorded in 139 sessions from 4 hemispheres of 3 rhesus macaque monkeys. The data was collected across 3 experiments, designed to characterize the response properties of MST neurons to complex motion stimuli. This data can be used to elucidate visual information processing at the level of single neurons in a high-level area of primate visual cortex. Providing open access to this dataset also promotes the 3R-principle of responsible animal research.

Measurement(s)	spike train • eye movement measurement
Technology Type(s)	single-unit recording • eye tracking device
Factor Type(s)	direction, location, and speed of moving random dot patterns
Sample Characteristic - Organism	Macaca mulatta
Sample Characteristic - Environment	laboratory environment
Sample Characteristic - Location	Germany

1024-channel electrophysiological recordings in macaque V1 and V4 during resting state

Co-variations in resting state activity are thought to arise from a variety of correlated inputs to neurons, such as bottom-up activity from lower areas, feedback from higher areas, recurrent processing in local circuits, and fluctuations in neuromodulatory systems. Most studies have examined resting state activity throughout the brain using MRI scans, or observed local co-variations in activity by recording from a small number of electrodes. We carried out electrophysiological recordings from over a thousand chronically implanted electrodes in the visual cortex of non-human primates, yielding a resting state dataset with unprecedentedly high channel counts and spatiotemporal resolution. Such signals could be used to observe brain waves across larger regions of cortex, offering a temporally detailed picture of brain activity. In this paper, we provide the dataset, describe the raw and processed data formats and data acquisition methods, and indicate how the data can be used to yield new insights into the ‘background’ activity that influences the processing of visual information in our brain.

Measurement(s)	brain activity measurement • eye movement
Technology Type(s)	extracellular electrophysiology recording • multi-electrode array system • infrared camera
Factor Type(s)	visual stimulation
Sample Characteristic - Organism	Macaca mulatta

Soft Devices for High-Resolution Neuro-Stimulation: The Interplay Between Low-Rigidity and Resolution

The field of neurostimulation has evolved over the last few decades from a crude, low-resolution approach to a highly sophisticated methodology entailing the use of state-of-the-art technologies. Neurostimulation has been tested for a growing number of neurological applications, demonstrating great promise and attracting growing attention in both academia and industry. Despite tremendous progress, long-term stability of the implants, their large dimensions, their rigidity and the methods of their introduction and anchoring to sensitive neural tissue remain challenging. The purpose of this review is to provide a concise introduction to the field of high-resolution neurostimulation from a technological perspective and to focus on opportunities stemming from developments in materials sciences and engineering to reduce device rigidity while optimizing electrode small dimensions. We discuss how these factors may contribute to smaller, lighter, softer and higher electrode density devices.

Global organization of neuronal activity only requires unstructured local connectivity

Modern electrophysiological recordings simultaneously capture single-unit spiking activities of hundreds of neurons spread across large cortical distances. Yet, this parallel activity is often confined to relatively low-dimensional manifolds. This implies strong coordination also among neurons that are most likely not even connected. Here, we combine in vivo recordings with network models and theory to characterize the nature of mesoscopic coordination patterns in macaque motor cortex and to expose their origin: We find that heterogeneity in local connectivity supports network states with complex long-range cooperation between neurons that arises from multi-synaptic, short-range connections. Our theory explains the experimentally observed spatial organization of covariances in resting state recordings as well as the behaviorally related modulation of covariance patterns during a reach-to-grasp task. The ubiquity of heterogeneity in local cortical circuits suggests that the brain uses the described mechanism to flexibly adapt neuronal coordination to momentary demands.

Acceleration of the SPADE Method Using a Custom-Tailored FP-Growth Implementation

The SPADE (spatio-temporal Spike PAttern Detection and Evaluation) method was developed to find reoccurring spatio-temporal patterns in neuronal spike activity (parallel spike trains). However, depending on the number of spike trains and the length of recording, this method can exhibit long runtimes. Based on a realistic benchmark data set, we identified that the combination of pattern mining (using the FP-Growth algorithm) and the result filtering account for 85–90% of the method's total runtime. Therefore, in this paper, we propose a customized FP-Growth implementation tailored to the requirements of SPADE, which significantly accelerates pattern mining and result filtering. Our version allows for parallel and distributed execution, and due to the improvements made, an execution on heterogeneous and low-power embedded devices is now also possible. The implementation has been evaluated using a traditional workstation based on an Intel Broadwell Xeon E5-1650 v4 as a baseline. Furthermore, the heterogeneous microserver platform RECS|Box has been used for evaluating the implementation on two HiSilicon Hi1616 (Kunpeng 916), an Intel Coffee Lake-ER Xeon E-2276ME, an Intel Broadwell Xeon D-D1577, and three NVIDIA Tegra devices (Jetson AGX Xavier, Jetson Xavier NX, and Jetson TX2). Depending on the platform, our implementation is between 27 and 200 times faster than the original implementation. At the same time, the energy consumption was reduced by up to two orders of magnitude.

Selection of Essential Neural Activity Timesteps for Intracortical Brain-Computer Interface Based on Recurrent Neural Network

Intracortical brain–computer interfaces (iBCIs) translate neural activity into control commands, thereby allowing paralyzed persons to control devices via their brain signals. Recurrent neural networks (RNNs) are widely used as neural decoders because they can learn neural response dynamics from continuous neural activity. Nevertheless, excessively long or short input neural activity for an RNN may decrease its decoding performance. Based on the temporal attention module exploiting relations in features over time, we propose a temporal attention-aware timestep selection (TTS) method that improves the interpretability of the salience of each timestep in an input neural activity. Furthermore, TTS determines the appropriate input neural activity length for accurate neural decoding. Experimental results show that the proposed TTS efficiently selects 28 essential timesteps for RNN-based neural decoders, outperforming state-of-the-art neural decoders on two nonhuman primate datasets (R2=0.76±0.05 for monkey Indy and CC=0.91±0.01 for monkey N). In addition, it reduces the computation time for offline training (reducing 5–12%) and online prediction (reducing 16–18%). When visualizing the attention mechanism in TTS, the preparatory neural activity is consecutively highlighted during arm movement, and the most recent neural activity is highlighted during the resting state in nonhuman primates. Selecting only a few essential timesteps for an RNN-based neural decoder provides sufficient decoding performance and requires only a short computation time.

Inferring entire spiking activity from local field potentials

Extracellular recordings are typically analysed by separating them into two distinct signals: local field potentials (LFPs) and spikes. Previous studies have shown that spikes, in the form of single-unit activity (SUA) or multiunit activity (MUA), can be inferred solely from LFPs with moderately good accuracy. SUA and MUA are typically extracted via threshold-based technique which may not be reliable when the recordings exhibit a low signal-to-noise ratio (SNR). Another type of spiking activity, referred to as entire spiking activity (ESA), can be extracted by a threshold-less, fast, and automated technique and has led to better performance in several tasks. However, its relationship with the LFPs has not been investigated. In this study, we aim to address this issue by inferring ESA from LFPs intracortically recorded from the motor cortex area of three monkeys performing different tasks. Results from long-term recording sessions and across subjects revealed that ESA can be inferred from LFPs with good accuracy. On average, the inference performance of ESA was consistently and significantly higher than those of SUA and MUA. In addition, local motor potential (LMP) was found to be the most predictive feature. The overall results indicate that LFPs contain substantial information about spiking activity, particularly ESA. This could be useful for understanding LFP-spike relationship and for the development of LFP-based BMIs.

On the Complexity of Resting State Spiking Activity in Monkey Motor Cortex

Resting state has been established as a classical paradigm of brain activity studies, mostly based on large-scale measurements such as functional magnetic resonance imaging or magneto- and electroencephalography. This term typically refers to a behavioral state characterized by the absence of any task or stimuli. The corresponding neuronal activity is often called idle or ongoing. Numerous modeling studies on spiking neural networks claim to mimic such idle states, but compare their results with task- or stimulus-driven experiments, or to results from experiments with anesthetized subjects. Both approaches might lead to misleading conclusions. To provide a proper basis for comparing physiological and simulated network dynamics, we characterize simultaneously recorded single neurons' spiking activity in monkey motor cortex at rest and show the differences from spontaneous and task- or stimulus-induced movement conditions. We also distinguish between rest with open eyes and sleepy rest with eyes closed. The resting state with open eyes shows a significantly higher dimensionality, reduced firing rates, and less balance between population level excitation and inhibition than behavior-related states.

Robust and accurate decoding of hand kinematics from entire spiking activity using deep learning

OBJECTIVE

Brain-machine interfaces (BMIs) seek to restore lost motor functions in individuals with neurological disorders by enabling them to control external devices directly with their thoughts. This work aims to improve robustness and decoding accuracy that currently become major challenges in the clinical translation of intracortical BMIs.

APPROACH

We propose entire spiking activity (ESA) -an envelope of spiking activity that can be extracted by a simple, threshold-less, and automated technique- as the input signal. We couple ESA with deep learning-based decoding algorithm that uses quasi-recurrent neural network (QRNN) architecture. We evaluate comprehensively the performance of ESA-driven QRNN decoder for decoding hand kinematics from neural signals chronically recorded from the primary motor cortex area of three non-human primates performing different tasks.

MAIN RESULTS

Our proposed method yields consistently higher decoding performance than any other combinations of the input signal and decoding algorithm previously reported across long term recording sessions. It can sustain high decoding performance even when removing spikes from the raw signals, when using the different number of channels, and when using a smaller amount of training data.

SIGNIFICANCE

Overall results demonstrate exceptionally high decoding accuracy and chronic robustness, which is highly desirable given it is an unresolved challenge in BMIs.

A long-term monitoring dataset of fish assemblages in rocky tidepools on the northern coast of Taiwan

The assemblage of fish species in the rocky intertidal zone is highly affected by the destructive impact of human activities and has an extended impact on land-sea interactions. There are a few long-term research projects that have focused on rocky intertidal ecosystems, especially on the resident fish community. Here, we describe a long-term time series dataset of fish collected by counting the number of anesthetized fishes at sampling stations in rocky tidepools in the intertidal zones on the northern coast of Taiwan. The species assemblages were monitored seasonally at three stations from 1999 to 2018. In total, 144 samples containing 1,577 individuals belonging to 106 species were recorded in the surveys. The resulting data can be used as background information for conservation and resilience studies of the fish community in coastal areas and to establish reasonable conservation strategies. This study presents valuable data to ecologists and fisheries biologists interested in understanding the temporal patterns of species abundance, richness, and composition in relation to environmental factors, climate change, and anthropogenic pressures.

Measurement(s)	Fish
Technology Type(s)	Manual Count
Factor Type(s)	time of data collection on species assemblages
Sample Characteristic - Environment	tidal pool • coast
Sample Characteristic - Location	Taiwan Island

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.11898174

CMOS-based bio-image sensor spatially resolves neural activity-dependent proton dynamics in the living brain

Recent studies have shown that protons can function as neurotransmitters in cultured neurons. To further investigate regional and neural activity-dependent proton dynamics in the brain, the development of a device with both wide-area detectability and high spatial-ltemporal resolution is necessary. Therefore, we develop an image sensor with a high spatial-temporal resolution specifically designed for measuring protons in vivo. Here, we demonstrate that spatially deferent neural stimulation by visual stimulation induced distinct patterns of proton changes in the visual cortex. This result indicates that our biosensor can detect micrometer and millisecond scale changes of protons across a wide area. Our study demonstrates that a CMOS-based proton image sensor with high spatial and temporal precision can be used to detect pH changes associated with biological events. We believe that our sensor may have broad applicability in future biological studies.

Protons have been discovered to play a role in neuronal signaling, but current methods to measure pH in the brain of animal models are limited. Here the authors develop a miniaturized proton image sensor that fits into a living mouse brain and can measure pH changes at micrometer and millisecond resolution scales.

odMLtables: A User-Friendly Approach for Managing Metadata of Neurophysiological Experiments

An essential aspect of scientific reproducibility is a coherent and complete acquisition of metadata along with the actual data of an experiment. The high degree of complexity and heterogeneity of neuroscience experiments requires a rigorous management of the associated metadata. The odML framework represents a solution to organize and store complex metadata digitally in a hierarchical format that is both human and machine readable. However, this hierarchical representation of metadata is difficult to handle when metadata entries need to be collected and edited manually during the daily routines of a laboratory. With odMLtables, we present an open-source software solution that enables users to collect, manipulate, visualize, and store metadata in tabular representations (in xls or csv format) by providing functionality to convert these tabular collections to the hierarchically structured metadata format odML, and to either extract or merge subsets of a complex metadata collection. With this, odMLtables bridges the gap between handling metadata in an intuitive way that integrates well with daily lab routines and commonly used software products on the one hand, and the implementation of a complete, well-defined metadata collection for the experiment in a standardized format on the other hand. We demonstrate usage scenarios of the odMLtables tools in common lab routines in the context of metadata acquisition and management, and show how the tool can assist in exploring published datasets that provide metadata in the odML format.

Second type of criticality in the brain uncovers rich multiple-neuron dynamics

Cortical networks that have been found to operate close to a critical point exhibit joint activations of large numbers of neurons. However, in motor cortex of the awake macaque monkey, we observe very different dynamics: massively parallel recordings of 155 single-neuron spiking activities show weak fluctuations on the population level. This a priori suggests that motor cortex operates in a noncritical regime, which in models, has been found to be suboptimal for computational performance. However, here, we show the opposite: The large dispersion of correlations across neurons is the signature of a second critical regime. This regime exhibits a rich dynamical repertoire hidden from macroscopic brain signals but essential for high performance in such concepts as reservoir computing. An analytical link between the eigenvalue spectrum of the dynamics, the heterogeneity of connectivity, and the dispersion of correlations allows us to assess the closeness to the critical point.

Reproducible Neural Network Simulations: Statistical Methods for Model Validation on the Level of Network Activity Data

Computational neuroscience relies on simulations of neural network models to bridge the gap between the theory of neural networks and the experimentally observed activity dynamics in the brain. The rigorous validation of simulation results against reference data is thus an indispensable part of any simulation workflow. Moreover, the availability of different simulation environments and levels of model description require also validation of model implementations against each other to evaluate their equivalence. Despite rapid advances in the formalized description of models, data, and analysis workflows, there is no accepted consensus regarding the terminology and practical implementation of validation workflows in the context of neural simulations. This situation prevents the generic, unbiased comparison between published models, which is a key element of enhancing reproducibility of computational research in neuroscience. In this study, we argue for the establishment of standardized statistical test metrics that enable the quantitative validation of network models on the level of the population dynamics. Despite the importance of validating the elementary components of a simulation, such as single cell dynamics, building networks from validated building blocks does not entail the validity of the simulation on the network scale. Therefore, we introduce a corresponding set of validation tests and present an example workflow that practically demonstrates the iterative model validation of a spiking neural network model against its reproduction on the SpiNNaker neuromorphic hardware system. We formally implement the workflow using a generic Python library that we introduce for validation tests on neural network activity data. Together with the companion study (Trensch et al., 2018), the work presents a consistent definition, formalization, and implementation of the verification and validation process for neural network simulations.

VIOLA-A Multi-Purpose and Web-Based Visualization Tool for Neuronal-Network Simulation Output

Neuronal network models and corresponding computer simulations are invaluable tools to aid the interpretation of the relationship between neuron properties, connectivity, and measured activity in cortical tissue. Spatiotemporal patterns of activity propagating across the cortical surface as observed experimentally can for example be described by neuronal network models with layered geometry and distance-dependent connectivity. In order to cover the surface area captured by today's experimental techniques and to achieve sufficient self-consistency, such models contain millions of nerve cells. The interpretation of the resulting stream of multi-modal and multi-dimensional simulation data calls for integrating interactive visualization steps into existing simulation-analysis workflows. Here, we present a set of interactive visualization concepts called views for the visual analysis of activity data in topological network models, and a corresponding reference implementation VIOLA (VIsualization Of Layer Activity). The software is a lightweight, open-source, web-based, and platform-independent application combining and adapting modern interactive visualization paradigms, such as coordinated multiple views, for massively parallel neurophysiological data. For a use-case demonstration we consider spiking activity data of a two-population, layered point-neuron network model incorporating distance-dependent connectivity subject to a spatially confined excitation originating from an external population. With the multiple coordinated views, an explorative and qualitative assessment of the spatiotemporal features of neuronal activity can be performed upfront of a detailed quantitative data analysis of specific aspects of the data. Interactive multi-view analysis therefore assists existing data analysis workflows. Furthermore, ongoing efforts including the European Human Brain Project aim at providing online user portals for integrated model development, simulation, analysis, and provenance tracking, wherein interactive visual analysis tools are one component. Browser-compatible, web-technology based solutions are therefore required. Within this scope, with VIOLA we provide a first prototype.

Development, Modeling, Fabrication, and Characterization of a Magnetic, Micro-Spring-Suspended System for the Safe Electrical Interconnection of Neural Implants

The development of innovative tools for neuroscientific research is based on in vivo tests typically applied to small animals. Most often, the interfacing of neural probes relies on commercially available connector systems which are difficult to handle during connection, particularly when freely behaving animals are involved. Furthermore, the connectors often exert high mechanical forces during plugging and unplugging, potentially damaging the fragile bone structure. In order to facilitate connector usage and increase the safety of laboratory animals, we developed a new magnetic connector system circumventing the drawbacks of existing tools. The connector system uses multiple magnet pairs and spring-suspended electrical contact pads realized using micro-electromechanical systems (MEMS) technologies. While the contact pad suspension increases the system tolerance in view of geometrical variations, we achieved a reliable self-alignment of the connector parts at ±50 µm provided by the specifically oriented magnet pairs and without the need of alignment pins. While connection forces are negligible, we can adjust the forces during connector release by modifying the magnet distance. With the connector test structures developed here, we achieved an electrical connection yield of 100%. Based on these findings, we expect that in vivo experiments with freely behaving animals will be facilitated with improved animal safety.

Behavioral Context Determines Network State and Variability Dynamics in Monkey Motor Cortex

Variability of spiking activity is ubiquitous throughout the brain but little is known about its contextual dependance. Trial-to-trial spike count variability, estimated by the Fano Factor (FF), and within-trial spike time irregularity, quantified by the coefficient of variation (CV), reflect variability on long and short time scales, respectively. We co-analyzed FF and the local coefficient of variation (CV2) in monkey motor cortex comparing two behavioral contexts, movement preparation (wait) and execution (movement). We find that the FF significantly decreases from wait to movement, while the CV2 increases. The more regular firing (expressed by a low CV2) during wait is related to an increased power of local field potential (LFP) beta oscillations and phase locking of spikes to these oscillations. In renewal processes, a widely used model for spiking activity under stationary input conditions, both measures are related as FF ≈ CV². This expectation was met during movement, but not during wait where FF ≫ CV2². Our interpretation is that during movement preparation, ongoing brain processes result in changing network states and thus in high trial-to-trial variability (expressed by a high FF). During movement execution, the network is recruited for performing the stereotyped motor task, resulting in reliable single neuron output. Our interpretation is in the light of recent computational models that generate non-stationary network conditions.

LFP beta amplitude is linked to mesoscopic spatio-temporal phase patterns

Beta oscillations observed in motor cortical local field potentials (LFPs) recorded on separate electrodes of a multi-electrode array have been shown to exhibit non-zero phase shifts that organize into planar waves. Here, we generalize this concept to additional classes of salient patterns that fully describe the spatial organization of beta oscillations. During a delayed reach-to-grasp task we distinguish planar, synchronized, random, circular, and radial phase patterns in monkey primary motor and dorsal premotor cortices. We observe that patterns correlate with the beta amplitude (envelope): Coherent planar/radial wave propagation accelerates with growing amplitude, and synchronized patterns are observed at largest amplitudes. In contrast, incoherent random or circular patterns are observed almost exclusively when beta is strongly attenuated. The occurrence probability of a particular pattern modulates with behavioral epochs in the same way as beta amplitude: Coherent patterns are more present during movement preparation where amplitudes are large, while incoherent phase patterns are dominant during movement execution where amplitudes are small. Thus, we uncover a trigonal link between the spatial arrangement of beta phases, beta amplitude, and behavior. Together with previous findings, we discuss predictions on the spatio-temporal organization of precisely coordinated spiking on the mesoscopic scale as a function of beta power.