Extracellular voltage threshold settings can be tuned for optimal encoding of movement and stimulus parameters

Decoding the Spike-Band Subthreshold Motor Cortical Activity

Intracortical Brain-Computer Interfaces (iBCI) use single-unit activity (SUA), multiunit activity (MUA) and local field potentials (LFP) to control neuroprosthetic devices. SUA and MUA are usually extracted from the bandpassed recording through amplitude thresholding, while subthreshold data are ignored. Here, we show that subthreshold data can actually be decoded to determine behavioral variables with test set accuracy of up to 100%. Although the utility of SUA, MUA and LFP for decoding behavioral variables has been explored previously, this study investigates the utility of spike-band subthreshold activity exclusively. We provide evidence suggesting that this activity can be used to keep decoding performance at acceptable levels even when SUA quality is reduced over time. To the best of our knowledge, the signals that we derive from the subthreshold activity may be the weakest neural signals that have ever been extracted from extracellular neural recordings, while still being decodable with test set accuracy of up to 100%. These results are relevant for the development of fully data-driven and automated methods for amplitude thresholding spike-band extracellular neural recordings in iBCIs containing thousands of electrodes.

Calibration-Free and Hardware-Efficient Neural Spike Detection for Brain Machine Interfaces

Recent translational efforts in brain-machine interfaces (BMI) are demonstrating the potential to help people with neurological disorders. The current trend in BMI technology is to increase the number of recording channels to the thousands, resulting in the generation of vast amounts of raw data. This in turn places high bandwidth requirements for data transmission, which increases power consumption and thermal dissipation of implanted systems. On-implant compression and/or feature extraction are therefore becoming essential to limiting this increase in bandwidth, but add further power constraints - the power required for data reduction must remain less than the power saved through bandwidth reduction. Spike detection is a common feature extraction technique used for intracortical BMIs. In this article, we develop a novel firing-rate-based spike detection algorithm that requires no external training and is hardware efficient and therefore ideally suited for real-time applications. Key performance and implementation metrics such as detection accuracy, adaptability in chronic deployment, power consumption, area utilization, and channel scalability are benchmarked against existing methods using various datasets. The algorithm is first validated using a reconfigurable hardware (FPGA) platform and then ported to a digital ASIC implementation in both 65 nm and 0.18 μm CMOS technologies. The 128-channel ASIC design implemented in a 65 nm CMOS technology occupies 0.096 mm2 silicon area and consumes 4.86 μW from a 1.2 V power supply. The adaptive algorithm achieves a 96% spike detection accuracy on a commonly used synthetic dataset, without the need for any prior training.

Power-efficientin vivobrain-machine interfaces via brain-state estimation

Objective.Advances in brain-machine interfaces (BMIs) can potentially improve the quality of life of millions of users with spinal cord injury or other neurological disorders by allowing them to interact with the physical environment at their will.Approach.To reduce the power consumption of the brain-implanted interface, this article presents the first hardware realization of anin vivointention-aware interface via brain-state estimation.Main Results.It is shown that incorporating brain-state estimation reduces thein vivopower consumption and reduces total energy dissipation by over 1.8× compared to those of the current systems, enabling longer better life for implanted circuits. The synthesized application-specific integrated circuit (ASIC) of the designed intention-aware multi-unit spike detection system in a standard 180 nm CMOS process occupies 0.03 mm²of silicon area and consumes 0.63 µW of power per channel, which is the least power consumption among the currentin vivoASIC realizations.Significance.The proposed interface is the first practical approach towards realizing asynchronous BMIs while reducing the power consumption of the BMI interface and enhancing neural decoding performance compared to those of the conventional synchronous BMIs.

In vivo neural spike detection with adaptive noise estimation

The ability to reliably detect neural spikes from a relatively large population of neurons contaminated with noise is imperative for reliable decoding of recorded neural information. This article first analyzes the accuracy and feasibility of various potential spike detection techniques for in vivo realizations. Then an accurate and computationally-efficient spike detection module that can autonomously adapt to variations in recording channels' statistics is presented. The accuracy of the chosen candidate spike detection technique is evaluated using both synthetic and real neural recordings. The designed detector also offers the highest decoding performance over two animal behavioral datasets among alternative detection methods. The implementation results of the designed 128-channel spike detection module in a standard 180-nm CMOS process is among the most area and power-efficient spike detection ASICs and operates within the tissue-safe constraints for brain implants, while offering adaptive noise estimation.

Spinal Cord Injury-Induced Changes in Encoding and Decoding of Bipedal Walking by Motor Cortical Ensembles

Recent studies have shown that motor recovery following spinal cord injury (SCI) is task-specific. However, most consequential conclusions about locomotor functional recovery from SCI have been derived from quadrupedal locomotion paradigms. In this study, two monkeys were trained to perform a bipedal walking task, mimicking human walking, before and after T8 spinal cord hemisection. Importantly, there is no pharmacological therapy with nerve growth factor for monkeys after SCI; thus, in this study, the changes that occurred in the brain were spontaneous. The impairment of locomotion on the ipsilateral side was more severe than that on the contralateral side. We used information theory to analyze single-cell activity from the left primary motor cortex (M1), and results show that neuronal populations in the unilateral primary motor cortex gradually conveyed more information about the bilateral hindlimb muscle activities during the training of bipedal walking after SCI. We further demonstrated that, after SCI, progressively expanded information from the neuronal population reconstructed more accurate control of muscle activity. These results suggest that, after SCI, the unilateral primary motor cortex could gradually regain control of bilateral coordination and motor recovery and in turn enhance the performance of brain–machine interfaces.

Temporally precise population coding of dynamic sounds by auditory cortex

Fluctuations in the amplitude envelope of complex sounds provide critical cues for hearing, particularly for speech and animal vocalizations. Responses to amplitude modulation (AM) in the ascending auditory pathway have chiefly been described for single neurons. How neural populations might collectively encode and represent information about AM remains poorly characterized, even in primary auditory cortex (A1). We modeled population responses to AM based on data recorded from A1 neurons in awake squirrel monkeys and evaluated how accurately single trial responses to modulation frequencies from 4 to 512 Hz could be decoded as functions of population size, composition, and correlation structure. We found that a population-based decoding model that simulated convergent, equally weighted inputs was highly accurate and remarkably robust to the inclusion of neurons that were individually poor decoders. By contrast, average rate codes based on convergence performed poorly; effective decoding using average rates was only possible when the responses of individual neurons were segregated, as in classical population decoding models using labeled lines. The relative effectiveness of dynamic rate coding in auditory cortex was explained by shared modulation phase preferences among cortical neurons, despite heterogeneity in rate-based modulation frequency tuning. Our results indicate significant population-based synchrony in primary auditory cortex and suggest that robust population coding of the sound envelope information present in animal vocalizations and speech can be reliably achieved even with indiscriminate pooling of cortical responses. These findings highlight the importance of firing rate dynamics in population-based sensory coding.NEW & NOTEWORTHY Fundamental questions remain about population coding in primary auditory cortex (A1). In particular, issues of spike timing in models of neural populations have been largely ignored. We find that spike-timing in response to sound envelope fluctuations is highly similar across neuron populations in A1. This property of shared envelope phase preference allows for a simple population model involving unweighted convergence of neuronal responses to classify amplitude modulation frequencies with high accuracy.

Single-neuronal predictions of others' beliefs in humans

Human social behaviour crucially depends on our ability to reason about others. This capacity for theory of mind has a vital role in social cognition because it enables us not only to form a detailed understanding of the hidden thoughts and beliefs of other individuals but also to understand that they may differ from our own^1-3. Although a number of areas in the human brain have been linked to social reasoning^4,5 and its disruption across a variety of psychosocial disorders^6-8, the basic cellular mechanisms that underlie human theory of mind remain undefined. Here, using recordings from single cells in the human dorsomedial prefrontal cortex, we identify neurons that reliably encode information about others' beliefs across richly varying scenarios and that distinguish self- from other-belief-related representations. By further following their encoding dynamics, we show how these cells represent the contents of the others' beliefs and accurately predict whether they are true or false. We also show how they track inferred beliefs from another's specific perspective and how their activities relate to behavioural performance. Together, these findings reveal a detailed cellular process in the human dorsomedial prefrontal cortex for representing another's beliefs and identify candidate neurons that could support theory of mind.

The Neurophysiological Representation of Imagined Somatosensory Percepts in Human Cortex

Intracortical microstimulation (ICMS) in human primary somatosensory cortex (S1) has been used to successfully evoke naturalistic sensations. However, the neurophysiological mechanisms underlying the evoked sensations remain unknown. To understand how specific stimulation parameters elicit certain sensations we must first understand the representation of those sensations in the brain. In this study we record from intracortical microelectrode arrays implanted in S1, premotor cortex, and posterior parietal cortex of a male human participant performing a somatosensory imagery task. The sensations imagined were those previously elicited by ICMS of S1, in the same array of the same participant. In both spike and local field potential recordings, features of the neural signal can be used to classify different imagined sensations. These features are shown to be stable over time. The sensorimotor cortices only encode the imagined sensation during the imagery task, while posterior parietal cortex encodes the sensations starting with cue presentation. These findings demonstrate that different aspects of the sensory experience can be individually decoded from intracortically recorded human neural signals across the cortical sensory network. Activity underlying these unique sensory representations may inform the stimulation parameters for precisely eliciting specific sensations via ICMS in future work.

SIGNIFICANCE STATEMENT Electrical stimulation of human cortex is increasingly more common for providing feedback in neural devices. Understanding the relationship between naturally evoked and artificially evoked neurophysiology for the same sensations will be important in advancing such devices. Here, we investigate the neural activity in human primary somatosensory, premotor, and parietal cortices during somatosensory imagery. The sensations imagined were those previously elicited during intracortical microstimulation (ICMS) of the same somatosensory electrode array. We elucidate the neural features during somatosensory imagery that significantly encode different aspects of individual sensations and demonstrate feature stability over almost a year. The correspondence between neurophysiology elicited with or without stimulation for the same sensations will inform methods to deliver more precise feedback through stimulation in the future.

Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus

OBJECTIVE

To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the 'hand knob' area of precentral gyrus, a step toward the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak.

APPROACH

Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode's binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the 'Brain-to-Speech' pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes' onset times.

MAIN RESULTS

A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while an RNN classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio.

SIGNIFICANCE

The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.

Power-saving design opportunities for wireless intracortical brain-computer interfaces

The efficacy of wireless intracortical brain-computer interfaces (iBCIs) is limited in part by the number of recording channels, which is constrained by the power budget of the implantable system. Designing wireless iBCIs that provide the high-quality recordings of today's wired neural interfaces may lead to inadvertent over-design at the expense of power consumption and scalability. Here, we report analyses of neural signals collected from experimental iBCI measurements in rhesus macaques and from a clinical-trial participant with implanted 96-channel Utah multielectrode arrays to understand the trade-offs between signal quality and decoder performance. Moreover, we propose an efficient hardware design for clinically viable iBCIs, and suggest that the circuit design parameters of current recording iBCIs can be relaxed considerably without loss of performance. The proposed design may allow for an order-of-magnitude power savings and lead to clinically viable iBCIs with a higher channel count.

A neural network for online spike classification that improves decoding accuracy

Separating neural signals from noise can improve brain-computer interface performance and stability. However, most algorithms for separating neural action potentials from noise are not suitable for use in real time and have shown mixed effects on decoding performance. With the goal of removing noise that impedes online decoding, we sought to automate the intuition of human spike-sorters to operate in real time with an easily tunable parameter governing the stringency with which spike waveforms are classified. We trained an artificial neural network with one hidden layer on neural waveforms that were hand-labeled as either spikes or noise. The network output was a likelihood metric for each waveform it classified, and we tuned the network's stringency by varying the minimum likelihood value for a waveform to be considered a spike. Using the network's labels to exclude noise waveforms, we decoded remembered target location during a memory-guided saccade task from electrode arrays implanted in prefrontal cortex of rhesus macaque monkeys. The network classified waveforms in real time, and its classifications were qualitatively similar to those of a human spike-sorter. Compared with decoding with threshold crossings, in most sessions we improved decoding performance by removing waveforms with low spike likelihood values. Furthermore, decoding with our network's classifications became more beneficial as time since array implantation increased. Our classifier serves as a feasible preprocessing step, with little risk of harm, that could be applied to both off-line neural data analyses and online decoding.NEW & NOTEWORTHY Although there are many spike-sorting methods that isolate well-defined single units, these methods typically involve human intervention and have inconsistent effects on decoding. We used human classified neural waveforms as training data to create an artificial neural network that could be tuned to separate spikes from noise that impaired decoding. We found that this network operated in real time and was suitable for both off-line data processing and online decoding.

Extracellular voltage thresholds for maximizing information extraction in primate auditory cortex: implications for a brain computer interface

OBJECTIVE

Research by Oby et al (2016) demonstrated that the optimal threshold for extracting information from visual and motor cortices may differ from the optimal threshold for identifying single neurons via spike sorting methods. The optimal threshold for extracting information from auditory cortex has yet to be identified, nor has the optimal temporal scale for representing auditory cortical activity. Here, we describe a procedure to jointly optimize the extracellular threshold and bin size with respect to the decoding accuracy achieved by a linear classifier for a diverse set of auditory stimuli.

APPROACH

We used linear multichannel arrays to record extracellular neural activity from the auditory cortex of awake squirrel monkeys passively listening to both simple and complex sounds. We executed a grid search of the coordinate space defined by the voltage threshold (in units of standard deviation) and the bin size (in units of milliseconds), and computed decoding accuracy at each point.

MAIN RESULTS

The optimal threshold for information extraction was consistently near two standard deviations below the voltage trace mean, which falls significantly below the range of three to five standard deviations typically used as inputs to spike sorting algorithms in basic research and in brain-computer interface (BCI) applications. The optimal binwidth was minimized at the optimal voltage threshold, particularly for acoustic stimuli dominated by temporally dynamic features, indicating that permissive thresholding permits readout of cortical responses with temporal precision on the order of a few milliseconds.

SIGNIFICANCE

The improvements in decoding accuracy we observed for optimal readout parameters suggest that standard thresholding methods substantially underestimate the information present in auditory cortical spiking patterns. The fact that optimal thresholds were relatively low indicates that local populations of cortical neurons exhibit high temporal coherence that could be leveraged in service of future auditory BCI applications.

Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis

Speaking is a sensorimotor behavior whose neural basis is difficult to study with single neuron resolution due to the scarcity of human intracortical measurements. We used electrode arrays to record from the motor cortex 'hand knob' in two people with tetraplegia, an area not previously implicated in speech. Neurons modulated during speaking and during non-speaking movements of the tongue, lips, and jaw. This challenges whether the conventional model of a 'motor homunculus' division by major body regions extends to the single-neuron scale. Spoken words and syllables could be decoded from single trials, demonstrating the potential of intracortical recordings for brain-computer interfaces to restore speech. Two neural population dynamics features previously reported for arm movements were also present during speaking: a component that was mostly invariant across initiating different words, followed by rotatory dynamics during speaking. This suggests that common neural dynamical motifs may underlie movement of arm and speech articulators.

Watch, Imagine, Attempt: Motor Cortex Single-Unit Activity Reveals Context-Dependent Movement Encoding in Humans With Tetraplegia

Planning and performing volitional movement engages widespread networks in the human brain, with motor cortex considered critical to the performance of skilled limb actions. Motor cortex is also engaged when actions are observed or imagined, but the manner in which ensembles of neurons represent these volitional states (VoSs) is unknown. Here we provide direct demonstration that observing, imagining or attempting action activates shared neural ensembles in human motor cortex. Two individuals with tetraplegia (due to brainstem stroke or amyotrophic lateral sclerosis, ALS) were verbally instructed to watch, imagine, or attempt reaching actions displayed on a computer screen. Neural activity in the precentral gyrus incorporated information about both cognitive state and movement kinematics; the three conditions presented overlapping but unique, statistically distinct activity patterns. These findings demonstrate that individual neurons in human motor cortex reflect information related to sensory inputs and VoS in addition to movement features, and are a key part of a broader network linking perception and cognition to action.

Neural Population Dynamics Underlying Motor Learning Transfer

Covert motor learning can sometimes transfer to overt behavior. We investigated the neural mechanism underlying transfer by constructing a two-context paradigm. Subjects performed cursor movements either overtly using arm movements, or covertly via a brain-machine interface that moves the cursor based on motor cortical activity (in lieu of arm movement). These tasks helped evaluate whether and how cortical changes resulting from "covert rehearsal" affect overt performance. We found that covert learning indeed transfers to overt performance and is accompanied by systematic population-level changes in motor preparatory activity. Current models of motor cortical function ascribe motor preparation to achieving initial conditions favorable for subsequent movement-period neural dynamics. We found that covert and overt contexts share these initial conditions, and covert rehearsal manipulates them in a manner that persists across context changes, thus facilitating overt motor learning. This transfer learning mechanism might provide new insights into other covert processes like mental rehearsal.

Cluster-based analysis improves predictive validity of spike-triggered receptive field estimates

Spectrotemporal receptive field (STRF) characterization is a central goal of auditory physiology. STRFs are often approximated by the spike-triggered average (STA), which reflects the average stimulus preceding a spike. In many cases, the raw STA is subjected to a threshold defined by gain values expected by chance. However, such correction methods have not been universally adopted, and the consequences of specific gain-thresholding approaches have not been investigated systematically. Here, we evaluate two classes of statistical correction techniques, using the resulting STRF estimates to predict responses to a novel validation stimulus. The first, more traditional technique eliminated STRF pixels (time-frequency bins) with gain values expected by chance. This correction method yielded significant increases in prediction accuracy, including when the threshold setting was optimized for each unit. The second technique was a two-step thresholding procedure wherein clusters of contiguous pixels surviving an initial gain threshold were then subjected to a cluster mass threshold based on summed pixel values. This approach significantly improved upon even the best gain-thresholding techniques. Additional analyses suggested that allowing threshold settings to vary independently for excitatory and inhibitory subfields of the STRF resulted in only marginal additional gains, at best. In summary, augmenting reverse correlation techniques with principled statistical correction choices increased prediction accuracy by over 80% for multi-unit STRFs and by over 40% for single-unit STRFs, furthering the interpretational relevance of the recovered spectrotemporal filters for auditory systems analysis.

Sums of Spike Waveform Features for Motor Decoding

Traditionally, the key step before decoding motor intentions from cortical recordings is spike sorting, the process of identifying which neuron was responsible for an action potential. Recently, researchers have started investigating approaches to decoding which omit the spike sorting step, by directly using information about action potentials' waveform shapes in the decoder, though this approach is not yet widespread. Particularly, one recent approach involves computing the moments of waveform features and using these moment values as inputs to decoders. This computationally inexpensive approach was shown to be comparable in accuracy to traditional spike sorting. In this study, we use offline data recorded from two Rhesus monkeys to further validate this approach. We also modify this approach by using sums of exponentiated features of spikes, rather than moments. Our results show that using waveform feature sums facilitates significantly higher hand movement reconstruction accuracy than using waveform feature moments, though the magnitudes of differences are small. We find that using the sums of one simple feature, the spike amplitude, allows better offline decoding accuracy than traditional spike sorting by expert (correlation of 0.767, 0.785 vs. 0.744, 0.738, respectively, for two monkeys, average 16% reduction in mean-squared-error), as well as unsorted threshold crossings (0.746, 0.776; average 9% reduction in mean-squared-error). Our results suggest that the sums-of-features framework has potential as an alternative to both spike sorting and using unsorted threshold crossings, if developed further. Also, we present data comparing sorted vs. unsorted spike counts in terms of offline decoding accuracy. Traditional sorted spike counts do not include waveforms that do not match any template (“hash”), but threshold crossing counts do include this hash. On our data and in previous work, hash contributes to decoding accuracy. Thus, using the comparison between sorted spike counts and threshold crossing counts to evaluate the benefit of sorting is confounded by the presence of hash. We find that when the comparison is controlled for hash, performing sorting is better than not. These results offer a new perspective on the question of to sort or not to sort.

Motor Cortical Visuomotor Feedback Activity Is Initially Isolated from Downstream Targets in Output-Null Neural State Space Dimensions

Neural circuits must transform new inputs into outputs without prematurely affecting downstream circuits while still maintaining other ongoing communication with these targets. We investigated how this isolation is achieved in the motor cortex when macaques received visual feedback signaling a movement perturbation. To overcome limitations in estimating the mapping from cortex to arm movements, we also conducted brain-machine interface (BMI) experiments where we could definitively identify neural firing patterns as output-null or output-potent. This revealed that perturbation-evoked responses were initially restricted to output-null patterns that cancelled out at the neural population code readout and only later entered output-potent neural dimensions. This mechanism was facilitated by the circuit's large null space and its ability to strongly modulate output-potent dimensions when generating corrective movements. These results show that the nervous system can temporarily isolate portions of a circuit's activity from its downstream targets by restricting this activity to the circuit's output-null neural dimensions.