Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm

Tutorial: using NEURON for neuromechanical simulations

The dynamical properties of the brain and the dynamics of the body strongly influence one another. Their interaction generates complex adaptive behavior. While a wide variety of simulation tools exist for neural dynamics or biomechanics separately, there are few options for integrated brain-body modeling. Here, we provide a tutorial to demonstrate how the widely-used NEURON simulation platform can support integrated neuromechanical modeling. As a first step toward incorporating biomechanics into a NEURON simulation, we provide a framework for integrating inputs from a "periphery" and outputs to that periphery. In other words, "body" dynamics are driven in part by "brain" variables, such as voltages or firing rates, and "brain" dynamics are influenced by "body" variables through sensory feedback. To couple the "brain" and "body" components, we use NEURON's pointer construct to share information between "brain" and "body" modules. This approach allows separate specification of brain and body dynamics and code reuse. Though simple in concept, the use of pointers can be challenging due to a complicated syntax and several different programming options. In this paper, we present five different computational models, with increasing levels of complexity, to demonstrate the concepts of code modularity using pointers and the integration of neural and biomechanical modeling within NEURON. The models include: (1) a neuromuscular model of calcium dynamics and muscle force, (2) a neuromechanical, closed-loop model of a half-center oscillator coupled to a rudimentary motor system, (3) a closed-loop model of neural control for respiration, (4) a pedagogical model of a non-smooth "brain/body" system, and (5) a closed-loop model of feeding behavior in the sea hare Aplysia californica that incorporates biologically-motivated non-smooth dynamics. This tutorial illustrates how NEURON can be integrated with a broad range of neuromechanical models.

Code available at

https://github.com/fietkiewicz/PointerBuilder.

Training a spiking neuronal network model of visual-motor cortex to play a virtual racket-ball game using reinforcement learning

Recent models of spiking neuronal networks have been trained to perform behaviors in static environments using a variety of learning rules, with varying degrees of biological realism. Most of these models have not been tested in dynamic visual environments where models must make predictions on future states and adjust their behavior accordingly. The models using these learning rules are often treated as black boxes, with little analysis on circuit architectures and learning mechanisms supporting optimal performance. Here we developed visual/motor spiking neuronal network models and trained them to play a virtual racket-ball game using several reinforcement learning algorithms inspired by the dopaminergic reward system. We systematically investigated how different architectures and circuit-motifs (feed-forward, recurrent, feedback) contributed to learning and performance. We also developed a new biologically-inspired learning rule that significantly enhanced performance, while reducing training time. Our models included visual areas encoding game inputs and relaying the information to motor areas, which used this information to learn to move the racket to hit the ball. Neurons in the early visual area relayed information encoding object location and motion direction across the network. Neuronal association areas encoded spatial relationships between objects in the visual scene. Motor populations received inputs from visual and association areas representing the dorsal pathway. Two populations of motor neurons generated commands to move the racket up or down. Model-generated actions updated the environment and triggered reward or punishment signals that adjusted synaptic weights so that the models could learn which actions led to reward. Here we demonstrate that our biologically-plausible learning rules were effective in training spiking neuronal network models to solve problems in dynamic environments. We used our models to dissect the circuit architectures and learning rules most effective for learning. Our model shows that learning mechanisms involving different neural circuits produce similar performance in sensory-motor tasks. In biological networks, all learning mechanisms may complement one another, accelerating the learning capabilities of animals. Furthermore, this also highlights the resilience and redundancy in biological systems.

Real-time recognition of different imagined actions on the same side of a single limb based on the fNIRS correlation coefficient

Functional near-infrared spectroscopy (fNIRS) is a type of functional brain imaging. Brain-computer interfaces (BCIs) based on fNIRS have recently been implemented. Most existing fNIRS-BCI studies have involved off-line analyses, but few studies used online performance testing. Furthermore, existing online fNIRS-BCI experimental paradigms have not yet carried out studies using different imagined movements of the same side of a single limb. In the present study, a real-time fNIRS-BCI system was constructed to identify two imagined movements of the same side of a single limb (right forearm and right hand). Ten healthy subjects were recruited and fNIRS signal was collected and real-time analyzed with two imagined movements (leftward movement involving the right forearm and right-hand clenching). In addition to the mean and slope features of fNIRS signals, the correlation coefficient between fNIRS signals induced by different imagined actions was extracted. A support vector machine (SVM) was used to classify the imagined actions. The average accuracy of real-time classification of the two imagined movements was 72.25 ± 0.004%. The findings suggest that different imagined movements on the same side of a single limb can be recognized real-time based on fNIRS, which may help to further guide the practical application of online fNIRS-BCIs.

Multiscale Computer Model of the Spinal Dorsal Horn Reveals Changes in Network Processing Associated with Chronic Pain

Pain-related sensory input is processed in the spinal dorsal horn (SDH) before being relayed to the brain. That processing profoundly influences whether stimuli are correctly or incorrectly perceived as painful. Significant advances have been made in identifying the types of excitatory and inhibitory neurons that comprise the SDH, and there is some information about how neuron types are connected, but it remains unclear how the overall circuit processes sensory input or how that processing is disrupted under chronic pain conditions. To explore SDH function, we developed a computational model of the circuit that is tightly constrained by experimental data. Our model comprises conductance-based neuron models that reproduce the characteristic firing patterns of spinal neurons. Excitatory and inhibitory neuron populations, defined by their expression of genetic markers, spiking pattern, or morphology, were synaptically connected according to available qualitative data. Using a genetic algorithm, synaptic weights were tuned to reproduce projection neuron firing rates (model output) based on primary afferent firing rates (model input) across a range of mechanical stimulus intensities. Disparate synaptic weight combinations could produce equivalent circuit function, revealing degeneracy that may underlie heterogeneous responses of different circuits to perturbations or pathologic insults. To validate our model, we verified that it responded to the reduction of inhibition (i.e., disinhibition) and ablation of specific neuron types in a manner consistent with experiments. Thus validated, our model offers a valuable resource for interpreting experimental results and testing hypotheses in silico to plan experiments for examining normal and pathologic SDH circuit function.SIGNIFICANCE STATEMENT We developed a multiscale computer model of the posterior part of spinal cord gray matter (spinal dorsal horn), which is involved in perceiving touch and pain. The model reproduces several experimental observations and makes predictions about how specific types of spinal neurons and synapses influence projection neurons that send information to the brain. Misfiring of these projection neurons can produce anomalous sensations associated with chronic pain. Our computer model will not only assist in planning future experiments, but will also be useful for developing new pharmacotherapy for chronic pain disorders, connecting the effect of drugs acting at the molecular scale with emergent properties of neurons and circuits that shape the pain experience.

Noise-Correlation Is Modulated by Reward Expectation in the Primary Motor Cortex Bilaterally During Manual and Observational Tasks in Primates

Reward modulation is represented in the motor cortex (M1) and could be used to implement more accurate decoding models to improve brain-computer interfaces (BCIs; Zhao et al., 2018). Analyzing trial-to-trial noise-correlations between neural units in the presence of rewarding (R) and non-rewarding (NR) stimuli adds to our understanding of cortical network dynamics. We utilized Pearson's correlation coefficient to measure shared variability between simultaneously recorded units (32-112) and found significantly higher noise-correlation and positive correlation between the populations' signal- and noise-correlation during NR trials as compared to R trials. This pattern is evident in data from two non-human primates (NHPs) during single-target center out reaching tasks, both manual and action observation versions. We conducted a mean matched noise-correlation analysis to decouple known interactions between event-triggered firing rate changes and neural correlations. Isolated reward discriminatory units demonstrated stronger correlational changes than units unresponsive to reward firing rate modulation, however, the qualitative response was similar, indicating correlational changes within the network as a whole can serve as another information channel to be exploited by BCIs that track the underlying cortical state, such as reward expectation, or attentional modulation. Reward expectation and attention in return can be utilized with reinforcement learning (RL) towards autonomous BCI updating.

Adaptive filtering of physiological noises in fNIRS data

The study presents a recursive least-squares estimation method with an exponential forgetting factor for noise removal in functional near-infrared spectroscopy data and extraction of hemodynamic responses (HRs) from the measured data. The HR is modeled as a linear regression form in which the expected HR, the first and second derivatives of the expected HR, a short-separation measurement data, three physiological noises, and the baseline drift are included as components in the regression vector. The proposed method is applied to left-motor-cortex experiments on the right thumb and little finger movements in five healthy male participants. The algorithm is evaluated with respect to its performance improvement in terms of contrast-to-noise ratio in comparison with Kalman filter, low-pass filtering, and independent component method. The experimental results show that the proposed model achieves reductions of 77% and 99% in terms of the number of channels exhibiting higher contrast-to-noise ratios in oxy-hemoglobin and deoxy-hemoglobin, respectively. The approach is robust in obtaining consistent HR data. The proposed method is applied for both offline and online noise removal.

Geppetto: a reusable modular open platform for exploring neuroscience data and models

Geppetto is an open-source platform that provides generic middleware infrastructure for building both online and desktop tools for visualizing neuroscience models and data and managing simulations. Geppetto underpins a number of neuroscience applications, including Open Source Brain (OSB), Virtual Fly Brain (VFB), NEURON-UI and NetPyNE-UI. OSB is used by researchers to create and visualize computational neuroscience models described in NeuroML and simulate them through the browser. VFB is the reference hub for Drosophila melanogaster neural anatomy and imaging data including neuropil, segmented neurons, microscopy stacks and gene expression pattern data. Geppetto is also being used to build a new user interface for NEURON, a widely used neuronal simulation environment, and for NetPyNE, a Python package for network modelling using NEURON. Geppetto defines domain agnostic abstractions used by all these applications to represent their models and data and offers a set of modules and components to integrate, visualize and control simulations in a highly accessible way. The platform comprises a backend which can connect to external data sources, model repositories and simulators together with a highly customizable frontend.

This article is part of a discussion meeting issue ‘Connectome to behaviour: modelling C. elegans at cellular resolution’.

A Survey of Robotics Control Based on Learning-Inspired Spiking Neural Networks

Biological intelligence processes information using impulses or spikes, which makes those living creatures able to perceive and act in the real world exceptionally well and outperform state-of-the-art robots in almost every aspect of life. To make up the deficit, emerging hardware technologies and software knowledge in the fields of neuroscience, electronics, and computer science have made it possible to design biologically realistic robots controlled by spiking neural networks (SNNs), inspired by the mechanism of brains. However, a comprehensive review on controlling robots based on SNNs is still missing. In this paper, we survey the developments of the past decade in the field of spiking neural networks for control tasks, with particular focus on the fast emerging robotics-related applications. We first highlight the primary impetuses of SNN-based robotics tasks in terms of speed, energy efficiency, and computation capabilities. We then classify those SNN-based robotic applications according to different learning rules and explicate those learning rules with their corresponding robotic applications. We also briefly present some existing platforms that offer an interaction between SNNs and robotics simulations for exploration and exploitation. Finally, we conclude our survey with a forecast of future challenges and some associated potential research topics in terms of controlling robots based on SNNs.

Evolutionary algorithm optimization of biological learning parameters in a biomimetic neuroprosthesis

Biomimetic simulation permits neuroscientists to better understand the complex neuronal dynamics of the brain. Embedding a biomimetic simulation in a closed-loop neuroprosthesis, which can read and write signals from the brain, will permit applications for amelioration of motor, psychiatric, and memory-related brain disorders. Biomimetic neuroprostheses require real-time adaptation to changes in the external environment, thus constituting an example of a dynamic data-driven application system. As model fidelity increases, so does the number of parameters and the complexity of finding appropriate parameter configurations. Instead of adapting synaptic weights via machine learning, we employed major biological learning methods: spike-timing dependent plasticity and reinforcement learning. We optimized the learning metaparameters using evolutionary algorithms, which were implemented in parallel and which used an island model approach to obtain sufficient speed. We employed these methods to train a cortical spiking model to utilize macaque brain activity, indicating a selected target, to drive a virtual musculoskeletal arm with realistic anatomical and biomechanical properties to reach to that target. The optimized system was able to reproduce macaque data from a comparable experimental motor task. These techniques can be used to efficiently tune the parameters of multiscale systems, linking realistic neuronal dynamics to behavior, and thus providing a useful tool for neuroscience and neuroprosthetics.

Bundled-Optode Method in Functional Near-Infrared Spectroscopy

In this paper, a theory for detection of the absolute concentrations of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) from hemodynamic responses using a bundled-optode configuration in functional near-infrared spectroscopy (fNIRS) is proposed. The proposed method is then applied to the identification of two fingers (i.e., little and thumb) during their flexion and extension. This experiment involves a continuous-wave-type dual-wavelength (760 and 830 nm) fNIRS and five healthy male subjects. The active brain locations of two finger movements are identified based on the analysis of the t- and p-values of the averaged HbOs, which are quite distinctive. Our experimental results, furthermore, revealed that the hemodynamic responses of two-finger movements are different: The mean, peak, and time-to-peak of little finger movements are higher than those of thumb movements. It is noteworthy that the developed method can be extended to 3-dimensional fNIRS imaging.

Restoring Behavior via Inverse Neurocontroller in a Lesioned Cortical Spiking Model Driving a Virtual Arm

Neural stimulation can be used as a tool to elicit natural sensations or behaviors by modulating neural activity. This can be potentially used to mitigate the damage of brain lesions or neural disorders. However, in order to obtain the optimal stimulation sequences, it is necessary to develop neural control methods, for example by constructing an inverse model of the target system. For real brains, this can be very challenging, and often unfeasible, as it requires repeatedly stimulating the neural system to obtain enough probing data, and depends on an unwarranted assumption of stationarity. By contrast, detailed brain simulations may provide an alternative testbed for understanding the interactions between ongoing neural activity and external stimulation. Unlike real brains, the artificial system can be probed extensively and precisely, and detailed output information is readily available. Here we employed a spiking network model of sensorimotor cortex trained to drive a realistic virtual musculoskeletal arm to reach a target. The network was then perturbed, in order to simulate a lesion, by either silencing neurons or removing synaptic connections. All lesions led to significant behvaioral impairments during the reaching task. The remaining cells were then systematically probed with a set of single and multiple-cell stimulations, and results were used to build an inverse model of the neural system. The inverse model was constructed using a kernel adaptive filtering method, and was used to predict the neural stimulation pattern required to recover the pre-lesion neural activity. Applying the derived neurostimulation to the lesioned network improved the reaching behavior performance. This work proposes a novel neurocontrol method, and provides theoretical groundwork on the use biomimetic brain models to develop and evaluate neurocontrollers that restore the function of damaged brain regions and the corresponding motor behaviors.