Embedding a Panoramic Representation of Infrared Light in the Adult Rat Somatosensory Cortex through a Sensory Neuroprosthesis

A Neurobiological-Behavioral Approach to Predicting and Influencing Private Events

The primary goals of behavior analysis are the prediction and influence of behavior. These goals are largely achieved through the identification of functional relations between behaviors and the stimulating environment. Behavior-behavior relations are insufficient to meet these goals. Although this environment-behavior approach has been highly successful when applied to public behaviors, extensions to private events have been limited. This article discusses technical and conceptual challenges to the study of private events. We introduce a neurobiological-behavioral approach which seeks to understand private behavior as environmentally controlled in part by private neurobiological stimuli. These stimuli may enter into functional relations with both public and private behaviors. The analysis builds upon several current approaches to private events, delineates private behaviors and private stimulation, and emphasizes the reciprocal interaction between the two. By doing so, this approach can improve treatment and assessment of behavior and advance understanding of concepts such as motivating operations. We then describe the array of stimulus functions that neurobiological stimuli may acquire, including eliciting, discriminative, motivating, reinforcing, and punishing effects, and describe how the overall approach expands the concept of contextual influence. Finally, we describe how advances in behavioral neuroscience that enable the measurement and analysis of private behaviors and stimuli are allowing these once private events to affect the public world. Applications in the area of human-computer interfaces are discussed.

Brain-machine interface learning is facilitated by specific patterning of distributed cortical feedback

Neuroprosthetics offer great hope for motor-impaired patients. One obstacle is that fine motor control requires near-instantaneous, rich somatosensory feedback. Such distributed feedback may be recreated in a brain-machine interface using distributed artificial stimulation across the cortical surface. Here, we hypothesized that neuronal stimulation must be contiguous in its spatiotemporal dynamics to be efficiently integrated by sensorimotor circuits. Using a closed-loop brain-machine interface, we trained head-fixed mice to control a virtual cursor by modulating the activity of motor cortex neurons. We provided artificial feedback in real time with distributed optogenetic stimulation patterns in the primary somatosensory cortex. Mice developed a specific motor strategy and succeeded to learn the task only when the optogenetic feedback pattern was spatially and temporally contiguous while it moved across the topography of the somatosensory cortex. These results reveal spatiotemporal properties of the sensorimotor cortical integration that set constraints on the design of neuroprosthetics.

SENECA: building a fully digital neuromorphic processor, design trade-offs and challenges

Neuromorphic processors aim to emulate the biological principles of the brain to achieve high efficiency with low power consumption. However, the lack of flexibility in most neuromorphic architecture designs results in significant performance loss and inefficient memory usage when mapping various neural network algorithms. This paper proposes SENECA, a digital neuromorphic architecture that balances the trade-offs between flexibility and efficiency using a hierarchical-controlling system. A SENECA core contains two controllers, a flexible controller (RISC-V) and an optimized controller (Loop Buffer). This flexible computational pipeline allows for deploying efficient mapping for various neural networks, on-device learning, and pre-post processing algorithms. The hierarchical-controlling system introduced in SENECA makes it one of the most efficient neuromorphic processors, along with a higher level of programmability. This paper discusses the trade-offs in digital neuromorphic processor design, explains the SENECA architecture, and provides detailed experimental results when deploying various algorithms on the SENECA platform. The experimental results show that the proposed architecture improves energy and area efficiency and illustrates the effect of various trade-offs in algorithm design. A SENECA core consumes 0.47 mm² when synthesized in the GF-22 nm technology node and consumes around 2.8 pJ per synaptic operation. SENECA architecture scales up by connecting many cores with a network-on-chip. The SENECA platform and the tools used in this project are freely available for academic research upon request.

Responses of Cortical Neurons to Intracortical Microstimulation in Awake Primates

Intracortical microstimulation (ICMS) is commonly used in many experimental and clinical paradigms; however, its effects on the activation of neurons are still not completely understood. To document the responses of cortical neurons in awake nonhuman primates to stimulation, we recorded single-unit activity while delivering single-pulse stimulation via Utah arrays implanted in primary motor cortex (M1) of three macaque monkeys. Stimuli between 5 and 50 μA delivered to single channels reliably evoked spikes in neurons recorded throughout the array with delays of up to 12 ms. ICMS pulses also induced a period of inhibition lasting up to 150 ms that typically followed the initial excitatory response. Higher current amplitudes led to a greater probability of evoking a spike and extended the duration of inhibition. The likelihood of evoking a spike in a neuron was dependent on the spontaneous firing rate as well as the delay between its most recent spike time and stimulus onset. Tonic repetitive stimulation between 2 and 20 Hz often modulated both the probability of evoking spikes and the duration of inhibition; high-frequency stimulation was more likely to change both responses. On a trial-by-trial basis, whether a stimulus evoked a spike did not affect the subsequent inhibitory response; however, their changes over time were often positively or negatively correlated. Our results document the complex dynamics of cortical neural responses to electrical stimulation that need to be considered when using ICMS for scientific and clinical applications.

Learning in a closed-loop brain-machine interface with distributed optogenetic cortical feedback

Objective.Distributed microstimulations at the cortical surface can efficiently deliver feedback to a subject during the manipulation of a prosthesis through a brain-machine interface (BMI). Such feedback can convey vast amounts of information to the prosthesis user and may be key to obtain an accurate control and embodiment of the prosthesis. However, so far little is known of the physiological constraints on the decoding of such patterns. Here, we aimed to test a rotary optogenetic feedback that was designed to encode efficiently the 360° movements of the robotic actuators used in prosthetics. We sought to assess its use by mice that controlled a prosthesis joint through a closed-loop BMI.Approach.We tested the ability of mice to optimize the trajectory of a virtual prosthesis joint in order to solve a rewarded reaching task. They could control the speed of the joint by modulating the activity of individual neurons in the primary motor cortex. During the task, the patterned optogenetic stimulation projected on the primary somatosensory cortex continuously delivered information to the mouse about the position of the joint.Main results.We showed that mice are able to exploit the continuous, rotating cortical feedback in the active behaving context of the task. Mice achieved better control than in the absence of feedback by detecting reward opportunities more often, and also by moving the joint faster towards the reward angular zone, and by maintaining it longer in the reward zone. Mice controlling acceleration rather than speed of the joint failed to improve motor control.Significance.These findings suggest that in the context of a closed-loop BMI, distributed cortical feedback with optimized shapes and topology can be exploited to control movement. Our study has direct applications on the closed-loop control of rotary joints that are frequently encountered in robotic prostheses.

Existing function in primary visual cortex is not perturbed by new skill acquisition of a non-matched sensory task

Acquisition of new skills has the potential to disturb existing network function. To directly assess whether previously acquired cortical function is altered during learning, mice were trained in an abstract task in which selected activity patterns were rewarded using an optical brain-computer interface device coupled to primary visual cortex (V1) neurons. Excitatory neurons were longitudinally recorded using 2-photon calcium imaging. Despite significant changes in local neural activity during task performance, tuning properties and stimulus encoding assessed outside of the trained context were not perturbed. Similarly, stimulus tuning was stable in neurons that remained responsive following a different, visual discrimination training task. However, visual discrimination training increased the rate of representational drift. Our results indicate that while some forms of perceptual learning may modify the contribution of individual neurons to stimulus encoding, new skill learning is not inherently disruptive to the quality of stimulus representation in adult V1.

Continuity within the somatosensory cortical map facilitates learning

The topographic organization is a prominent feature of sensory cortices, but its functional role remains controversial. Particularly, it is not well determined how integration of activity within a cortical area depends on its topography during sensory-guided behavior. Here, we train mice expressing channelrhodopsin in excitatory neurons to track a photostimulation bar that rotated smoothly over the topographic whisker representation of the primary somatosensory cortex. Mice learn to discriminate angular positions of the light bar to obtain a reward. They fail not only when the spatiotemporal continuity of the photostimulation is disrupted in this area but also when cortical areas displaying map discontinuities, such as the trunk and legs, or areas without topographic map, such as the posterior parietal cortex, are photostimulated. In contrast, when cortical topographic continuity enables to predict future sensory activation, mice demonstrate anticipation of reward availability. These findings could be helpful for optimizing feedback while designing cortical neuroprostheses.

Topographic maps and neural tuning for sensory substitution dimensions learned in adulthood in a congenital blind subject

Topographic maps, a key principle of brain organization, emerge during development. It remains unclear, however, whether topographic maps can represent a new sensory experience learned in adulthood. MaMe, a congenitally blind individual, has been extensively trained in adulthood for perception of a 2D auditory-space (soundscape) where the y- and x-axes are represented by pitch and time, respectively. Using population receptive field mapping we found neural populations tuned topographically to pitch, not only in the auditory cortices but also in the parietal and occipito-temporal cortices. Topographic neural tuning to time was revealed in the parietal and occipito-temporal cortices. Some of these maps were found to represent both axes concurrently, enabling MaMe to represent unique locations in the soundscape space. This case study provides proof of concept for the existence of topographic maps tuned to the newly learned soundscape dimensions. These results suggest that topographic maps can be adapted or recycled in adulthood to represent novel sensory experiences.

Neural Modulation of the Primary Auditory Cortex by Intracortical Microstimulation with a Bio-Inspired Electronic System

Nowadays, the majority of the progress in the development of implantable neuroprostheses has been achieved by improving the knowledge of brain functions so as to restore sensorial impairments. Intracortical microstimulation (ICMS) is a widely used technique to investigate site-specific cortical responses to electrical stimuli. Herein, we investigated the neural modulation induced in the primary auditory cortex (A1) by an acousto-electric transduction of ultrasonic signals using a bio-inspired intracortical microstimulator. The developed electronic system emulates the transduction of ultrasound signals in the cochlea, providing bio-inspired electrical stimuli. Firstly, we identified the receptive fields in the primary auditory cortex devoted to encoding ultrasonic waves at different frequencies, mapping each area with neurophysiological patterns. Subsequently, the activity elicited by bio-inspired ICMS in the previously identified areas, bypassing the sense organ, was investigated. The observed evoked response by microstimulation resulted as highly specific to the stimuli, and the spatiotemporal dynamics of neural oscillatory activity in the alpha, beta, and gamma waves were related to the stimuli preferred by the neurons at the stimulated site. The alpha waves modulated cortical excitability only during the activation of the specific tonotopic neuronal populations, inhibiting neural responses in unrelated areas. Greater neuronal activity in the posterior area of A1 was observed in the beta band, whereas a gamma rhythm was induced in the anterior A1. The results evidence that the proposed bio-inspired acousto-electric ICMS triggers high-frequency oscillations, encoding information about the stimulation sites and involving a large-scale integration in the brain.

A Mathematical Theory of Cortex-Receptor Artificial Extension

Many physiology experiments demonstrate that an organism's cortex and receptor system can be artificially extended, giving the organism new types of perceptual capabilities. To examine artificial extension of the cortex-receptor system, I propose a computational model that allows new types of sensory pathways to be added directly to the computational model itself in an online manner. A synapse expandable artificial neuron model that can grow new synapses, forming a bridge between the novel perceptual information and the existing neural network is introduced to absorb the novel sensory pathway. The experimental results show that the computational model can effectively integrate sudden emerged sensory channels and the neural circuits in the computational model can be reused for novel modalities without influencing the original modality.

Creating a neuroprosthesis for active tactile exploration of textures

Intracortical microstimulation (ICMS) of the primary somatosensory cortex (S1) can produce percepts that mimic somatic sensation and, thus, has potential as an approach to sensorize prosthetic limbs. However, it is not known whether ICMS could recreate active texture exploration-the ability to infer information about object texture by using one's fingertips to scan a surface. Here, we show that ICMS of S1 can convey information about the spatial frequencies of invisible virtual gratings through a process of active tactile exploration. Two rhesus monkeys scanned pairs of visually identical screen objects with the fingertip of a hand avatar-controlled first via a joystick and later via a brain-machine interface-to find the object with denser virtual gratings. The gratings consisted of evenly spaced ridges that were signaled through individual ICMS pulses generated whenever the avatar's fingertip crossed a ridge. The monkeys learned to interpret these ICMS patterns, evoked by the interplay of their voluntary movements and the virtual textures of each object, to perform a sensory discrimination task. Discrimination accuracy followed Weber's law of just-noticeable differences (JND) across a range of grating densities; a finding that matches normal cutaneous sensation. Moreover, 1 monkey developed an active scanning strategy where avatar velocity was integrated with the ICMS pulses to interpret the texture information. We propose that this approach could equip upper-limb neuroprostheses with direct access to texture features acquired during active exploration of natural objects.

Learning active sensing strategies using a sensory brain-machine interface

Diverse organisms, from insects to humans, actively seek out sensory information that best informs goal-directed actions. Efficient active sensing requires congruity between sensor properties and motor strategies, as typically honed through evolution. However, it has been difficult to study whether active sensing strategies are also modified with experience. Here, we used a sensory brain-machine interface paradigm, permitting both free behavior and experimental manipulation of sensory feedback, to study learning of active sensing strategies. Rats performed a searching task in a water maze in which the only task-relevant sensory feedback was provided by intracortical microstimulation (ICMS) encoding egocentric bearing to the hidden goal location. The rats learned to use the artificial goal direction sense to find the platform with the same proficiency as natural vision. Manipulation of the acuity of the ICMS feedback revealed distinct search strategy adaptations. Using an optimization model, the different strategies were found to minimize the effort required to extract the most salient task-relevant information. The results demonstrate that animals can adjust motor strategies to match novel sensor properties for efficient goal-directed behavior.

Cortical Neuroprosthesis Merges Visible and Invisible Light Without Impairing Native Sensory Function

Adult rats equipped with a sensory prosthesis, which transduced infrared (IR) signals into electrical signals delivered to somatosensory cortex (S1), took approximately 4 d to learn a four-choice IR discrimination task. Here, we show that when such IR signals are projected to the primary visual cortex (V1), rats that are pretrained in a visual-discrimination task typically learn the same IR discrimination task on their first day of training. However, without prior training on a visual discrimination task, the learning rates for S1- and V1-implanted animals converged, suggesting there is no intrinsic difference in learning rate between the two areas. We also discovered that animals were able to integrate IR information into the ongoing visual processing stream in V1, performing a visual-IR integration task in which they had to combine IR and visual information. Furthermore, when the IR prosthesis was implanted in S1, rats showed no impairment in their ability to use their whiskers to perform a tactile discrimination task. Instead, in some rats, this ability was actually enhanced. Cumulatively, these findings suggest that cortical sensory neuroprostheses can rapidly augment the representational scope of primary sensory areas, integrating novel sources of information into ongoing processing while incurring minimal loss of native function.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.