Partially Mixed Selectivity in Human Posterior Parietal Association Cortex

Mixed selectivity: Cellular computations for complexity

The property of mixed selectivity has been discussed at a computational level and offers a strategy to maximize computational power by adding versatility to the functional role of each neuron. Here, we offer a biologically grounded implementational-level mechanistic explanation for mixed selectivity in neural circuits. We define pure, linear, and nonlinear mixed selectivity and discuss how these response properties can be obtained in simple neural circuits. Neurons that respond to multiple, statistically independent variables display mixed selectivity. If their activity can be expressed as a weighted sum, then they exhibit linear mixed selectivity; otherwise, they exhibit nonlinear mixed selectivity. Neural representations based on diverse nonlinear mixed selectivity are high dimensional; hence, they confer enormous flexibility to a simple downstream readout neural circuit. However, a simple neural circuit cannot possibly encode all possible mixtures of variables simultaneously, as this would require a combinatorially large number of mixed selectivity neurons. Gating mechanisms like oscillations and neuromodulation can solve this problem by dynamically selecting which variables are mixed and transmitted to the readout.

Mixed selectivity in monkey anterior intraparietal area during visual and motor processes

Classical studies suggest that the anterior intraparietal area (AIP) contributes to the encoding of specific information such as objects and actions of self and others, through a variety of neuronal classes, such as canonical, motor and mirror neurons. However, these studies typically focused on a single variable, leaving it unclear whether distinct sets of AIP neurons encode a single or multiple sources of information and how multimodal coding emerges. Here, we chronically recorded monkey AIP neurons in a variety of tasks and conditions classically employed in separate experiments. Most cells exhibited mixed selectivity for observed objects, executed actions, and observed actions, enhanced when this information came from the monkey's peripersonal working space. In contrast with the classical view, our findings indicate that multimodal coding emerges in AIP from partially-mixed selectivity of individual neurons for a variety of information relevant for planning actions directed to both physical objects and other subjects.

Position- and scale-invariant object-centered spatial localization in monkey frontoparietal cortex dynamically adapts to cognitive demand

Egocentric encoding is a well-known property of brain areas along the dorsal pathway. Different to previous experiments, which typically only demanded egocentric spatial processing during movement preparation, we designed a task where two male rhesus monkeys memorized an on-the-object target position and then planned a reach to this position after the object re-occurred at variable location with potentially different size. We found allocentric (in addition to egocentric) encoding in the dorsal stream reach planning areas, parietal reach region and dorsal premotor cortex, which is invariant with respect to the position, and, remarkably, also the size of the object. The dynamic adjustment from predominantly allocentric encoding during visual memory to predominantly egocentric during reach planning in the same brain areas and often the same neurons, suggests that the prevailing frame of reference is less a question of brain area or processing stream, but more of the cognitive demands.

Reward-based option competition in human dorsal stream and transition from stochastic exploration to exploitation in continuous space

Primates exploring and exploiting a continuous sensorimotor space rely on dynamic maps in the dorsal stream. Two complementary perspectives exist on how these maps encode rewards. Reinforcement learning models integrate rewards incrementally over time, efficiently resolving the exploration/exploitation dilemma. Working memory buffer models explain rapid plasticity of parietal maps but lack a plausible exploration/exploitation policy. The reinforcement learning model presented here unifies both accounts, enabling rapid, information-compressing map updates and efficient transition from exploration to exploitation. As predicted by our model, activity in human frontoparietal dorsal stream regions, but not in MT+, tracks the number of competing options, as preferred options are selectively maintained on the map, while spatiotemporally distant alternatives are compressed out. When valuable new options are uncovered, posterior β1/α oscillations desynchronize within 0.4 to 0.7 s, consistent with option encoding by competing β1-stabilized subpopulations. Together, outcomes matching locally cached reward representations rapidly update parietal maps, biasing choices toward often-sampled, rewarded options.

Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex

Objective. Enable neural control of individual prosthetic fingers for participants with upper-limb paralysis.Approach. Two tetraplegic participants were each implanted with a 96-channel array in the left posterior parietal cortex (PPC). One of the participants was additionally implanted with a 96-channel array near the hand knob of the left motor cortex (MC). Across tens of sessions, we recorded neural activity while the participants attempted to move individual fingers of the right hand. Offline, we classified attempted finger movements from neural firing rates using linear discriminant analysis with cross-validation. The participants then used the neural classifier online to control individual fingers of a brain-machine interface (BMI). Finally, we characterized the neural representational geometry during individual finger movements of both hands.Main Results. The two participants achieved 86% and 92% online accuracy during BMI control of the contralateral fingers (chance = 17%). Offline, a linear decoder achieved ten-finger decoding accuracies of 70% and 66% using respective PPC recordings and 75% using MC recordings (chance = 10%). In MC and in one PPC array, a factorized code linked corresponding finger movements of the contralateral and ipsilateral hands.Significance. This is the first study to decode both contralateral and ipsilateral finger movements from PPC. Online BMI control of contralateral fingers exceeded that of previous finger BMIs. PPC and MC signals can be used to control individual prosthetic fingers, which may contribute to a hand restoration strategy for people with tetraplegia.

Neural mechanisms underlying the hierarchical construction of perceived aesthetic value

Little is known about how the brain computes the perceived aesthetic value of complex stimuli such as visual art. Here, we used computational methods in combination with functional neuroimaging to provide evidence that the aesthetic value of a visual stimulus is computed in a hierarchical manner via a weighted integration over both low and high level stimulus features contained in early and late visual cortex, extending into parietal and lateral prefrontal cortices. Feature representations in parietal and lateral prefrontal cortex may in turn be utilized to produce an overall aesthetic value in the medial prefrontal cortex. Such brain-wide computations are not only consistent with a feature-based mechanism for value construction, but also resemble computations performed by a deep convolutional neural network. Our findings thus shed light on the existence of a general neurocomputational mechanism for rapidly and flexibly producing value judgements across an array of complex novel stimuli and situations.

New insights on single-neuron selectivity in the era of population-level approaches

In the past, neuroscience was focused on individual neurons seen as the functional units of the nervous system, but this approach fell short over time to account for new experimental evidence, especially for what concerns associative and motor cortices. For this reason and thanks to great technological advances, a part of modern research has shifted the focus from the responses of single neurons to the activity of neural ensembles, now considered the real functional units of the system. However, on a microscale, individual neurons remain the computational components of these networks, thus the study of population dynamics cannot prescind from studying also individual neurons which represent their natural substrate. In this new framework, ideas such as the capability of single cells to encode a specific stimulus (neural selectivity) may become obsolete and need to be profoundly revised. One step in this direction was made by introducing the concept of “mixed selectivity,” the capacity of single cells to integrate multiple variables in a flexible way, allowing individual neurons to participate in different networks. In this review, we outline the most important features of mixed selectivity and we also present recent works demonstrating its presence in the associative areas of the posterior parietal cortex. Finally, in discussing these findings, we present some open questions that could be addressed by future studies.

Task specificity in mouse parietal cortex

Parietal cortex is implicated in a variety of behavioral processes, but it is unknown whether and how its individual neurons participate in multiple tasks. We trained head-fixed mice to perform two visual decision tasks involving a steering wheel or a virtual T-maze and recorded from the same parietal neurons during these two tasks. Neurons that were active during the T-maze task were typically inactive during the steering-wheel task and vice versa. Recording from the same neurons in the same apparatus without task stimuli yielded the same specificity as in the task, suggesting that task specificity depends on physical context. To confirm this, we trained some mice in a third task combining the steering wheel context with the visual environment of the T-maze. This hybrid task engaged the same neurons as those engaged in the steering-wheel task. Thus, participation by neurons in mouse parietal cortex is task specific, and this specificity is determined by physical context.

Machine learning methods detect arm movement impairments in a patient with parieto-occipital lesion using only early kinematic information

Patients with lesions of the parieto-occipital cortex typically misreach visual targets that they correctly perceive (optic ataxia). Although optic ataxia was described more than 30 years ago, distinguishing this condition from physiological behavior using kinematic data is still far from being an achievement. Here, combining kinematic analysis with machine learning methods, we compared the reaching performance of a patient with bilateral occipitoparietal damage with that of 10 healthy controls. They performed visually guided reaches toward targets located at different depths and directions. Using the horizontal, sagittal, and vertical deviation of the trajectories, we extracted classification accuracy in discriminating the reaching performance of patient from that of controls. Specifically, accurate predictions of the patient's deviations were detected after the 20% of the movement execution in all the spatial positions tested. This classification based on initial trajectory decoding was possible for both directional and depth components of the movement, suggesting the possibility of applying this method to characterize pathological motor behavior in wider frameworks.

Are reaching and grasping effector-independent? Similarities and differences in reaching and grasping kinematics between the hand and foot

While reaching and grasping are highly prevalent manual actions, neuroimaging studies provide evidence that their neural representations may be shared between different body parts, i.e., effectors. If these actions are guided by effector-independent mechanisms, similar kinematics should be observed when the action is performed by the hand or by a cortically remote and less experienced effector, such as the foot. We tested this hypothesis with two characteristic components of action: the initial ballistic stage of reaching, and the preshaping of the digits during grasping based on object size. We examined if these kinematic features reflect effector-independent mechanisms by asking participants to reach toward and to grasp objects of different widths with their hand and foot. First, during both reaching and grasping, the velocity profile up to peak velocity matched between the hand and the foot, indicating a shared ballistic acceleration phase. Second, maximum grip aperture and time of maximum grip aperture of grasping increased with object size for both effectors, indicating encoding of object size during transport. Differences between the hand and foot were found in the deceleration phase and time of maximum grip aperture, likely due to biomechanical differences and the participants’ inexperience with foot actions. These findings provide evidence for effector-independent visuomotor mechanisms of reaching and grasping that generalize across body parts.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Implicit mechanisms of intention

High-level cortical regions encode motor decisions before or even absent awareness, suggesting that neural processes predetermine behavior before conscious choice. Such early neural encoding challenges popular conceptions of human agency. It also raises fundamental questions for brain-machine interfaces (BMIs) that traditionally assume that neural activity reflects the user's conscious intentions. Here, we study the timing of human posterior parietal cortex single-neuron activity recorded from implanted microelectrode arrays relative to the explicit urge to initiate movement. Participants were free to choose when to move, whether to move, and what to move, and they retrospectively reported the time they felt the urge to move. We replicate prior studies by showing that posterior parietal cortex (PPC) neural activity sharply rises hundreds of milliseconds before the reported urge. However, we find that this "preconscious" activity is part of a dynamic neural population response that initiates much earlier, when the participant first chooses to perform the task. Together with details of neural timing, our results suggest that PPC encodes an internal model of the motor planning network that transforms high-level task objectives into appropriate motor behavior. These new data challenge traditional interpretations of early neural activity and offer a more holistic perspective on the interplay between choice, behavior, and their neural underpinnings. Our results have important implications for translating BMIs into more complex real-world environments. We find that early neural dynamics are sufficient to drive BMI movements before the participant intends to initiate movement. Appropriate algorithms ensure that BMI movements align with the subject's awareness of choice.

Decoding grasp and speech signals from the cortical grasp circuit in a tetraplegic human

The cortical grasp network encodes planning and execution of grasps and processes spoken and written aspects of language. High-level cortical areas within this network are attractive implant sites for brain-machine interfaces (BMIs). While a tetraplegic patient performed grasp motor imagery and vocalized speech, neural activity was recorded from the supramarginal gyrus (SMG), ventral premotor cortex (PMv), and somatosensory cortex (S1). In SMG and PMv, five imagined grasps were well represented by firing rates of neuronal populations during visual cue presentation. During motor imagery, these grasps were significantly decodable from all brain areas. During speech production, SMG encoded both spoken grasp types and the names of five colors. Whereas PMv neurons significantly modulated their activity during grasping, SMG's neural population broadly encoded features of both motor imagery and speech. Together, these results indicate that brain signals from high-level areas of the human cortex could be used for grasping and speech BMI applications.

The neuroecology of the water-to-land transition and the evolution of the vertebrate brain

The water-to-land transition in vertebrate evolution offers an unusual opportunity to consider computational affordances of a new ecology for the brain. All sensory modalities are changed, particularly a greatly enlarged visual sensorium owing to air versus water as a medium, and expanded by mobile eyes and neck. The multiplication of limbs, as evolved to exploit aspects of life on land, is a comparable computational challenge. As the total mass of living organisms on land is a hundredfold larger than the mass underwater, computational improvements promise great rewards. In water, the midbrain tectum coordinates approach/avoid decisions, contextualized by water flow and by the animal's body state and learning. On land, the relative motions of sensory surfaces and effectors must be resolved, adding on computational architectures from the dorsal pallium, such as the parietal cortex. For the large-brained and long-living denizens of land, making the right decision when the wrong one means death may be the basis of planning, which allows animals to learn from hypothetical experience before enactment. Integration of value-weighted, memorized panoramas in basal ganglia/frontal cortex circuitry, with allocentric cognitive maps of the hippocampus and its associated cortices becomes a cognitive habit-to-plan transition as substantial as the change in ecology. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Neurons as will and representation

Memory recollections and voluntary actions are often perceived as spontaneously generated irrespective of external stimuli. Although products of our neurons, they are only rarely accessible in humans at the neuronal level. Here I review insights gleaned from unique neurosurgical opportunities to record and stimulate single-neuron activity in people who can declare their thoughts, memories and wishes. I discuss evidence that the subjective experience of human recollection and that of voluntary action arise from the activity of two internal neuronal generators, the former from medial temporal lobe reactivation and the latter from frontoparietal preactivation. I characterize properties of these generators and their interaction, enabling flexible recruitment of memory-based choices for action as well as recruitment of action-based plans for the representation of conceptual knowledge in memories. Both internal generators operate on surprisingly explicit but different neuronal codes, which appear to arise with distinct single-neuron activity, often observed before participants' reports of conscious awareness. I discuss prediction of behaviour based on these codes, and the potential for their modulation. The prospects of editing human memories and volitions by enhancement, inception or deletion of specific, selected content raise therapeutic possibilities and ethical concerns.

Exploring Cognition with Brain-Machine Interfaces

Traditional brain-machine interfaces decode cortical motor commands to control external devices. These commands are the product of higher-level cognitive processes, occurring across a network of brain areas, that integrate sensory information, plan upcoming motor actions, and monitor ongoing movements. We review cognitive signals recently discovered in the human posterior parietal cortex during neuroprosthetic clinical trials. These signals are consistent with small regions of cortex having a diverse role in cognitive aspects of movement control and body monitoring, including sensorimotor integration, planning, trajectory representation, somatosensation, action semantics, learning, and decision making. These variables are encoded within the same population of cells using structured representations that bind related sensory and motor variables, an architecture termed partially mixed selectivity. Diverse cognitive signals provide complementary information to traditional motor commands to enable more natural and intuitive control of external devices.

Parietal maps of visual signals for bodily action planning

The posterior parietal cortex (PPC) has long been understood as a high-level integrative station for computing motor commands for the body based on sensory (i.e., mostly tactile and visual) input from the outside world. In the last decade, accumulating evidence has shown that the parietal areas not only extract the pragmatic features of manipulable objects, but also subserve sensorimotor processing of others’ actions. A paradigmatic case is that of the anterior intraparietal area (AIP), which encodes the identity of observed manipulative actions that afford potential motor actions the observer could perform in response to them. On these bases, we propose an AIP manipulative action-based template of the general planning functions of the PPC and review existing evidence supporting the extension of this model to other PPC regions and to a wider set of actions: defensive and locomotor actions. In our model, a hallmark of PPC functioning is the processing of information about the physical and social world to encode potential bodily actions appropriate for the current context. We further extend the model to actions performed with man-made objects (e.g., tools) and artifacts, because they become integral parts of the subject’s body schema and motor repertoire. Finally, we conclude that existing evidence supports a generally conserved neural circuitry that transforms integrated sensory signals into the variety of bodily actions that primates are capable of preparing and performing to interact with their physical and social world.

Distributed functions of prefrontal and parietal cortices during sequential categorical decisions

Comparing sequential stimuli is crucial for guiding complex behaviors. To understand mechanisms underlying sequential decisions, we compared neuronal responses in the prefrontal cortex (PFC), the lateral intraparietal (LIP), and medial intraparietal (MIP) areas in monkeys trained to decide whether sequentially presented stimuli were from matching (M) or nonmatching (NM) categories. We found that PFC leads M/NM decisions, whereas LIP and MIP appear more involved in stimulus evaluation and motor planning, respectively. Compared to LIP, PFC showed greater nonlinear integration of currently visible and remembered stimuli, which correlated with the monkeys' M/NM decisions. Furthermore, multi-module recurrent networks trained on the same task exhibited key features of PFC and LIP encoding, including nonlinear integration in the PFC-like module, which was causally involved in the networks' decisions. Network analysis found that nonlinear units have stronger and more widespread connections with input, output, and within-area units, indicating putative circuit-level mechanisms for sequential decisions.

Intracortical Microelectrode Array Unit Yield under Chronic Conditions: A Comparative Evaluation

While microelectrode arrays (MEAs) offer the promise of elucidating functional neural circuitry and serve as the basis for a cortical neuroprosthesis, the challenge of designing and demonstrating chronically reliable technology remains. Numerous studies report “chronic” data but the actual time spans and performance measures corresponding to the experimental work vary. In this study, we reviewed the experimental durations that constitute chronic studies across a range of MEA types and animal species to gain an understanding of the widespread variability in reported study duration. For rodents, which are the most commonly used animal model in chronic studies, we examined active electrode yield (AEY) for different array types as a means to contextualize the study duration variance, as well as investigate and interpret the performance of custom devices in comparison to conventional MEAs. We observed wide-spread variance within species for the chronic implantation period and an AEY that decayed linearly in rodent models that implanted commercially-available devices. These observations provide a benchmark for comparing the performance of new technologies and highlight the need for consistency in chronic MEA studies. Additionally, to fully derive performance under chronic conditions, the duration of abiotic failure modes, biological processes induced by indwelling probes, and intended application of the device are key determinants.

Human Somatosensory Cortex Is Modulated during Motor Planning

Recent data and motor control theory argues that movement planning involves preparing the neural state of primary motor cortex (M1) for forthcoming action execution. Theories related to internal models, feedback control, and predictive coding also emphasize the importance of sensory prediction (and processing) before (and during) the movement itself, explaining why motor-related deficits can arise from damage to primary somatosensory cortex (S1). Motivated by this work, here we examined whether motor planning, in addition to changing the neural state of M1, changes the neural state of S1, preparing it for the sensory feedback that arises during action. We tested this idea in two human functional MRI studies (N = 31, 16 females) involving delayed object manipulation tasks, focusing our analysis on premovement activity patterns in M1 and S1. We found that the motor effector to be used in the upcoming action could be decoded, well before movement, from neural activity in M1 in both studies. Critically, we found that this effector information was also present, well before movement, in S1. In particular, we found that the encoding of effector information in area 3b (S1 proper) was linked to the contralateral hand, similarly to that found in M1, whereas in areas 1 and 2 this encoding was present in both the contralateral and ipsilateral hemispheres. Together, these findings suggest that motor planning not only prepares the motor system for movement but also changes the neural state of the somatosensory system, presumably allowing it to anticipate the sensory information received during movement.SIGNIFICANCE STATEMENT Whereas recent work on motor cortex has emphasized the critical role of movement planning in preparing neural activity for movement generation, it has not investigated the extent to which planning also modulates the activity in the adjacent primary somatosensory cortex. This reflects a key gap in knowledge, given that recent motor control theories emphasize the importance of sensory feedback processing in effective movement generation. Here, we find through a convergence of experiments and analyses, that the planning of object manipulation tasks, in addition to modulating the activity in the motor cortex, changes the state of neural activity in different subfields of the human S1. We suggest that this modulation prepares the S1 for the sensory information it will receive during action execution.

Motor-like neural dynamics in two parietal areas during arm reaching

The classical view on motor control makes a clear distinction between the role of motor cortex in controlling muscles and parietal cortex in processing movement plans and goals. However, the strong parieto-frontal connections argue against such clear-cut separation of function. Modern dynamical approaches revealed that population activity in motor cortex can be captured by a limited number of patterns, called neural states that are preserved across diverse motor behaviors. Whether such dynamics are also present in parietal cortex is unclear. Here, we studied neural dynamics in the primate parietal cortex during arm movements and found three main states temporally coupled to the planning, execution and target holding epochs. Strikingly, as reported recently in motor cortex, execution was subdivided into distinct, arm acceleration- and deceleration-related, states. These results suggest that dynamics across parieto-frontal areas are highly consistent and hint that parietal population activity largely reflects timing constraints while motor actions unfold.

A Poisson generalized linear model application to disentangle the effects of various parameters on neurophysiological discharges

The protocol provides an extensive guide to apply the generalized linear model framework to neurophysiological recordings. This flexible technique can be adapted to test and quantify the contributions of many different parameters (e.g., kinematics, target position, choice, reward) on neural activity. To weight the influence of each parameter, we developed an intuitive metric (“w-value”) that can be used to build a “functional fingerprint” characteristic for each neuron. We also provide suggestions to extract complementary useful information from the method.

For complete details on the use and execution of this protocol, please refer to Diomedi et al. (2020).

•

GLM applied to neuronal discharges reveals parameters that modulate their activity

•

Applying regularizers helps to discard minority parameters and reduces noise

•

Each neuron can be characterized by the weight assigned to each parameter tested

•

Results are well suited for population analyses (i.e., clustering, correlation, etc)

The Neurophysiological Representation of Imagined Somatosensory Percepts in Human Cortex

Intracortical microstimulation (ICMS) in human primary somatosensory cortex (S1) has been used to successfully evoke naturalistic sensations. However, the neurophysiological mechanisms underlying the evoked sensations remain unknown. To understand how specific stimulation parameters elicit certain sensations we must first understand the representation of those sensations in the brain. In this study we record from intracortical microelectrode arrays implanted in S1, premotor cortex, and posterior parietal cortex of a male human participant performing a somatosensory imagery task. The sensations imagined were those previously elicited by ICMS of S1, in the same array of the same participant. In both spike and local field potential recordings, features of the neural signal can be used to classify different imagined sensations. These features are shown to be stable over time. The sensorimotor cortices only encode the imagined sensation during the imagery task, while posterior parietal cortex encodes the sensations starting with cue presentation. These findings demonstrate that different aspects of the sensory experience can be individually decoded from intracortically recorded human neural signals across the cortical sensory network. Activity underlying these unique sensory representations may inform the stimulation parameters for precisely eliciting specific sensations via ICMS in future work.

SIGNIFICANCE STATEMENT Electrical stimulation of human cortex is increasingly more common for providing feedback in neural devices. Understanding the relationship between naturally evoked and artificially evoked neurophysiology for the same sensations will be important in advancing such devices. Here, we investigate the neural activity in human primary somatosensory, premotor, and parietal cortices during somatosensory imagery. The sensations imagined were those previously elicited during intracortical microstimulation (ICMS) of the same somatosensory electrode array. We elucidate the neural features during somatosensory imagery that significantly encode different aspects of individual sensations and demonstrate feature stability over almost a year. The correspondence between neurophysiology elicited with or without stimulation for the same sensations will inform methods to deliver more precise feedback through stimulation in the future.

Neural encoding of actual and imagined touch within human posterior parietal cortex

In the human posterior parietal cortex (PPC), single units encode high-dimensional information with partially mixed representations that enable small populations of neurons to encode many variables relevant to movement planning, execution, cognition, and perception. Here, we test whether a PPC neuronal population previously demonstrated to encode visual and motor information is similarly engaged in the somatosensory domain. We recorded neurons within the PPC of a human clinical trial participant during actual touch presentation and during a tactile imagery task. Neurons encoded actual touch at short latency with bilateral receptive fields, organized by body part, and covered all tested regions. The tactile imagery task evoked body part-specific responses that shared a neural substrate with actual touch. Our results are the first neuron-level evidence of touch encoding in human PPC and its cognitive engagement during a tactile imagery task, which may reflect semantic processing, attention, sensory anticipation, or imagined touch.

The human primary somatosensory cortex encodes imagined movement in the absence of sensory information

Classical systems neuroscience positions primary sensory areas as early feed-forward processing stations for refining incoming sensory information. This view may oversimplify their role given extensive bi-directional connectivity with multimodal cortical and subcortical regions. Here we show that single units in human primary somatosensory cortex encode imagined reaches in a cognitive motor task, but not other sensory–motor variables such as movement plans or imagined arm position. A population reference-frame analysis demonstrates coding relative to the cued starting hand location suggesting that imagined reaching movements are encoded relative to imagined limb position. These results imply a potential role for primary somatosensory cortex in cognitive imagery, engagement during motor production in the absence of sensation or expected sensation, and suggest that somatosensory cortex can provide control signals for future neural prosthetic systems.

Matiar Jafari, Tyson Aflalo et al. show that the human primary somatosensory cortex is activated when subjects imagine reaches in a cognitive motor task, but not when they plan movement or imagine a static limb position. These results highlight a role for this region in cognitive imagery and motor control in the absence of sensory information.

A shared neural substrate for action verbs and observed actions in human posterior parietal cortex

High-level sensory and motor cortical areas are activated when processing the meaning of language, but it is unknown whether, and how, words share a neural substrate with corresponding sensorimotor representations. We recorded from single neurons in human posterior parietal cortex (PPC) while participants viewed action verbs and corresponding action videos from multiple views. We find that PPC neurons exhibit a common neural substrate for action verbs and observed actions. Further, videos were encoded with mixtures of invariant and idiosyncratic responses across views. Action verbs elicited selective responses from a fraction of these invariant and idiosyncratic neurons, without preference, thus associating with a statistical sampling of the diverse sensory representations related to the corresponding action concept. Controls indicated that the results are not the product of visual imagery or arbitrary learned associations. Our results suggest that language may activate the consolidated visual experience of the reader.

Mixed Selectivity in Macaque Medial Parietal Cortex during Eye-Hand Reaching

The activity of neurons of the medial posterior parietal area V6A in macaque monkeys is modulated by many aspects of reach task. In the past, research was mostly focused on modulating the effect of single parameters upon the activity of V6A cells. Here, we used Generalized Linear Models (GLMs) to simultaneously test the contribution of several factors upon V6A cells during a fix-to-reach task. This approach resulted in the definition of a representative “functional fingerprint” for each neuron. We first studied how the features are distributed in the population. Our analysis highlighted the virtual absence of units strictly selective for only one factor and revealed that most cells are characterized by “mixed selectivity.” Then, exploiting our GLM framework, we investigated the dynamics of spatial parameters encoded within V6A. We found that the tuning is not static, but changed along the trial, indicating the sequential occurrence of visuospatial transformations helpful to guide arm movement.

•

The parietal cortex integrates a variety of sensorimotor inputs to guide reaching

•

GLM disentangled the effect of various reaching parameters upon cell activity

•

V6A neurons were not functionally clustered, but characterized by mixed selectivity

•

Spatial selectivity was dynamic and reached its peak during the movement phase

The Motor Cortex Has Independent Representations for Ipsilateral and Contralateral Arm Movements But Correlated Representations for Grasping

Motor commands for the arm and hand generally arise from the contralateral motor cortex, where most of the relevant corticospinal tract originates. However, the ipsilateral motor cortex shows activity related to arm movement despite the lack of direct connections. The extent to which the activity related to ipsilateral movement is independent from that related to contralateral movement is unclear based on conflicting conclusions in prior work. Here we investigate bilateral arm and hand movement tasks completed by two human subjects with intracortical microelectrode arrays implanted in the left hand and arm area of the motor cortex. Neural activity was recorded while they attempted to perform arm and hand movements in a virtual environment. This enabled us to quantify the strength and independence of motor cortical activity related to continuous movements of each arm. We also investigated the subjects' ability to control both arms through a brain-computer interface. Through a number of experiments, we found that ipsilateral arm movement was represented independently of, but more weakly than, contralateral arm movement. However, the representation of grasping was correlated between the two hands. This difference between hand and arm representation was unexpected and poses new questions about the different ways the motor cortex controls the hands and arms.

Multidimensional processing in the amygdala

Brain-wide circuits that coordinate affective and social behaviours intersect in the amygdala. Consequently, amygdala lesions cause a heterogeneous array of social and non-social deficits. Social behaviours are not localized to subdivisions of the amygdala even though the inputs and outputs that carry social signals are anatomically restricted to distinct subnuclear regions. This observation may be explained by the multidimensional response properties of the component neurons. Indeed, the multitudes of circuits that converge in the amygdala enlist the same subset of neurons into different ensembles that combine social and non-social elements into high-dimensional representations. These representations may enable flexible, context-dependent social decisions. As such, multidimensional processing may operate in parallel with subcircuits of genetically identical neurons that serve specialized and functionally dissociable functions. When combined, the activity of specialized circuits may grant specificity to social behaviours, whereas multidimensional processing facilitates the flexibility and nuance needed for complex social behaviour.

Decoding of standard and non-standard visuomotor associations from parietal cortex

OBJECTIVE

Neural signals can be decoded and used to move neural prostheses with the purpose of restoring motor function in patients with mobility impairments. Such patients typically have intact eye movement control and visual function, suggesting that cortical visuospatial signals could be used to guide external devices. Neurons in parietal cortex mediate sensory-motor transformations, encode the spatial coordinates for reaching goals, hand position and movements, and other spatial variables. We studied how spatial information is represented at the population level, and the possibility to decode not only the position of visual targets and the plans to reach them, but also conditional, non-spatial motor responses.

APPROACH

The animals first fixated one of nine targets in 3D space and then, after the target changed color, either reached toward it, or performed a non-spatial motor response (lift hand from a button). Spiking activity of parietal neurons was recorded in monkeys during two tasks. We then decoded different task related parameters.

MAIN RESULTS

We first show that a maximum-likelihood estimation (MLE) algorithm trained separately in each task transformed neural activity into accurate metric predictions of target location. Furthermore, by combining MLE with a Naïve Bayes classifier, we decoded the monkey's motor intention (reach or hand lift) and the different phases of the tasks. These results show that, although V6A encodes the spatial location of a target during a delay period, the signals they carry are updated around the movement execution in an intention/motor specific way.

SIGNIFICANCE

These findings show the presence of multiple levels of information in parietal cortex that could be decoded and used in brain machine interfaces to control both goal-directed movements and more cognitive visuomotor associations.

Posterior parietal cortex contributions to cross-modal brain plasticity upon sensory loss

Sensory loss causes compensatory behavior, like echolocation upon vision loss or improved visual motion detection upon deafness. This is enabled by recruitment of the deprived cortical area by the intact senses. Such cross-modal plasticity can however hamper rehabilitation via sensory substitution devices. To steer rehabilitation towards the desired outcome for the patient, having control over the cross-modal take-over is essential. Evidence accumulates to support a role for the posterior parietal cortex (PPC) in multimodal plasticity. This area shows increased activity after sensory loss, keeping similar functions but driven by other senses. Patient-specific factors like stress, social situation, age and attention, have a significant influence on the PPC and on cross-modal plasticity. We propose that understanding the response of the PPC to sensory loss and context is extremely important for determining the best possible implant-based therapies, and that mouse research holds potential to help unraveling the underlying anatomical, cellular and neuromodulatory mechanisms.

Stable readout of observed actions from format-dependent activity of monkey's anterior intraparietal neurons

The anterior intraparietal area (AIP) is a crucial hub in the observed manipulative action (OMA) network of primates. While macaques observe manipulative action videos, their AIP neuronal activity robustly encodes first the viewpoint from which the action is observed, then the actor’s body posture, and finally the observed-action identity. Despite the lack of fully invariant OMA-selective single neurons, OMA exemplars could be decoded accurately from the activity of a set of units that maintain stable OMA selectivity despite rescaling their firing rate across formats. We propose that by integrating signals multiplicatively about others’ action and their visual format, the AIP can provide a stable readout of OMA identity at the population level.

Humans accurately identify observed actions despite large dynamic changes in their retinal images and a variety of visual presentation formats. A large network of brain regions in primates participates in the processing of others’ actions, with the anterior intraparietal area (AIP) playing a major role in routing information about observed manipulative actions (OMAs) to the other nodes of the network. This study investigated whether the AIP also contributes to invariant coding of OMAs across different visual formats. We recorded AIP neuronal activity from two macaques while they observed videos portraying seven manipulative actions (drag, drop, grasp, push, roll, rotate, squeeze) in four visual formats. Each format resulted from the combination of two actor’s body postures (standing, sitting) and two viewpoints (lateral, frontal). Out of 297 recorded units, 38% were OMA-selective in at least one format. Robust population code for viewpoint and actor’s body posture emerged shortly after stimulus presentation, followed by OMA selectivity. Although we found no fully invariant OMA-selective neuron, we discovered a population code that allowed us to classify action exemplars irrespective of the visual format. This code depends on a multiplicative mixing of signals about OMA identity and visual format, particularly evidenced by a set of units maintaining a relatively stable OMA selectivity across formats despite considerable rescaling of their firing rate depending on the visual specificities of each format. These findings suggest that the AIP integrates format-dependent information and the visual features of others’ actions, leading to a stable readout of observed manipulative action identity.

Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way

Decades after the motor homunculus was first proposed, it is still unknown how different body parts are intermixed and interrelated in human motor cortical areas at single-neuron resolution. Using multi-unit recordings, we studied how face, head, arm, and leg movements are represented in the hand knob area of premotor cortex (precentral gyrus) in people with tetraplegia. Contrary to traditional expectations, we found strong representation of all movements and a partially "compositional" neural code that linked together all four limbs. The code consisted of (1) a limb-coding component representing the limb to be moved and (2) a movement-coding component where analogous movements from each limb (e.g., hand grasp and toe curl) were represented similarly. Compositional coding might facilitate skill transfer across limbs, and it provides a useful framework for thinking about how the motor system constructs movement. Finally, we leveraged these results to create a whole-body intracortical brain-computer interface that spreads targets across all limbs.

Preservation of Partially Mixed Selectivity in Human Posterior Parietal Cortex across Changes in Task Context

Recent studies in posterior parietal cortex (PPC) have found multiple effectors and cognitive strategies represented within a shared neural substrate in a structure termed "partially mixed selectivity" (Zhang et al., 2017). In this study, we examine whether the structure of these representations is preserved across changes in task context and is thus a robust and generalizable property of the neural population. Specifically, we test whether the structure is conserved from an open-loop motor imagery task (training) to a closed-loop cortical control task (online), a change that has led to substantial changes in neural behavior in prior studies in motor cortex. Recording from a 4 × 4 mm electrode array implanted in PPC of a human tetraplegic patient participating in a brain-machine interface (BMI) clinical trial, we studied the representations of imagined/attempted movements of the left/right hand and compare their individual BMI control performance using a one-dimensional cursor control task. We found that the structure of the representations is largely maintained between training and online control. Our results demonstrate for the first time that the structure observed in the context of an open-loop motor imagery task is maintained and accessible in the context of closed-loop BMI control. These results indicate that it is possible to decode the mixed variables found from a small patch of cortex in PPC and use them individually for BMI control. Furthermore, they show that the structure of the mixed representations is maintained and robust across changes in task context.

Tactile Perception: Beyond the Somatotopy of the Somatosensory Cortex

New research demonstrates systematic errors of tactile localisation, involving confusions of body parts and body sides. Such errors do not follow the organisation of topographic maps in somatosensory cortex, suggesting that tactile localisation involves coding of abstract features of limbs.

From thought to action: The brain-machine interface in posterior parietal cortex

A dramatic example of translational monkey research is the development of neural prosthetics for assisting paralyzed patients. A neuroprosthesis consists of implanted electrodes that can record the intended movement of a paralyzed part of the body, a computer algorithm that decodes the intended movement, and an assistive device such as a robot limb or computer that is controlled by these intended movement signals. This type of neuroprosthetic system is also referred to as a brain-machine interface (BMI) since it interfaces the brain with an external machine. In this review, we will concentrate on BMIs in which microelectrode recording arrays are implanted in the posterior parietal cortex (PPC), a high-level cortical area in both humans and monkeys that represents intentions to move. This review will first discuss the basic science research performed in healthy monkeys that established PPC as a good source of intention signals. Next, it will describe the first PPC implants in human patients with tetraplegia from spinal cord injury. From these patients the goals of movements could be quickly decoded, and the rich number of action variables found in PPC indicates that it is an appropriate BMI site for a very wide range of neuroprosthetic applications. We will discuss research on learning to use BMIs in monkeys and humans and the advances that are still needed, requiring both monkey and human research to enable BMIs to be readily available in the clinic.

No effect of triple-pulse TMS medial to intraparietal sulcus on online correction for target perturbations during goal-directed hand and foot reaches

Posterior parietal cortex (PPC) is central to sensorimotor processing for goal-directed hand and foot movements. Yet, the specific role of PPC subregions in these functions is not clear. Previous human neuroimaging and transcranial magnetic stimulation (TMS) work has suggested that PPC lateral to the intraparietal sulcus (IPS) is involved in directing the arm, shaping the hand, and correcting both finger-shaping and hand trajectory during movement. The lateral localization of these functions agrees with the comparably lateral position of the hand and fingers within the motor and somatosensory homunculi along the central sulcus; this might suggest that, in analogy, (goal-directed) foot movements would be mediated by medial portions of PPC. However, foot movement planning activates similar regions for both hand and foot movement along the caudal-to-rostral axis of PPC, with some effector-specificity evident only rostrally, near the central regions of sensorimotor cortex. Here, we attempted to test the causal involvement of PPC regions medial to IPS in hand and foot reaching as well as online correction evoked by target displacement. Participants made hand and foot reaches towards identical visual targets. Sometimes, the target changed position 100–117 ms into the movement. We disturbed cortical processing over four positions medial to IPS with three pulses of TMS separated by 40 ms, both during trials with and without target displacement. We timed TMS to disrupt reach execution and online correction. TMS did not affect endpoint error, endpoint variability, or reach trajectories for hand or foot. While these negative results await replication with different TMS timing and parameters, we conclude that regions medial to IPS are involved in planning, rather than execution and online control, of goal-directed limb movements.

Multidimensional Neural Selectivity in the Primate Amygdala

The amygdala contributes to multiple functions including attention allocation, sensory processing, decision-making, and the elaboration of emotional behaviors. The diversity of functions attributed to the amygdala is reflected in the response selectivity of its component neurons. Previous work claimed that subsets of neurons differentiate between broad categories of stimuli (e.g., objects vs faces, rewards vs punishment), while other subsets are narrowly specialized to respond to individual faces or facial features (e.g., eyes). Here we explored the extent to which the same neurons contribute to more than one neural subpopulation in a task that activated multiple functions of the amygdala. The subjects (Macaca mulatta) watched videos depicting conspecifics or inanimate objects, and learned by trial and error to choose the individuals or objects associated with the highest rewards. We found that the same neurons responded selectively to two or more of the following task events or stimulus features: (1) alerting, task-related stimuli (fixation icon, video start, and video end); (2) reward magnitude; (3) stimulus categories (social vs nonsocial); and (4) stimulus-unique features (faces, eyes). A disproportionate number of neurons showed selectivity for all of the examined stimulus features and task events. These results suggest that neurons that appear specialized and uniquely tuned to specific stimuli (e.g., face cells, eye cells) are likely to respond to multiple other types of stimuli or behavioral events, if/when these become behaviorally relevant in the context of a complex task. This multidimensional selectivity supports a flexible, context-dependent evaluation of inputs and subsequent decision making based on the activity of the same neural ensemble.

An Information-Driven 2-Pathway Characterization of Occipitotemporal and Posterior Parietal Visual Object Representations

Recent studies have demonstrated the existence of rich visual representations in both occipitotemporal cortex (OTC) and posterior parietal cortex (PPC). Using fMRI decoding and a bottom-up data-driven approach, we showed that although robust object category representations exist in both OTC and PPC, there is an information-driven 2-pathway separation among these regions in the representational space, with occipitotemporal regions arranging hierarchically along 1 pathway and posterior parietal regions along another pathway. We obtained 10 independent replications of this 2-pathway distinction, accounting for 58-81% of the total variance of the region-wise differences in visual representation. The separation of the PPC regions from higher occipitotemporal regions was not driven by a difference in tolerance to changes in low-level visual features, did not rely on the presence of special object categories, and was present whether or not object category was task relevant. Our information-driven 2-pathway structure differs from the well-known ventral-what and dorsal-where/how characterization of posterior brain regions. Here both pathways contain rich nonspatial visual representations. The separation we see likely reflects a difference in neural coding scheme used by PPC to represent visual information compared with that of OTC.

Feeling a Touch to the Hand on the Foot

Where we perceive a touch putatively depends on topographic maps that code the touch's location on the skin [1] as well as its position in external space [2-5]. However, neither somatotopic nor external-spatial representations can account for atypical tactile percepts in some neurological patients and amputees; referral of touch to an absent or anaesthetized hand after stimulation of a foot [6, 7] or the contralateral hand [8-10] challenges the role of topographic representations when attributing touch to the limbs. Here, we show that even healthy adults systematically misattribute touch to other limbs. Participants received two tactile stimuli, each to a different limb-hand or foot-and reported which of all four limbs had been stimulated first. Hands and feet were either uncrossed or crossed to dissociate body-based and external-spatial representations [11-14]. Remarkably, participants regularly attributed the first touch to a limb that had received neither of the two stimuli. The erroneously reported, non-stimulated limb typically matched the correct limb with respect to limb type or body side. Touch was misattributed to non-stimulated limbs of the other limb type and body side only if they were placed at the correct limb's canonical (default) side of space. The touch's actual location in external space was irrelevant. These errors replicated across several contexts, and modeling linked them to incoming sensory evidence rather than to decision strategies. The results highlight the importance of the touched body part's identity and canonical location but challenge the role of external-spatial tactile representations when attributing touch to a limb.

Brain-Inspired Coding of Robot Body Schema Through Visuo-Motor Integration of Touched Events

Representing objects in space is difficult because sensorimotor events are anchored in different reference frames, which can be either eye-, arm-, or target-centered. In the brain, Gain-Field (GF) neurons in the parietal cortex are involved in computing the necessary spatial transformations for aligning the tactile, visual and proprioceptive signals. In reaching tasks, these GF neurons exploit a mechanism based on multiplicative interaction for binding simultaneously touched events from the hand with visual and proprioception information.By doing so, they can infer new reference frames to represent dynamically the location of the body parts in the visual space (i.e., the body schema) and nearby targets (i.e., its peripersonal space). In this line, we propose a neural model based on GF neurons for integrating tactile events with arm postures and visual locations for constructing hand- and target-centered receptive fields in the visual space. In robotic experiments using an artificial skin, we show how our neural architecture reproduces the behaviors of parietal neurons (1) for encoding dynamically the body schema of our robotic arm without any visual tags on it and (2) for estimating the relative orientation and distance of targets to it. We demonstrate how tactile information facilitates the integration of visual and proprioceptive signals in order to construct the body space.

Mixed Spatial and Movement Representations in the Primate Posterior Parietal Cortex

The posterior parietal cortex (PPC) of humans and non-human primates plays a key role in the sensory and motor transformations required to guide motor actions to objects of interest in the environment. Despite decades of research, the anatomical and functional organization of this region is still a matter of contention. It is generally accepted that specialized parietal subregions and their functional counterparts in the frontal cortex participate in distinct segregated networks related to eye, arm and hand movements. However, experimental evidence obtained primarily from single neuron recording studies in non-human primates has demonstrated a rich mixing of signals processed by parietal neurons, calling into question ideas for a strict functional specialization. Here, we present a brief account of this line of research together with the basic trends in the anatomical connectivity patterns of the parietal subregions. We review, the evidence related to the functional communication between subregions of the PPC and describe progress towards using parietal neuron activity in neuroprosthetic applications. Recent literature suggests a role for the PPC not as a constellation of specialized functional subdomains, but as a dynamic network of sensorimotor loci that combine multiple signals and work in concert to guide motor behavior.

Intrinsic Variable Learning for Brain-Machine Interface Control by Human Anterior Intraparietal Cortex

Although animal studies provided significant insights in understanding the neural basis of learning and adaptation, they often cannot dissociate between different learning mechanisms due to the lack of verbal communication. To overcome this limitation, we examined the mechanisms of learning and its limits in a human intracortical brain-machine interface (BMI) paradigm. A tetraplegic participant controlled a 2D computer cursor by modulating single-neuron activity in the anterior intraparietal area (AIP). By perturbing the neuron-to-movement mapping, the participant learned to modulate the activity of the recorded neurons to solve the perturbations by adopting a target re-aiming strategy. However, when no cognitive strategies were adequate to produce correct responses, AIP failed to adapt to perturbations. These findings suggest that learning is constrained by the pre-existing neuronal structure, although it is possible that AIP needs more training time to learn to generate novel activity patterns when cognitive re-adaptation fails to solve the perturbations.

Cognition in Sensorimotor Control: Interfacing With the Posterior Parietal Cortex

Millions of people worldwide are afflicted with paralysis from a disruption of neural pathways between the brain and the muscles. Because their cortical architecture is often preserved, these patients are able to plan movements despite an inability to execute them. In such people, brain machine interfaces have great potential to restore lost function through neuroprosthetic devices, circumventing dysfunctional corticospinal circuitry. These devices have typically derived control signals from the motor cortex (M1) which provides information highly correlated with desired movement trajectories. However, sensorimotor control simultaneously engages multiple cognitive processes such as intent, state estimation, decision making, and the integration of multisensory feedback. As such, cortical association regions upstream of M1 such as the posterior parietal cortex (PPC) that are involved in higher order behaviors such as planning and learning, rather than in encoding movement itself, may enable enhanced, cognitive control of neuroprosthetics, termed cognitive neural prosthetics (CNPs). We illustrate in this review, through a small sampling, the cognitive functions encoded in the PPC and discuss their neural representation in the context of their relevance to motor neuroprosthetics. We aim to highlight through examples a role for cortical signals from the PPC in developing CNPs, and to inspire future avenues for exploration in their research and development.

Watch, Imagine, Attempt: Motor Cortex Single-Unit Activity Reveals Context-Dependent Movement Encoding in Humans With Tetraplegia

Planning and performing volitional movement engages widespread networks in the human brain, with motor cortex considered critical to the performance of skilled limb actions. Motor cortex is also engaged when actions are observed or imagined, but the manner in which ensembles of neurons represent these volitional states (VoSs) is unknown. Here we provide direct demonstration that observing, imagining or attempting action activates shared neural ensembles in human motor cortex. Two individuals with tetraplegia (due to brainstem stroke or amyotrophic lateral sclerosis, ALS) were verbally instructed to watch, imagine, or attempt reaching actions displayed on a computer screen. Neural activity in the precentral gyrus incorporated information about both cognitive state and movement kinematics; the three conditions presented overlapping but unique, statistically distinct activity patterns. These findings demonstrate that individual neurons in human motor cortex reflect information related to sensory inputs and VoS in addition to movement features, and are a key part of a broader network linking perception and cognition to action.

Single-Neuron Representation of Memory Strength and Recognition Confidence in Left Human Posterior Parietal Cortex

The human posterior parietal cortex (PPC) is thought to contribute to memory retrieval, but little is known about its specific role. We recorded single PPC neurons of two human tetraplegic subjects implanted with microelectrode arrays, who performed a recognition memory task. We found two groups of neurons that signaled memory-based choices. Memory-selective neurons preferred either novel or familiar stimuli, scaled their response as a function of confidence, and signaled subjective choices regardless of truth. Confidence-selective neurons signaled confidence regardless of stimulus familiarity. Memory-selective signals appeared 553 ms after stimulus onset, but before action onset. Neurons also encoded spoken numbers, but these number-tuned neurons did not carry recognition signals. Together, this functional separation reveals action-independent coding of declarative memory-based familiarity and confidence of choices in human PPC. These data suggest that, in addition to sensory-motor integration, a function of human PPC is to utilize memory signals to make choices.