The language of the brain: real-world neural population codes

The geometry of the neural state space of decisions

How do populations of neurons collectively encode and process information during cognitive tasks? We analyze high-yield population recordings from the macaque lateral intraparietal area (LIP) during a reaction-time random-dot-motion direction-discrimination task. We find that the trajectories of neural population activity patterns during single decisions lie within a curved two-dimensional manifold. The reaction time of trajectories systematically varies along one dimension, such that slow and fast decisions trace distinct activity patterns. Trajectories transition from a deliberation stage, in which they are noisy and remain similar between the choices, to a commitment stage, in which they are far less noisy and diverge sharply for the different choices. The deliberation phase is pronounced for slower decisions and gradually diminishes as reaction time decreases. A mechanistic circuit model provides an explanation for the observed properties, and suggests the transition between stages represents a transition from more sensory-driven to more circuit-driven dynamics. It yields two striking predictions we verify in the data. First, whether neurons are more choice selective for slow or fast trials varies systematically with the retinotopic location of their response fields. Second, the slower the trial, the more saccades undershoot the choice target. The results highlight the roles of distributed and dynamic activity patterns and intrinsic circuit dynamics in the population implementation of a cognitive task.

Micro-electrode array recording of extracellular electrical potentials of liquid static surface fermented Hericium erinaceus

Hericium erinaceus is a basidiomycetes fungus with previously uncharacterised extracellular electrophysiology. Here, we present results of recordings of the electrical potentials of fungal biofilms of this species using microelectrode arrays (MEAs). In particular, we focused on modelling the temporal and spatial progression of the low frequency (≤ 1 Hz) potentials. Culture media control studies showed that the electrical potential activity results from the growth and subsequent spiking behaviours of the mycelium extracellular matrices. An antifungal assay using nystatin suspension, 10,000 unit/mL in DPBS, provided evidence for the biological origin of electrical potentials due to targeting of the selective permeability of the cell membrane and subsequent cessation of electrical activity. Conversely, injection of L-glutamic acid increased the combined multi-channel mean firing rate from 0.04 Hz to 0.1 Hz. Analysis of bursting and spatial propagation of the extracellular signals are also presented.

Multiunit Frontal Eye Field Activity Codes the Visuomotor Transformation, But Not Gaze Prediction or Retrospective Target Memory, in a Delayed Saccade Task

Single-unit (SU) activity-action potentials isolated from one neuron-has traditionally been employed to relate neuronal activity to behavior. However, recent investigations have shown that multiunit (MU) activity-ensemble neural activity recorded within the vicinity of one microelectrode-may also contain accurate estimations of task-related neural population dynamics. Here, using an established model-fitting approach, we compared the spatial codes of SU response fields with corresponding MU response fields recorded from the frontal eye fields (FEFs) in head-unrestrained monkeys (Macaca mulatta) during a memory-guided saccade task. Overall, both SU and MU populations showed a simple visuomotor transformation: the visual response coded target-in-eye coordinates, transitioning progressively during the delay toward a future gaze-in-eye code in the saccade motor response. However, the SU population showed additional secondary codes, including a predictive gaze code in the visual response and retention of a target code in the motor response. Further, when SUs were separated into regular/fast spiking neurons, these cell types showed different spatial code progressions during the late delay period, only converging toward gaze coding during the final saccade motor response. Finally, reconstructing MU populations (by summing SU data within the same sites) failed to replicate either the SU or MU pattern. These results confirm the theoretical and practical potential of MU activity recordings as a biomarker for fundamental sensorimotor transformations (e.g., target-to-gaze coding in the oculomotor system), while also highlighting the importance of SU activity for coding more subtle (e.g., predictive/memory) aspects of sensorimotor behavior.

Identifying Distinct Neural Features between the Initial and Corrective Phases of Precise Reaching Using AutoLFADS

Many initial movements require subsequent corrective movements, but how the motor cortex transitions to make corrections and how similar the encoding is to initial movements is unclear. In our study, we explored how the brain's motor cortex signals both initial and corrective movements during a precision reaching task. We recorded a large population of neurons from two male rhesus macaques across multiple sessions to examine the neural firing rates during not only initial movements but also subsequent corrective movements. AutoLFADS, an autoencoder-based deep-learning model, was applied to provide a clearer picture of neurons' activity on individual corrective movements across sessions. Decoding of reach velocity generalized poorly from initial to corrective submovements. Unlike initial movements, it was challenging to predict the velocity of corrective movements using traditional linear methods in a single, global neural space. We identified several locations in the neural space where corrective submovements originated after the initial reaches, signifying firing rates different than the baseline before initial movements. To improve corrective movement decoding, we demonstrate that a state-dependent decoder incorporating the population firing rates at the initiation of correction improved performance, highlighting the diverse neural features of corrective movements. In summary, we show neural differences between initial and corrective submovements and how the neural activity encodes specific combinations of velocity and position. These findings are inconsistent with assumptions that neural correlations with kinematic features are global and independent, emphasizing that traditional methods often fall short in describing these diverse neural processes for online corrective movements.

Adaptation-induced sharpening of orientation tuning curves in the mouse visual cortex

OBJECTIVE

Orientation selectivity is an emergent property of visual neurons across species with columnar and noncolumnar organization of the visual cortex. The emergence of orientation selectivity is more established in columnar cortical areas than in noncolumnar ones. Thus, how does orientation selectivity emerge in noncolumnar cortical areas after an adaptation protocol? Adaptation refers to the constant presentation of a nonoptimal stimulus (adapter) to a neuron under observation for a specific time. Previously, it had been shown that adaptation has varying effects on the tuning properties of neurons, such as orientation, spatial frequency, motion and so on.

BASIC METHODS

We recorded the mouse primary visual neurons (V1) at different orientations in the control (preadaptation) condition. This was followed by adapting neurons uninterruptedly for 12 min and then recording the same neurons postadaptation. An orientation selectivity index (OSI) for neurons was computed to compare them pre- and post-adaptation.

MAIN RESULTS

We show that 12-min adaptation increases the OSI of visual neurons ( n  = 113), that is, sharpens their tuning. Moreover, the OSI postadaptation increases linearly as a function of the OSI preadaptation.

CONCLUSION

The increased OSI postadaptation may result from a specific dendritic neural mechanism, potentially facilitating the rapid learning of novel features.

Identifying distinct neural features between the initial and corrective phases of precise reaching using AutoLFADS

Many initial movements require subsequent corrective movements, but how motor cortex transitions to make corrections and how similar the encoding is to initial movements is unclear. In our study, we explored how the brain's motor cortex signals both initial and corrective movements during a precision reaching task. We recorded a large population of neurons from two male rhesus macaques across multiple sessions to examine the neural firing rates during not only initial movements but also subsequent corrective movements. AutoLFADS, an auto-encoder-based deep-learning model, was applied to provide a clearer picture of neurons' activity on individual corrective movements across sessions. Decoding of reach velocity generalized poorly from initial to corrective submovements. Unlike initial movements, it was challenging to predict the velocity of corrective movements using traditional linear methods in a single, global neural space. We identified several locations in the neural space where corrective submovements originated after the initial reaches, signifying firing rates different than the baseline before initial movements. To improve corrective movement decoding, we demonstrate that a state-dependent decoder incorporating the population firing rates at the initiation of correction improved performance, highlighting the diverse neural features of corrective movements. In summary, we show neural differences between initial and corrective submovements and how the neural activity encodes specific combinations of velocity and position. These findings are inconsistent with assumptions that neural correlations with kinematic features are global and independent, emphasizing that traditional methods often fall short in describing these diverse neural processes for online corrective movements.

Significance Statement

We analyzed submovement neural population dynamics during precision reaching. Using an auto- encoder-based deep-learning model, AutoLFADS, we examined neural activity on a single-trial basis. Our study shows distinct neural dynamics between initial and corrective submovements. We demonstrate the existence of unique neural features within each submovement class that encode complex combinations of position and reach direction. Our study also highlights the benefit of state-specific decoding strategies, which consider the neural firing rates at the onset of any given submovement, when decoding complex motor tasks such as corrective submovements.

Applications of information geometry to spiking neural network activity

The space of possible behaviors that complex biological systems may exhibit is unimaginably vast, and these systems often appear to be stochastic, whether due to variable noisy environmental inputs or intrinsically generated chaos. The brain is a prominent example of a biological system with complex behaviors. The number of possible patterns of spikes emitted by a local brain circuit is combinatorially large, although the brain may not make use of all of them. Understanding which of these possible patterns are actually used by the brain, and how those sets of patterns change as properties of neural circuitry change is a major goal in neuroscience. Recently, tools from information geometry have been used to study embeddings of probabilistic models onto a hierarchy of model manifolds that encode how model outputs change as a function of their parameters, giving a quantitative notion of "distances" between outputs. We apply this method to a network model of excitatory and inhibitory neural populations to understand how the competition between membrane and synaptic response timescales shapes the network's information geometry. The hyperbolic embedding allows us to identify the statistical parameters to which the model behavior is most sensitive, and demonstrate how the ranking of these coordinates changes with the balance of excitation and inhibition in the network.

Integration of landmark and saccade target signals in macaque frontal cortex visual responses

Visual landmarks influence spatial cognition and behavior, but their influence on visual codes for action is poorly understood. Here, we test landmark influence on the visual response to saccade targets recorded from 312 frontal and 256 supplementary eye field neurons in rhesus macaques. Visual response fields are characterized by recording neural responses to various target-landmark combinations, and then we test against several candidate spatial models. Overall, frontal/supplementary eye fields response fields preferentially code either saccade targets (40%/40%) or landmarks (30%/4.5%) in gaze fixation-centered coordinates, but most cells show multiplexed target-landmark coding within intermediate reference frames (between fixation-centered and landmark-centered). Further, these coding schemes interact: neurons with near-equal target and landmark coding show the biggest shift from fixation-centered toward landmark-centered target coding. These data show that landmark information is preserved and influences target coding in prefrontal visual responses, likely to stabilize movement goals in the presence of noisy egocentric signals.

Multiscalar electrical spiking in Schizophyllum commune

Growing colonies of the split-gill fungus Schizophyllum commune show action potential-like spikes of extracellular electrical potential. We analysed several days of electrical activity recording of the fungus and discovered three families of oscillatory patterns. Very slow activity at a scale of hours, slow activity at a scale of 10 min and very fast activity at scale of half-minute. We simulated the spiking behaviour using FitzHugh-Nagume model, uncovered mechanisms of spike shaping. We speculated that spikes of electrical potential might be associated with transportation of nutrients and metabolites.

Neurodynamical Computing at the Information Boundaries of Intelligent Systems

Artificial intelligence has not achieved defining features of biological intelligence despite models boasting more parameters than neurons in the human brain. In this perspective article, we synthesize historical approaches to understanding intelligent systems and argue that methodological and epistemic biases in these fields can be resolved by shifting away from cognitivist brain-as-computer theories and recognizing that brains exist within large, interdependent living systems. Integrating the dynamical systems view of cognition with the massive distributed feedback of perceptual control theory highlights a theoretical gap in our understanding of nonreductive neural mechanisms. Cell assemblies-properly conceived as reentrant dynamical flows and not merely as identified groups of neurons-may fill that gap by providing a minimal supraneuronal level of organization that establishes a neurodynamical base layer for computation. By considering information streams from physical embodiment and situational embedding, we discuss this computational base layer in terms of conserved oscillatory and structural properties of cortical-hippocampal networks. Our synthesis of embodied cognition, based in dynamical systems and perceptual control, aims to bypass the neurosymbolic stalemates that have arisen in artificial intelligence, cognitive science, and computational neuroscience.

Precise and stable edge orientation signaling by human first-order tactile neurons

Fast-adapting type 1 (FA-1) and slow-adapting type 1 (SA-1) first-order neurons in the human tactile system have distal axons that branch in the skin and form many transduction sites, yielding receptive fields with many highly sensitive zones or 'subfields.' We previously demonstrated that this arrangement allows FA-1 and SA-1 neurons to signal the geometric features of touched objects, specifically the orientation of raised edges scanned with the fingertips. Here, we show that such signaling operates for fine edge orientation differences (5-20°) and is stable across a broad range of scanning speeds (15-180 mm/s); that is, under conditions relevant for real-world hand use. We found that both FA-1 and SA-1 neurons weakly signal fine edge orientation differences via the intensity of their spiking responses and only when considering a single scanning speed. Both neuron types showed much stronger edge orientation signaling in the sequential structure of the evoked spike trains, and FA-1 neurons performed better than SA-1 neurons. Represented in the spatial domain, the sequential structure was strikingly invariant across scanning speeds, especially those naturally used in tactile spatial discrimination tasks. This speed invariance suggests that neurons' responses are structured via sequential stimulation of their subfields and thus links this capacity to their terminal organization in the skin. Indeed, the spatial precision of elicited action potentials rationally matched spatial acuity of subfield arrangements, which corresponds to a spatial period similar to the dimensions of individual fingertip ridges.

Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Natural vision is dynamic: as an animal moves, its visual input changes dramatically. How can the visual system reliably extract local features from an input dominated by self-generated signals? In Drosophila, diverse local visual features are represented by a group of projection neurons with distinct tuning properties. Here, we describe a connectome-based volumetric imaging strategy to measure visually evoked neural activity across this population. We show that local visual features are jointly represented across the population, and a shared gain factor improves trial-to-trial coding fidelity. A subset of these neurons, tuned to small objects, is modulated by two independent signals associated with self-movement, a motor-related signal, and a visual motion signal associated with rotation of the animal. These two inputs adjust the sensitivity of these feature detectors across the locomotor cycle, selectively reducing their gain during saccades and restoring it during intersaccadic intervals. This work reveals a strategy for reliable feature detection during locomotion.

Learning enhances encoding of time and temporal surprise in mouse primary sensory cortex

Primary sensory cortex has long been believed to play a straightforward role in the initial processing of sensory information. Yet, the superficial layers of cortex overall are sparsely active, even during sensory stimulation; additionally, cortical activity is influenced by other modalities, task context, reward, and behavioral state. Our study demonstrates that reinforcement learning dramatically alters representations among longitudinally imaged neurons in superficial layers of mouse primary somatosensory cortex. Learning an object detection task recruits previously unresponsive neurons, enlarging the neuronal population sensitive to touch and behavioral choice. Cortical responses decrease upon repeated stimulus presentation outside of the behavioral task. Moreover, training improves population encoding of the passage of time, and unexpected deviations in trial timing elicit even stronger responses than touches do. In conclusion, the superficial layers of sensory cortex exhibit a high degree of learning-dependent plasticity and are strongly modulated by non-sensory but behaviorally-relevant features, such as timing and surprise.

Activity in the superficial layers of the sensory cortex is believed to be largely driven by incoming sensory stimuli. Here the authors demonstrate how learning changes neural responses to sensations according to both behavioral relevance and timing, suggesting a high degree of non-sensory modulation.

Language of fungi derived from their electrical spiking activity

Fungi exhibit oscillations of extracellular electrical potential recorded via differential electrodes inserted into a substrate colonized by mycelium or directly into sporocarps. We analysed electrical activity of ghost fungi (Omphalotus nidiformis), Enoki fungi (Flammulina velutipes), split gill fungi (Schizophyllum commune) and caterpillar fungi (Cordyceps militaris). The spiking characteristics are species specific: a spike duration varies from 1 to 21 h and an amplitude from 0.03 to 2.1 mV. We found that spikes are often clustered into trains. Assuming that spikes of electrical activity are used by fungi to communicate and process information in mycelium networks, we group spikes into words and provide a linguistic and information complexity analysis of the fungal spiking activity. We demonstrate that distributions of fungal word lengths match that of human languages. We also construct algorithmic and Liz-Zempel complexity hierarchies of fungal sentences and show that species S. commune generate the most complex sentences.

Information Capacity of a Stochastically Responding Neuron Assembly

In this work, certain aspects of the structure of the overlapping groups of neurons encoding specific signals are examined. Individual neurons are assumed to respond stochastically to input signal. Identification of a particular signal is assumed to result from the aggregate activity of a group of neurons, which we call information pathway. Conditions for definite response and for non-interference of pathways are derived. These conditions constrain the response properties of individual neurons and the allowed overlap among pathways. Under these constrains, and under the simplifying assumption that all pathways have similar structure, the information capacity of the system is derived. Furthermore, we show that there is a definite advantage in the information capacity if pathway neurons areinterspersed among the neuron assembly.

Spatiotemporal Coding in the Macaque Supplementary Eye Fields: Landmark Influence in the Target-to-Gaze Transformation

Eye-centered (egocentric) and landmark-centered (allocentric) visual signals influence spatial cognition, navigation, and goal-directed action, but the neural mechanisms that integrate these signals for motor control are poorly understood. A likely candidate for egocentric/allocentric integration in the gaze control system is the supplementary eye fields (SEF), a mediofrontal structure with high-level “executive” functions, spatially tuned visual/motor response fields, and reciprocal projections with the frontal eye fields (FEF). To test this hypothesis, we trained two head-unrestrained monkeys (Macaca mulatta) to saccade toward a remembered visual target in the presence of a visual landmark that shifted during the delay, causing gaze end points to shift partially in the same direction. A total of 256 SEF neurons were recorded, including 68 with spatially tuned response fields. Model fits to the latter established that, like the FEF and superior colliculus (SC), spatially tuned SEF responses primarily showed an egocentric (eye-centered) target-to-gaze position transformation. However, the landmark shift influenced this default egocentric transformation: during the delay, motor neurons (with no visual response) showed a transient but unintegrated shift (i.e., not correlated with the target-to-gaze transformation), whereas during the saccade-related burst visuomotor (VM) neurons showed an integrated shift (i.e., correlated with the target-to-gaze transformation). This differed from our simultaneous FEF recordings (Bharmauria et al., 2020), which showed a transient shift in VM neurons, followed by an integrated response in all motor responses. Based on these findings and past literature, we propose that prefrontal cortex incorporates landmark-centered information into a distributed, eye-centered target-to-gaze transformation through a reciprocal prefrontal circuit.

Scaling of sensory information in large neural populations shows signatures of information-limiting correlations

How is information distributed across large neuronal populations within a given brain area? Information may be distributed roughly evenly across neuronal populations, so that total information scales linearly with the number of recorded neurons. Alternatively, the neural code might be highly redundant, meaning that total information saturates. Here we investigate how sensory information about the direction of a moving visual stimulus is distributed across hundreds of simultaneously recorded neurons in mouse primary visual cortex. We show that information scales sublinearly due to correlated noise in these populations. We compartmentalized noise correlations into information-limiting and nonlimiting components, then extrapolate to predict how information grows with even larger neural populations. We predict that tens of thousands of neurons encode 95% of the information about visual stimulus direction, much less than the number of neurons in primary visual cortex. These findings suggest that the brain uses a widely distributed, but nonetheless redundant code that supports recovering most sensory information from smaller subpopulations.

Spatiotemporal transformations for gaze control

Sensorimotor transformations require spatiotemporal coordination of signals, that is, through both time and space. For example, the gaze control system employs signals that are time-locked to various sensorimotor events, but the spatial content of these signals is difficult to assess during ordinary gaze shifts. In this review, we describe the various models and methods that have been devised to test this question, and their limitations. We then describe a new method that can (a) simultaneously test between all of these models during natural, head-unrestrained conditions, and (b) track the evolving spatial continuum from target (T) to future gaze coding (G, including errors) through time. We then summarize some applications of this technique, comparing spatiotemporal coding in the primate frontal eye field (FEF) and superior colliculus (SC). The results confirm that these areas preferentially encode eye-centered, effector-independent parameters, and show-for the first time in ordinary gaze shifts-a spatial transformation between visual and motor responses from T to G coding. We introduce a new set of spatial models (T-G continuum) that revealed task-dependent timing of this transformation: progressive during a memory delay between vision and action, and almost immediate without such a delay. We synthesize the results from our studies and supplement it with previous knowledge of anatomy and physiology to propose a conceptual model where cumulative transformation noise is realized as inaccuracies in gaze behavior. We conclude that the spatiotemporal transformation for gaze is both local (observed within and across neurons in a given area) and distributed (with common signals shared across remote but interconnected structures).

Integration of Eye-Centered and Landmark-Centered Codes in Frontal Eye Field Gaze Responses

The visual system is thought to separate egocentric and allocentric representations, but behavioral experiments show that these codes are optimally integrated to influence goal-directed movements. To test if frontal cortex participates in this integration, we recorded primate frontal eye field activity during a cue-conflict memory delay saccade task. To dissociate egocentric and allocentric coordinates, we surreptitiously shifted a visual landmark during the delay period, causing saccades to deviate by 37% in the same direction. To assess the cellular mechanisms, we fit neural response fields against an egocentric (eye-centered target-to-gaze) continuum, and an allocentric shift (eye-to-landmark-centered) continuum. Initial visual responses best-fit target position. Motor responses (after the landmark shift) predicted future gaze position but embedded within the motor code was a 29% shift toward allocentric coordinates. This shift appeared transiently in memory-related visuomotor activity, and then reappeared in motor activity before saccades. Notably, fits along the egocentric and allocentric shift continua were initially independent, but became correlated across neurons just before the motor burst. Overall, these results implicate frontal cortex in the integration of egocentric and allocentric visual information for goal-directed action, and demonstrate the cell-specific, temporal progression of signal multiplexing for this process in the gaze system.

Distributed processing of movement signaling

Basic neurophysiological research with monkeys has shown how neurons in the motor cortex have firing rates tuned to movement direction. This original finding would have been difficult to uncover without the use of a behaving primate paradigm in which subjects grasped a handle and moved purposefully to targets in different directions. Subsequent research, again using behaving primate models, extended these findings to continuous drawing and to arm and hand movements encompassing action across multiple joints. This research also led to robust extraction algorithms in which information from neuronal populations is used to decode movement intent. The ability to decode intended movement provided the foundation for neural prosthetics in which brain-controlled interfaces are used by paralyzed human subjects to control computer cursors or high-performance motorized prosthetic arms and hands. This translation of neurophysiological laboratory findings to therapy is a clear example of why using nonhuman primates for basic research is valuable for advancing treatment of neurological disorders. Recent research emphasizes the distribution of intention signaling through neuronal populations and shows how many movement parameters are encoded simultaneously. In addition to direction and velocity, the arm's impedance has now been found to be encoded as well. The ability to decode motion and force from neural populations will make it possible to extend neural prosthetic paradigms to precise interaction with objects, enabling paralyzed individuals to perform many tasks of daily living.