Inference in the Brain: Statistics Flowing in Redundant Population Codes

Both k-core percolation and directed graph analysis revealed succession and transition of voxels' spatiotemporal progress on dynamic correlation resting-state fMRI

Introduction

Voxel hierarchy on dynamic brain graphs is produced by k-core percolation on functional dynamic amplitude correlation of resting-state fMRI.

Methods

Directed graphs and their afferent/efferent capacities are produced by Markov modeling of the universal cover of undirected graphs simultaneously with the calculation of volume entropy. Using these methods, state stationarity was tested for resting-state positive and unsigned negative brain graphs separately on sliding-window representation. The spatiotemporal progress of voxels was visualized and quantified.

Results and discussion

The voxel hierarchy of positive graphs revealed abrupt changes in coreness k (k-core) and maximum k-core (kmaxcore) voxels on animation maps representing state transitions interspersed among the succession. Afferent voxel capacities of the positive graphs revealed transient modules composed of dominant voxels and independent components as well as their exchanges compatible with transitions. Moreover, this voxel hierarchy and afferent capacity corroborated each other only on the positive directed functional connectivity graphs but not on the unsigned negative graphs. The Spatiotemporal progression of voxels on positive dynamic graphs constructed a hierarchy by k-core percolation and afferent information flow by volume entropy and directed graph methods. We disclosed the non-stationarity and its temporal progress pattern at rest, accompanied by diverse resting-state transitions on resting-state fMRI graphs in normal human subjects.

Age-related differences in network controllability are mitigated by redundancy in large-scale brain networks

The aging brain undergoes major changes in its topology. The mechanisms by which the brain mitigates age-associated changes in topology to maintain robust control of brain networks are unknown. Here we use diffusion MRI data from cognitively intact participants (n = 480, ages 40-90) to study age-associated differences in the average controllability of structural brain networks, topological features that could mitigate these differences, and the overall effect on cognitive function. We find age-associated declines in average controllability in control hubs and large-scale networks, particularly within the frontoparietal control and default mode networks. Further, we find that redundancy, a hypothesized mechanism of reserve, quantified via the assessment of multi-step paths within networks, mitigates the effects of topological differences on average network controllability. Lastly, we discover that average network controllability, redundancy, and grey matter volume, each uniquely contribute to predictive models of cognitive function. In sum, our results highlight the importance of redundancy for robust control of brain networks and in cognitive function in healthy-aging.

Orthogonal neural representations support perceptual judgements of natural stimuli

In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of objects and features despite the tremendous richness of natural visual scenes.

Decision-related activity and movement selection in primate visual cortex

Fluctuations in the activity of sensory neurons often predict perceptual decisions. This connection can be quantified with a metric called choice probability (CP), and there is a longstanding debate about whether CP reflects a causal influence on decisions or an echo of decision-making activity elsewhere in the brain. Here, we show that CP can reflect a third variable, namely, the movement used to indicate the decision. In a standard visual motion discrimination task, neurons in the middle temporal (MT) area of primate cortex responded more strongly during trials that involved a saccade toward their receptive fields. This variability accounted for much of the CP observed across the neuronal population, and it arose through training. Moreover, pharmacological inactivation of MT biased behavioral responses away from the corresponding visual field locations. These results demonstrate that training on a task with fixed sensorimotor contingencies introduces movement-related activity in sensory brain regions and that this plasticity can shape the neural circuitry of perceptual decision-making.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Population coding of strategic variables during foraging in freely moving macaques

Until now, it has been difficult to examine the neural bases of foraging in naturalistic environments because previous approaches have relied on restrained animals performing trial-based foraging tasks. Here we allowed unrestrained monkeys to freely interact with concurrent reward options while we wirelessly recorded population activity in the dorsolateral prefrontal cortex. The animals decided when and where to forage based on whether their prediction of reward was fulfilled or violated. This prediction was not solely based on a history of reward delivery, but also on the understanding that waiting longer improves the chance of reward. The task variables were continuously represented in a subspace of the high-dimensional population activity, and this compressed representation predicted the animal's subsequent choices better than the true task variables and as well as the raw neural activity. Our results indicate that monkeys' foraging strategies are based on a cortical model of reward dynamics as animals freely explore their environment.

Response sub-additivity and variability quenching in visual cortex

Sub-additivity and variability are ubiquitous response motifs in the primary visual cortex (V1). Response sub-additivity enables the construction of useful interpretations of the visual environment, whereas response variability indicates the factors that limit the precision with which the brain can do this. There is increasing evidence that experimental manipulations that elicit response sub-additivity often also quench response variability. Here, we provide an overview of these phenomena and suggest that they may have common origins. We discuss empirical findings and recent model-based insights into the functional operations, computational objectives and circuit mechanisms underlying V1 activity. These different modelling approaches all predict that response sub-additivity and variability quenching often co-occur. The phenomenology of these two response motifs, as well as many of the insights obtained about them in V1, generalize to other cortical areas. Thus, the connection between response sub-additivity and variability quenching may be a canonical motif across the cortex.

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Although it may appear infeasible and impractical, building artificial intelligence (AI) using a bottom-up approach based on the understanding of neuroscience is straightforward. The lack of a generalized governing principle for biological neural networks (BNNs) forces us to address this problem by converting piecemeal information on the diverse features of neurons, synapses, and neural circuits into AI. In this review, we described recent attempts to build a biologically plausible neural network by following neuroscientifically similar strategies of neural network optimization or by implanting the outcome of the optimization, such as the properties of single computational units and the characteristics of the network architecture. In addition, we proposed a formalism of the relationship between the set of objectives that neural networks attempt to achieve, and neural network classes categorized by how closely their architectural features resemble those of BNN. This formalism is expected to define the potential roles of top-down and bottom-up approaches for building a biologically plausible neural network and offer a map helping the navigation of the gap between neuroscience and AI engineering.

Accumulation of network redundancy marks the early stage of Alzheimer's disease

Brain wiring redundancy counteracts aging-related cognitive decline by reserving additional communication channels as a neuroprotective mechanism. Such a mechanism plays a potentially important role in maintaining cognitive function during the early stages of neurodegenerative disorders such as Alzheimer's disease (AD). AD is characterized by severe cognitive decline and involves a long prodromal stage of mild cognitive impairment (MCI). Since MCI subjects are at high risk of converting to AD, identifying MCI individuals is essential for early intervention. To delineate the redundancy profile during AD progression and enable better MCI diagnosis, we define a metric that reflects redundant disjoint connections between brain regions and extract redundancy features in three high-order brain networks-medial frontal, frontoparietal, and default mode networks-based on dynamic functional connectivity (dFC) captured by resting-state functional magnetic resonance imaging (rs-fMRI). We show that redundancy increases significantly from normal control (NC) to MCI individuals and decreases slightly from MCI to AD individuals. We further demonstrate that statistical features of redundancy are highly discriminative and yield state-of-the-art accuracy of up to 96.8 ± 1.0% in support vector machine (SVM) classification between NC and MCI individuals. This study provides evidence supporting the notion that redundancy serves as a crucial neuroprotective mechanism in MCI.

Post-injury pain and behaviour: a control theory perspective

Injuries of various types occur commonly in the lives of humans and other animals and lead to a pattern of persistent pain and recuperative behaviour that allows safe and effective recovery. In this Perspective, we propose a control-theoretic framework to explain the adaptive processes in the brain that drive physiological post-injury behaviour. We set out an evolutionary and ethological view on how animals respond to injury, illustrating how the behavioural state associated with persistent pain and recuperation may be just as important as phasic pain in ensuring survival. Adopting a normative approach, we suggest that the brain implements a continuous optimal inference of the current state of injury from diverse sensory and physiological signals. This drives the various effector control mechanisms of behavioural homeostasis, which span the modulation of ongoing motivation and perception to drive rest and hyper-protective behaviours. However, an inherent problem with this is that these protective behaviours may partially obscure information about whether injury has resolved. Such information restriction may seed a tendency to aberrantly or persistently infer injury, and may thus promote the transition to pathological chronic pain states.

Evolutionary dynamics of mushroom body Kenyon cell types in hymenopteran brains from multifunctional type to functionally specialized types

Evolutionary dynamics of diversification of brain neuronal cell types that have underlain behavioral evolution remain largely unknown. Here, we compared transcriptomes and functions of Kenyon cell (KC) types that compose the mushroom bodies between the honey bee and sawfly, a primitive hymenopteran insect whose KCs likely have the ancestral properties. Transcriptome analyses show that the sawfly KC type shares some of the gene expression profile with each honey bee KC type, although unique gene expression profiles have also been acquired in each honey bee KC type. In addition, functional analysis of two sawfly genes suggested that the functions in learning and memory of the ancestral KC type were heterogeneously inherited among the KC types in the honey bee. Our findings strongly suggest that the functional evolution of KCs in Hymenoptera involved two previously hypothesized processes for evolution of cell function: functional segregation and divergence.

Age-related changes in network controllability are mitigated by redundancy in large-scale brain networks

The aging brain undergoes major changes in its topology. The mechanisms by which the brain mitigates age-associated changes in topology to maintain robust control of brain networks are unknown. Here we used diffusion MRI data from cognitively intact participants (n=480, ages 40-90) to study age-associated changes in the controllability of structural brain networks, features that could mitigate these changes, and the overall effect on cognitive function. We found age-associated declines in controllability in control hubs and large-scale networks, particularly within the and frontoparietal control and default mode networks. Redundancy, quantified via the assessment of multi-step paths within networks, mitigated the effects of changes in topology on network controllability. Lastly, network controllability, redundancy, and grey matter volume each played important complementary roles in cognitive function. In sum, our results highlight the importance of redundancy for robust control of brain networks and in cognitive function in healthy-aging.

The molecular memory code and synaptic plasticity: A synthesis

The most widely accepted view of memory in the brain holds that synapses are the storage sites of memory, and that memories are formed through associative modification of synapses. This view has been challenged on conceptual and empirical grounds. As an alternative, it has been proposed that molecules within the cell body are the storage sites of memory, and that memories are formed through biochemical operations on these molecules. This paper proposes a synthesis of these two views, grounded in a computational model of memory. Synapses are conceived as storage sites for the parameters of an approximate posterior probability distribution over latent causes. Intracellular molecules are conceived as storage sites for the parameters of a generative model. The model stipulates how these two components work together as part of an integrated algorithm for learning and inference.

Coding of latent variables in sensory, parietal, and frontal cortices during closed-loop virtual navigation

We do not understand how neural nodes operate and coordinate within the recurrent action-perception loops that characterize naturalistic self-environment interactions. Here, we record single-unit spiking activity and local field potentials (LFPs) simultaneously from the dorsomedial superior temporal area (MSTd), parietal area 7a, and dorsolateral prefrontal cortex (dlPFC) as monkeys navigate in virtual reality to 'catch fireflies'. This task requires animals to actively sample from a closed-loop virtual environment while concurrently computing continuous latent variables: (i) the distance and angle travelled (i.e., path integration) and (ii) the distance and angle to a memorized firefly location (i.e., a hidden spatial goal). We observed a patterned mixed selectivity, with the prefrontal cortex most prominently coding for latent variables, parietal cortex coding for sensorimotor variables, and MSTd most often coding for eye movements. However, even the traditionally considered sensory area (i.e., MSTd) tracked latent variables, demonstrating path integration and vector coding of hidden spatial goals. Further, global encoding profiles and unit-to-unit coupling (i.e., noise correlations) suggested a functional subnetwork composed by MSTd and dlPFC, and not between these and 7a, as anatomy would suggest. We show that the greater the unit-to-unit coupling between MSTd and dlPFC, the more the animals' gaze position was indicative of the ongoing location of the hidden spatial goal. We suggest this MSTd-dlPFC subnetwork reflects the monkeys' natural and adaptive task strategy wherein they continuously gaze toward the location of the (invisible) target. Together, these results highlight the distributed nature of neural coding during closed action-perception loops and suggest that fine-grain functional subnetworks may be dynamically established to subserve (embodied) task strategies.

Towards a cross-level understanding of Bayesian inference in the brain

Perception emerges from unconscious probabilistic inference, which guides behaviour in our ubiquitously uncertain environment. Bayesian decision theory is a prominent computational model that describes how people make rational decisions using noisy and ambiguous sensory observations. However, critical questions have been raised about the validity of the Bayesian framework in explaining the mental process of inference. Firstly, some natural behaviours deviate from Bayesian optimum. Secondly, the neural mechanisms that support Bayesian computations in the brain are yet to be understood. Taking Marr's cross level approach, we review the recent progress made in addressing these challenges. We first review studies that combined behavioural paradigms and modelling approaches to explain both optimal and suboptimal behaviours. Next, we evaluate the theoretical advances and the current evidence for ecologically feasible algorithms and neural implementations in the brain, which may enable probabilistic inference. We argue that this cross-level approach is necessary for the worthwhile pursuit to uncover mechanistic accounts of human behaviour.

Robust Coding of Eye Position in Posterior Parietal Cortex despite Context-Dependent Tuning

Neurons in posterior parietal cortex (PPC) encode many aspects of the sensory world (e.g., scene structure), the posture of the body, and plans for action. For a downstream computation, however, only some of these dimensions are relevant; the rest are "nuisance variables" because their influence on neural activity changes with sensory and behavioral context, potentially corrupting the read-out of relevant information. Here we show that a key postural variable for vision (eye position) is represented robustly in male macaque PPC across a range of contexts, although the tuning of single neurons depended strongly on context. Contexts were defined by different stages of a visually guided reaching task, including (1) a visually sparse epoch, (2) a visually rich epoch, (3) a "go" epoch in which the reach was cued, and (4) during the reach itself. Eye position was constant within trials but varied across trials in a 3 × 3 grid spanning 24° × 24°. Using demixed principal component analysis of neural spike-counts, we found that the subspace of the population response encoding eye position is orthogonal to that encoding task context. Accordingly, a context-naive (fixed-parameter) decoder was nevertheless able to estimate eye position reliably across contexts. Errors were small given the sample size (∼1.78°) and would likely be even smaller with larger populations. Moreover, they were comparable to that of decoders that were optimized for each context. Our results suggest that population codes in PPC shield encoded signals from crosstalk to support robust sensorimotor transformations across contexts.SIGNIFICANCE STATEMENT Neurons in posterior parietal cortex (PPC) which are sensitive to gaze direction are thought to play a key role in spatial perception and behavior (e.g., reaching, navigation), and provide a potential substrate for brain-controlled prosthetics. Many, however, change their tuning under different sensory and behavioral contexts, raising the prospect that they provide unreliable representations of egocentric space. Here, we analyze the structure of encoding dimensions for gaze direction and context in PPC during different stages of a visually guided reaching task. We use demixed dimensionality reduction and decoding techniques to show that the coding of gaze direction in PPC is mostly invariant to context. This suggests that PPC can provide reliable spatial information across sensory and behavioral contexts.

Attention improves information flow between neuronal populations without changing the communication subspace

Visual attention allows observers to change the influence of different parts of a visual scene on their behavior, suggesting that information can be flexibly shared between visual cortex and neurons involved in decision making. We investigated the neural substrate of flexible information routing by analyzing the activity of populations of visual neurons in the medial temporal area (MT) and oculo-motor neurons in the superior colliculus (SC) while rhesus monkeys switched spatial attention. We demonstrated that attention increases the efficacy of visuomotor communication: trial-to-trial variability in SC population activity could be better predicted by the activity of the MT population (and vice versa) when attention was directed toward their joint receptive fields. Surprisingly, this improvement in prediction was not explained by changes in the dimensionality of the shared subspace or in the magnitude of local or shared pairwise noise correlations. These results lay a foundation for future theoretical and experimental studies into how visual attention can flexibly change information flow between sensory and decision neurons.

Revealing nonlinear neural decoding by analyzing choices

Sensory data about most natural task-relevant variables are entangled with task-irrelevant nuisance variables. The neurons that encode these relevant signals typically constitute a nonlinear population code. Here we present a theoretical framework for quantifying how the brain uses or decodes its nonlinear information. Our theory obeys fundamental mathematical limitations on information content inherited from the sensory periphery, describing redundant codes when there are many more cortical neurons than primary sensory neurons. The theory predicts that if the brain uses its nonlinear population codes optimally, then more informative patterns should be more correlated with choices. More specifically, the theory predicts a simple, easily computed quantitative relationship between fluctuating neural activity and behavioral choices that reveals the decoding efficiency. This relationship holds for optimal feedforward networks of modest complexity, when experiments are performed under natural nuisance variation. We analyze recordings from primary visual cortex of monkeys discriminating the distribution from which oriented stimuli were drawn, and find these data are consistent with the hypothesis of near-optimal nonlinear decoding.

The association between hippocampal volume and memory in pathological aging is mediated by functional redundancy

Hippocampal neurodegeneration, a primary component of Alzheimer's disease pathology, relates to poor cognition; however, the mechanisms underlying this relationship are not well understood. Using a sample of cognitively normal older adults and individuals with mild cognitive impairment, this study aims to determine the topological properties of functional networks accompanying hippocampal atrophy in aging, along with their association to cognition and clinical progression. We considered two conceptually differing topological properties: redundancy (the existence of alternative channels of functional commutation) and local efficiency (the efficiency of local information exchange). Hippocampal redundancy, but not local efficiency, mediated the association between low hippocampal volume and low memory in both the whole sample and in ß-amyloid positive participants. Additionally, participants with high hippocampal volume, redundancy, and memory clustered separately from those with low values on all three measures, with the latter group showing higher conversion rates to dementia within three years. Together, these results demonstrate that reduced hippocampal redundancy is one mechanism through which hippocampal atrophy associates with memory impairment in healthy and pathological aging.

Modelling the neural code in large populations of correlated neurons

Neurons respond selectively to stimuli, and thereby define a code that associates stimuli with population response patterns. Certain correlations within population responses (noise correlations) significantly impact the information content of the code, especially in large populations. Understanding the neural code thus necessitates response models that quantify the coding properties of modelled populations, while fitting large-scale neural recordings and capturing noise correlations. In this paper, we propose a class of response model based on mixture models and exponential families. We show how to fit our models with expectation-maximization, and that they capture diverse variability and covariability in recordings of macaque primary visual cortex. We also show how they facilitate accurate Bayesian decoding, provide a closed-form expression for the Fisher information, and are compatible with theories of probabilistic population coding. Our framework could allow researchers to quantitatively validate the predictions of neural coding theories against both large-scale neural recordings and cognitive performance.

Supporting generalization in non-human primate behavior by tapping into structural knowledge: Examples from sensorimotor mappings, inference, and decision-making

The complex behaviors we ultimately wish to understand are far from those currently used in systems neuroscience laboratories. A salient difference are the closed loops between action and perception prominently present in natural but not laboratory behaviors. The framework of reinforcement learning and control naturally wades across action and perception, and thus is poised to inform the neurosciences of tomorrow, not only from a data analyses and modeling framework, but also in guiding experimental design. We argue that this theoretical framework emphasizes active sensing, dynamical planning, and the leveraging of structural regularities as key operations for intelligent behavior within uncertain, time-varying environments. Similarly, we argue that we may study natural task strategies and their neural circuits without over-training animals when the tasks we use tap into our animal's structural knowledge. As proof-of-principle, we teach animals to navigate through a virtual environment - i.e., explore a well-defined and repetitive structure governed by the laws of physics - using a joystick. Once these animals have learned to 'drive', without further training they naturally (i) show zero- or one-shot learning of novel sensorimotor contingencies, (ii) infer the evolving path of dynamically changing latent variables, and (iii) make decisions consistent with maximizing reward rate. Such task designs allow for the study of flexible and generalizable, yet controlled, behaviors. In turn, they allow for the exploitation of pillars of intelligence - flexibility, prediction, and generalization -, properties whose neural underpinning have remained elusive.

Scintillating Starbursts: Concentric Star Polygons Induce Illusory Ray Patterns

Here, we introduce and explore Scintillating Starbursts, a stimulus type made up of concentric star polygons that induce illusory scintillating rays or beams. We test experimentally which factors, such as contrast and number of vertices, modulate how observers experience this stimulus class. We explain how the illusion arises from the interplay of known visual processes, specifically central versus peripheral vision, and interpret the phenomenology evoked by these patterns. We discuss how Starbursts differ from similar and related visual illusions such as illusory contours, grid illusions such as the pincushion grid illusion as well as moiré patterns.

Lower functional hippocampal redundancy in mild cognitive impairment

With an increasing prevalence of mild cognitive impairment (MCI) and Alzheimer's disease (AD) in response to an aging population, it is critical to identify and understand neuroprotective mechanisms against cognitive decline. One potential mechanism is redundancy: the existence of duplicate elements within a system that provide alternative functionality in case of failure. As the hippocampus is one of the earliest sites affected by AD pathology, we hypothesized that functional hippocampal redundancy is protective against cognitive decline. We compared hippocampal functional redundancy derived from resting-state functional MRI networks in cognitively normal older adults, with individuals with early and late MCI, as well as the relationship between redundancy and cognition. Posterior hippocampal redundancy was reduced between cognitively normal and MCI groups, plateauing across early and late MCI. Higher hippocampal redundancy was related to better memory performance only for cognitively normal individuals. Critically, functional hippocampal redundancy did not come at the expense of network efficiency. Our results provide support that hippocampal redundancy protects against cognitive decline in aging.

Paradigm Shift Toward Digital Neuropsychology and High-Dimensional Neuropsychological Assessments: Review

Neuropsychologists in the digital age have increasing access to emerging technologies. The National Institutes of Health (NIH) initiatives for behavioral and social sciences have emphasized these developing scientific and technological potentials (eg, novel sensors) for augmented characterization of neurocognitive, behavioral, affective, and social processes. Perhaps these innovative technologies will lead to a paradigm shift from disintegrated and data-poor behavioral science to cohesive and data-rich science that permits improved translation from bench to bedside. The 4 main advances influencing the scientific priorities of a recent NIH Office of Behavioral and Social Sciences Research strategic plan include the following: integration of neuroscience into behavioral and social sciences, transformational advances in measurement science, digital intervention platforms, and large-scale population cohorts and data integration. This paper reviews these opportunities for novel brain-behavior characterizations. Emphasis is placed on the increasing concern of neuropsychology with these topics and the need for development in these areas to maintain relevance as a scientific discipline and advance scientific developments. Furthermore, the effects of such advancements necessitate discussion and modification of training as well as ethical and legal mandates for neuropsychological research and praxes.

Enhanced representation of natural sound sequences in the ventral auditory midbrain

The auditory midbrain (inferior colliculus, IC) plays an important role in sound processing, acting as hub for acoustic information extraction and for the implementation of fast audio-motor behaviors. IC neurons are topographically organized according to their sound frequency preference: dorsal IC regions encode low frequencies while ventral areas respond best to high frequencies, a type of sensory map defined as tonotopy. Tonotopic maps have been studied extensively using artificial stimuli (pure tones) but our knowledge of how these maps represent information about sequences of natural, spectro-temporally rich sounds is sparse. We studied this question by conducting simultaneous extracellular recordings across IC depths in awake bats (Carollia perspicillata) that listened to sequences of natural communication and echolocation sounds. The hypothesis was that information about these two types of sound streams is represented at different IC depths since they exhibit large differences in spectral composition, i.e., echolocation covers the high-frequency portion of the bat soundscape (> 45 kHz), while communication sounds are broadband and carry most power at low frequencies (20–25 kHz). Our results showed that mutual information between neuronal responses and acoustic stimuli, as well as response redundancy in pairs of neurons recorded simultaneously, increase exponentially with IC depth. The latter occurs regardless of the sound type presented to the bats (echolocation or communication). Taken together, our results indicate the existence of mutual information and redundancy maps at the midbrain level whose response cannot be predicted based on the frequency composition of natural sounds and classic neuronal tuning curves.

Rational thoughts in neural codes

Complex behaviors are often driven by an internal model, which integrates sensory information over time and facilitates long-term planning to reach subjective goals. A fundamental challenge in neuroscience is, How can we use behavior and neural activity to understand this internal model and its dynamic latent variables? Here we interpret behavioral data by assuming an agent behaves rationally—that is, it takes actions that optimize its subjective reward according to its understanding of the task and its relevant causal variables. We apply a method, inverse rational control (IRC), to learn an agent’s internal model and reward function by maximizing the likelihood of its measured sensory observations and actions. This thereby extracts rational and interpretable thoughts of the agent from its behavior. We also provide a framework for interpreting encoding, recoding, and decoding of neural data in light of this rational model for behavior. When applied to behavioral and neural data from simulated agents performing suboptimally on a naturalistic foraging task, this method successfully recovers their internal model and reward function, as well as the Markovian computational dynamics within the neural manifold that represent the task. This work lays a foundation for discovering how the brain represents and computes with dynamic latent variables.

Learning Credit Assignment

Deep learning has achieved impressive prediction accuracies in a variety of scientific and industrial domains. However, the nested nonlinear feature of deep learning makes the learning highly nontransparent, i.e., it is still unknown how the learning coordinates a huge number of parameters to achieve decision-making. To explain this hierarchical credit assignment, we propose a mean-field learning model by assuming that an ensemble of subnetworks, rather than a single network, is trained for a classification task. Surprisingly, our model reveals that apart from some deterministic synaptic weights connecting two neurons at neighboring layers, there exists a large number of connections that can be absent, and other connections can allow for a broad distribution of their weight values. Therefore, synaptic connections can be classified into three categories: very important ones, unimportant ones, and those of variability that may partially encode nuisance factors. Therefore, our model learns the credit assignment leading to the decision and predicts an ensemble of sub-networks that can accomplish the same task, thereby providing insights toward understanding the macroscopic behavior of deep learning through the lens of distinct roles of synaptic weights.

Tracking the Mind's Eye: Primate Gaze Behavior during Virtual Visuomotor Navigation Reflects Belief Dynamics

To take the best actions, we often need to maintain and update beliefs about variables that cannot be directly observed. To understand the principles underlying such belief updates, we need tools to uncover subjects' belief dynamics from natural behavior. We tested whether eye movements could be used to infer subjects' beliefs about latent variables using a naturalistic navigation task. Humans and monkeys navigated to a remembered goal location in a virtual environment that provided optic flow but lacked explicit position cues. We observed eye movements that appeared to continuously track the goal location even when no visible target was present there. Accurate goal tracking was associated with improved task performance, and inhibiting eye movements in humans impaired navigation precision. These results suggest that gaze dynamics play a key role in action selection during challenging visuomotor behaviors and may possibly serve as a window into the subject's dynamically evolving internal beliefs.

Multisensory neural processing: from cue integration to causal inference

Neurophysiological studies of multisensory processing have largely focused on how the brain integrates information from different sensory modalities to form a coherent percept. However, in the natural environment, an important extra step is needed: the brain faces the problem of causal inference, which involves determining whether different sources of sensory information arise from the same environmental cause, such that integrating them is advantageous Behavioral and computational studies have provided a strong foundation for studying causal inference, but studies of its neural basis have only recently been undertaken. This review focuses on recent advances regarding how the brain infers the causes of sensory inputs and uses this information to make robust perceptual estimates.

Task-Related Sensorimotor Adjustments Increase the Sensory Range in Electrolocation

Perception and motor control traditionally are studied separately. However, motor activity can serve as a scaffold to shape the sensory flow. This tight link between motor actions and sensing is particularly evident in active sensory systems. Here, we investigate how the weakly electric mormyrid fish Gnathonemus petersii of undetermined sex structure their sensing and motor behavior while learning a perceptual task. We find systematic adjustments of the motor behavior that correlate with an increased performance. Using a model to compute the electrosensory input, we show that these behavioral adjustments improve the sensory input. As we find low neuronal detection thresholds at the level of medullary electrosensory neurons, it seems that the behavior-driven improvements of the sensory input are highly suitable to overcome the sensory limitations, thereby increasing the sensory range. Our results show that motor control is an active component of sensory learning, demonstrating that a detailed understanding of contribution of motor actions to sensing is needed to understand even seemingly simple behaviors.SIGNIFICANCE STATEMENT Motor-guided sensation and perception are intertwined, with motor behavior serving as a scaffold to shape the sensory input. We characterized how the weakly electric mormyrid fish Gnathonemus petersii, as it learns a perceptual task, restructures its sensorimotor behavior. We find that systematic adjustments of the motor behavior correlate with increased performance and a shift of the sensory attention of the animal. Analyzing the afferent electrosensory input shows that a significant gain in information results from these sensorimotor adjustments. Our results show that motor control can be an active component of sensory learning. Researching the sensory corollaries of motor control thus can be crucial to understand sensory sensation and perception under naturalistic conditions.

Human confidence judgments reflect reliability-based hierarchical integration of contextual information

Our immediate observations must be supplemented with contextual information to resolve ambiguities. However, the context is often ambiguous too, and thus it should be inferred itself to guide behavior. Here, we introduce a novel hierarchical task (airplane task) in which participants should infer a higher-level, contextual variable to inform probabilistic inference about a hidden dependent variable at a lower level. By controlling the reliability of past sensory evidence through varying the sample size of the observations, we find that humans estimate the reliability of the context and combine it with current sensory uncertainty to inform their confidence reports. Behavior closely follows inference by probabilistic message passing between latent variables across hierarchical state representations. Commonly reported inferential fallacies, such as sample size insensitivity, are not present, and neither did participants appear to rely on simple heuristics. Our results reveal uncertainty-sensitive integration of information at different hierarchical levels and temporal scales.

Because our immediate observations are often ambiguous, we must use the context (prior beliefs) to guide inference, but the context may also be uncertain. Here, the authors show that humans can accurately estimate the reliability of the context and combine it with sensory uncertainty to form their decisions and estimate confidence.

Models of vestibular semicircular canal afferent neuron firing activity

Semicircular canal afferent neurons transmit information about head rotation to the brain. Mathematical models of how they do this have coevolved with concepts of how brains perceive the world. A 19th-century "camera" metaphor, in which sensory neurons project an image of the world captured by sense organs into the brain, gave way to a 20th-century view of sensory nerves as communication channels providing inputs to dynamical control systems. Now, in the 21st century, brains are being modeled as Bayesian observers who infer what is happening in the world given noisy, incomplete, and distorted sense data. The semicircular canals of the vestibular apparatus provide an experimentally accessible, low-dimensional system for developing and testing dynamical Bayesian generative models of sense data. In this review, we summarize advances in mathematical modeling of information transmission by semicircular canal afferent sensory neurons since the first such model was proposed nearly a century ago. Models of information transmission by vestibular afferent neurons may provide a foundation for developing realistic models of how brains perceive the world by inferring the causes of sense data.

Choice (-history) correlations in sensory cortex: cause or consequence?

One challenge in neuroscience, as in other areas of science, is to make inferences about the underlying causal structure from correlational data. Here, we discuss this challenge in the context of choice correlations in sensory neurons, that is, trial-by-trial correlations, unexplained by the stimulus, between the activity of sensory neurons and an animal's perceptual choice. Do these choice-correlations reflect feedforward, feedback signalling, both, or neither? We highlight recent results of correlational and causal examinations of choice and choice-history signals in sensory, and in part sensorimotor, cortex and address formal statistical frameworks to infer causal interactions from data.

Unsupervised identification of states from voltage recordings of neural networks

BACKGROUND

Modern techniques for multi-neuronal recording produce large amounts of data. There is no automatic procedure for the identification of states in recurrent voltage patterns.

NEW METHOD

We propose NetSAP (Network States And Pathways), a data-driven analysis method that is able to recognize multi-neuron voltage patterns (states). To capture the subtle differences between snapshots in voltage recordings, NetSAP infers the underlying functional neural network in a time-resolved manner with a sliding window approach. Then NetSAP identifies states from a reordering of the time series of inferred networks according to a user-defined metric. The procedure for unsupervised identification of states was developed originally for the analysis of molecular dynamics simulations of proteins.

RESULTS

We tested NetSAP on neural network simulations of GABAergic inhibitory interneurons. Most simulation parameters are chosen to reproduce literature observations, and we keep noise terms as control parameters to regulate the coherence of the simulated signals. NetSAP is able to identify multiple states even in the case of high internal noise and low signal coherence. We provide evidence that NetSAP is robust for networks with up to about 50% of the neurons spiking randomly. NetSAP is scalable and its code is open source.

COMPARISON WITH EXISTING METHODS

NetSAP outperforms common analysis techniques, such as PCA and k-means clustering, on a simulated recording of voltage traces of 50 neurons.

CONCLUSIONS

NetSAP analysis is an efficient tool to identify voltage patterns from neuronal recordings.

National Institutes of Health initiatives for advancing scientific developments in clinical neuropsychology

OBJECTIVE

The current review briefly addresses the history of neuropsychology as a context for discussion of developmental milestones that have advanced the profession, as well as areas where the progression has lagged. More recently in the digital/information age, utilization and incorporation of emerging technologies has been minimal, which has stagnated ongoing evolution of the practice of neuropsychology despite technology changing many aspects of daily living. These authors advocate for embracing National Institutes of Health (NIH) initiatives, or interchangeably referred to as transformative opportunities, for the behavioral and social sciences. These initiatives address the need for neuropsychologists to transition from fragmented and data-poor approaches to integrated and data-rich scientific approaches that ultimately improve translational applications. Specific to neuropsychology is the need for the adoption of novel means of brain-behavior characterizations.

METHOD

Narrative review Conclusions: Clinical neuropsychology has reached a developmental plateau where it is ready to embrace the measurement science and technological advances which have been readily adopted by the human neurosciences. While there are ways in which neuropsychology is making inroads into these areas, a great deal of growth is needed to maintain relevance as a scientific discipline (see Figures 1, 2, and 3) consistent with NIH initiatives to advance scientific developments. Moreover, implications of such progress require discussion and modification of training, ethical, and legal mandates of the practice of neuropsychology.

The Spatial Structure of Neural Encoding in Mouse Posterior Cortex during Navigation

Navigation engages many cortical areas, including visual, parietal, and retrosplenial cortices. These regions have been mapped anatomically and with sensory stimuli and studied individually during behavior. Here, we investigated how behaviorally driven neural activity is distributed and combined across these regions. We performed dense sampling of single-neuron activity across the mouse posterior cortex and developed unbiased methods to relate neural activity to behavior and anatomical space. Most parts of the posterior cortex encoded most behavior-related features. However, the relative strength with which features were encoded varied across space. Therefore, the posterior cortex could be divided into discriminable areas based solely on behaviorally relevant neural activity, revealing functional structure in association regions. Multimodal representations combining sensory and movement signals were strongest in posterior parietal cortex, where gradients of single-feature representations spatially overlapped. We propose that encoding of behavioral features is not constrained by retinotopic borders and instead varies smoothly over space within association regions.

Advancing System Performance with Redundancy: From Biological to Artificial Designs

Redundancy is a fundamental characteristic of many biological processes such as those in the genetic, visual, muscular, and nervous systems, yet its driven mechanism has not been fully comprehended. Until recently, the only understanding of redundancy is as a mean to attain fault tolerance, which is reflected in the design of many man-made systems. On the contrary, our previous work on redundant sensing (RS) has demonstrated an example where redundancy can be engineered solely for enhancing accuracy and precision. The design was inspired by the binocular structure of human vision, which we believe may share a similar operation. In this letter, we present a unified theory describing how such utilization of redundancy is feasible through two complementary mechanisms: representational redundancy (RPR) and entangled redundancy (ETR). We also point out two additional examples where our new understanding of redundancy can be applied to justify a system's superior performance. One is the human musculoskeletal system (HMS), a biological instance, and the other is the deep residual neural network (ResNet), an artificial counterpart. We envision that our theory would provide a framework for the future development of bio-inspired redundant artificial systems, as well as assist studies of the fundamental mechanisms governing various biological processes.

Understanding Sensory Information Processing Through Simultaneous Multi-area Population Recordings

The goal of sensory neuroscience is to understand how the brain creates its myriad of representations of the world, and uses these representations to produce perception and behavior. Circuits of neurons in spatially segregated regions of brain tissue have distinct functional specializations, and these regions are connected to form a functional processing hierarchy. Advances in technology for recording neuronal activity from multiple sites in multiple cortical areas mean that we are now able to collect data that reflects how information is transformed within and between connected members of this hierarchy. This advance is an important step in understanding the brain because, after the sensory organs have transduced a physical signal, every processing stage takes the activity of other neurons as its input, not measurements of the physical world. However, as we explore the potential of studying how populations of neurons in multiple areas respond in concert, we must also expand both the analytical tools that we use to make sense of these data and the scope of the theories that we attempt to define. In this article, we present an overview of some of the most promising analytical approaches for making inferences from population recordings in multiple brain areas, such as dimensionality reduction and measuring changes in correlated variability, and examine how they may be used to address longstanding questions in sensory neuroscience.

Cognition as a Window into Neuronal Population Space

Understanding how cognitive processes affect the responses of sensory neurons may clarify the relationship between neuronal population activity and behavior. However, tools for analyzing neuronal activity have not kept up with technological advances in recording from large neuronal populations. Here, we describe prevalent hypotheses of how cognitive processes affect sensory neurons, driven largely by a model based on the activity of single neurons or pools of neurons as the units of computation. We then use simple simulations to expand this model to a new conceptual framework that focuses on subspaces of population activity as the relevant units of computation, uses comparisons between brain areas or to behavior to guide analyses of these subspaces, and suggests that population activity is optimized to decode the large variety of stimuli and tasks that animals encounter in natural behavior. This framework provides new ways of understanding the ever-growing quantity of recorded population activity data.

Descending pathways generate perception of and neural responses to weak sensory input

Natural sensory stimuli frequently consist of a fast time-varying waveform whose amplitude or contrast varies more slowly. While changes in contrast carry behaviorally relevant information necessary for sensory perception, their processing by the brain remains poorly understood to this day. Here, we investigated the mechanisms that enable neural responses to and perception of low-contrast stimuli in the electrosensory system of the weakly electric fish Apteronotus leptorhynchus. We found that fish reliably detected such stimuli via robust behavioral responses. Recordings from peripheral electrosensory neurons revealed stimulus-induced changes in firing activity (i.e., phase locking) but not in their overall firing rate. However, central electrosensory neurons receiving input from the periphery responded robustly via both phase locking and increases in firing rate. Pharmacological inactivation of feedback input onto central electrosensory neurons eliminated increases in firing rate but did not affect phase locking for central electrosensory neurons in response to low-contrast stimuli. As feedback inactivation eliminated behavioral responses to these stimuli as well, our results show that it is changes in central electrosensory neuron firing rate that are relevant for behavior, rather than phase locking. Finally, recordings from neurons projecting directly via feedback to central electrosensory neurons revealed that they provide the necessary input to cause increases in firing rate. Our results thus provide the first experimental evidence that feedback generates both neural and behavioral responses to low-contrast stimuli that are commonly found in the natural environment.

Feedback input from more central to more peripheral brain areas is found ubiquitously in the central nervous system of vertebrates. In this study, we used a combination of electrophysiological, behavioral, and pharmacological approaches to reveal a novel function for feedback pathways in generating neural and behavioral responses to weak sensory input in the weakly electric fish. We first determined that weak sensory input gives rise to responses that are phase locked in both peripheral sensory neurons and in the central neurons that are their downstream targets. However, central neurons also responded to weak sensory inputs that were not relayed via a feedforward input from the periphery, because complete inactivation of the feedback pathway abolished increases in firing rate but not the phase locking in response to weak sensory input. Because such inactivation also abolished the behavioral responses, our results show that the increases in firing rate in central neurons, and not the phase locking, are decoded downstream to give rise to perception. Finally, we discovered that the neurons providing feedback input were also activated by weak sensory input, thereby offering further evidence that feedback is necessary to elicit increases in firing rate that are needed for perception.

A Dynamic Bayesian Observer Model Reveals Origins of Bias in Visual Path Integration

Path integration is a strategy by which animals track their position by integrating their self-motion velocity. To identify the computational origins of bias in visual path integration, we asked human subjects to navigate in a virtual environment using optic flow and found that they generally traveled beyond the goal location. Such a behavior could stem from leaky integration of unbiased self-motion velocity estimates or from a prior expectation favoring slower speeds that causes velocity underestimation. Testing both alternatives using a probabilistic framework that maximizes expected reward, we found that subjects' biases were better explained by a slow-speed prior than imperfect integration. When subjects integrate paths over long periods, this framework intriguingly predicts a distance-dependent bias reversal due to buildup of uncertainty, which we also confirmed experimentally. These results suggest that visual path integration in noisy environments is limited largely by biases in processing optic flow rather than by leaky integration.

Post-decision biases reveal a self-consistency principle in perceptual inference

Making a categorical judgment can systematically bias our subsequent perception of the world. We show that these biases are well explained by a self-consistent Bayesian observer whose perceptual inference process is causally conditioned on the preceding choice. We quantitatively validated the model and its key assumptions with a targeted set of three psychophysical experiments, focusing on a task sequence where subjects first had to make a categorical orientation judgment before estimating the actual orientation of a visual stimulus. Subjects exhibited a high degree of consistency between categorical judgment and estimate, which is difficult to reconcile with alternative models in the face of late, memory related noise. The observed bias patterns resemble the well-known changes in subjective preferences associated with cognitive dissonance, which suggests that the brain's inference processes may be governed by a universal self-consistency constraint that avoids entertaining 'dissonant' interpretations of the evidence.