Uncovering spatial topology represented by rat hippocampal population neuronal codes

Efficient, continual, and generalized learning in the brain - neural mechanism of Mental Schema 2.0

There has been tremendous progress in artificial neural networks (ANNs) over the past decade; however, the gap between ANNs and the biological brain as a learning device remains large. With the goal of closing this gap, this paper reviews learning mechanisms in the brain by focusing on three important issues in ANN research: efficiency, continuity, and generalization. We first discuss the method by which the brain utilizes a variety of self-organizing mechanisms to maximize learning efficiency, with a focus on the role of spontaneous activity of the brain in shaping synaptic connections to facilitate spatiotemporal learning and numerical processing. Then, we examined the neuronal mechanisms that enable lifelong continual learning, with a focus on memory replay during sleep and its implementation in brain-inspired ANNs. Finally, we explored the method by which the brain generalizes learned knowledge in new situations, particularly from the mathematical generalization perspective of topology. Besides a systematic comparison in learning mechanisms between the brain and ANNs, we propose "Mental Schema 2.0," a new computational property underlying the brain's unique learning ability that can be implemented in ANNs.

How our understanding of memory replay evolves

Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.

Nonlinear point-process estimation of neural spiking activity based on variational Bayesian inference

OBJECTIVE

Understanding neural encoding and decoding processes are crucial to the development of brain-machine interfaces (BMI). Higher decoding speed of neural signals is required for the large-scale neural data and the extremely low detection delay of closed-loop feedback experiment.

APPROACH

To achieve higher neural decoding speed, we proposed a novel adaptive higher-order nonlinear point-process filter based on the Variational Bayesian Inference (VBI) framework, called the HON-VBI. This algorithm avoids the complex Monte Carlo random sampling in the traditional method. Using the VBI method, it can quickly implement inferences of state posterior distribution and the tuning parameters.

MAIN RESULTS

Our result demonstrates the effectiveness and advantages of the HON-VBI by application for decoding the multichannel neural spike trains of the simulation data and real data. Compared with traditional methods, the HON-VBI greatly reduces the decoding time of large-scale neural spike trains. Through capturing the nonlinear evolution of system state and accurate estimating of time-varying tuning parameters, the decoding accuracy is improved.

SIGNIFICANCE

Our work can be applied to rapidly decode large-scale multichannel neural spike trains in brain-machine interfaces.

Hippocampal replay of experience at real-world speeds

Representations related to past experiences play a critical role in memory and decision-making processes. The rat hippocampus expresses these types of representations during sharp-wave ripple (SWR) events, and previous work identified a minority of SWRs that contain ‘replay’ of spatial trajectories at ∼20x the movement speed of the animal. Efforts to understand replay typically make multiple assumptions about which events to examine and what sorts of representations constitute replay. We therefore lack a clear understanding of both the prevalence and the range of representational dynamics associated with replay. Here, we develop a state space model that uses a combination of movement dynamics of different speeds to capture the spatial content and time evolution of replay during SWRs. Using this model, we find that the large majority of replay events contain spatially coherent, interpretable content. Furthermore, many events progress at real-world, rather than accelerated, movement speeds, consistent with actual experiences.

From Topological Analyses to Functional Modeling: The Case of Hippocampus

Topological data analyses are widely used for describing and conceptualizing large volumes of neurobiological data, e.g., for quantifying spiking outputs of large neuronal ensembles and thus understanding the functions of the corresponding networks. Below we discuss an approach in which convergent topological analyses produce insights into how information may be processed in mammalian hippocampus—a brain part that plays a key role in learning and memory. The resulting functional model provides a unifying framework for integrating spiking data at different timescales and following the course of spatial learning at different levels of spatiotemporal granularity. This approach allows accounting for contributions from various physiological phenomena into spatial cognition—the neuronal spiking statistics, the effects of spiking synchronization by different brain waves, the roles played by synaptic efficacies and so forth. In particular, it is possible to demonstrate that networks with plastic and transient synaptic architectures can encode stable cognitive maps, revealing the characteristic timescales of memory processing.

Efficient Position Decoding Methods Based on Fluorescence Calcium Imaging in the Mouse Hippocampus

Large-scale fluorescence calcium imaging methods have become widely adopted for studies of long-term hippocampal and cortical neuronal dynamics. Pyramidal neurons of the rodent hippocampus show spatial tuning in freely foraging or head-fixed navigation tasks. Development of efficient neural decoding methods for reconstructing the animal's position in real or virtual environments can provide a fast readout of spatial representations in closed-loop neuroscience experiments. Here, we develop an efficient strategy to extract features from fluorescence calcium imaging traces and further decode the animal's position. We validate our spike inference-free decoding methods in multiple in vivo calcium imaging recordings of the mouse hippocampus based on both supervised and unsupervised decoding analyses. We systematically investigate the decoding performance of our proposed methods with respect to the number of neurons, imaging frame rate, and signal-to-noise ratio. Our proposed supervised decoding analysis is ultrafast and robust, and thereby appealing for real-time position decoding applications based on calcium imaging.

Methods for Assessment of Memory Reactivation

It has been suggested that reactivation of previously acquired experiences or stored information in declarative memories in the hippocampus and neocortex contributes to memory consolidation and learning. Understanding memory consolidation depends crucially on the development of robust statistical methods for assessing memory reactivation. To date, several statistical methods have seen established for assessing memory reactivation based on bursts of ensemble neural spike activity during offline states. Using population-decoding methods, we propose a new statistical metric, the weighted distance correlation, to assess hippocampal memory reactivation (i.e., spatial memory replay) during quiet wakefulness and slow-wave sleep. The new metric can be combined with an unsupervised population decoding analysis, which is invariant to latent state labeling and allows us to detect statistical dependency beyond linearity in memory traces. We validate the new metric using two rat hippocampal recordings in spatial navigation tasks. Our proposed analysis framework may have a broader impact on assessing memory reactivations in other brain regions under different behavioral tasks.

Uncovering temporal structure in hippocampal output patterns

Place cell activity of hippocampal pyramidal cells has been described as the cognitive substrate of spatial memory. Replay is observed during hippocampal sharp-wave-ripple-associated population burst events (PBEs) and is critical for consolidation and recall-guided behaviors. PBE activity has historically been analyzed as a phenomenon subordinate to the place code. Here, we use hidden Markov models to study PBEs observed in rats during exploration of both linear mazes and open fields. We demonstrate that estimated models are consistent with a spatial map of the environment, and can even decode animals’ positions during behavior. Moreover, we demonstrate the model can be used to identify hippocampal replay without recourse to the place code, using only PBE model congruence. These results suggest that downstream regions may rely on PBEs to provide a substrate for memory. Additionally, by forming models independent of animal behavior, we lay the groundwork for studies of non-spatial memory.

Topological Schemas of Memory Spaces

Hippocampal cognitive map—a neuronal representation of the spatial environment—is widely discussed in the computational neuroscience literature for decades. However, more recent studies point out that hippocampus plays a major role in producing yet another cognitive framework—the memory space—that incorporates not only spatial, but also non-spatial memories. Unlike the cognitive maps, the memory spaces, broadly understood as “networks of interconnections among the representations of events,” have not yet been studied from a theoretical perspective. Here we propose a mathematical approach that allows modeling memory spaces constructively, as epiphenomena of neuronal spiking activity and thus to interlink several important notions of cognitive neurophysiology. First, we suggest that memory spaces have a topological nature—a hypothesis that allows treating both spatial and non-spatial aspects of hippocampal function on equal footing. We then model the hippocampal memory spaces in different environments and demonstrate that the resulting constructions naturally incorporate the corresponding cognitive maps and provide a wider context for interpreting spatial information. Lastly, we propose a formal description of the memory consolidation process that connects memory spaces to the Morris' cognitive schemas-heuristic representations of the acquired memories, used to explain the dynamics of learning and memory consolidation in a given environment. The proposed approach allows evaluating these constructs as the most compact representations of the memory space's structure.

3-D Maps and Compasses in the Brain

The world has a complex, three-dimensional (3-D) spatial structure, but until recently the neural representation of space was studied primarily in planar horizontal environments. Here we review the emerging literature on allocentric spatial representations in 3-D and discuss the relations between 3-D spatial perception and the underlying neural codes. We suggest that the statistics of movements through space determine the topology and the dimensionality of the neural representation, across species and different behavioral modes. We argue that hippocampal place-cell maps are metric in all three dimensions, and might be composed of 2-D and 3-D fragments that are stitched together into a global 3-D metric representation via the 3-D head-direction cells. Finally, we propose that the hippocampal formation might implement a neural analogue of a Kalman filter, a standard engineering algorithm used for 3-D navigation.

Deciphering Neural Codes of Memory during Sleep

Memories of experiences are stored in the cerebral cortex. Sleep is critical for the consolidation of hippocampal memory of wake experiences into the neocortex. Understanding representations of neural codes of hippocampal-neocortical networks during sleep would reveal important circuit mechanisms in memory consolidation and provide novel insights into memory and dreams. Although sleep-associated ensemble spike activity has been investigated, identifying the content of memory in sleep remains challenging. Here we revisit important experimental findings on sleep-associated memory (i.e., neural activity patterns in sleep that reflect memory processing) and review computational approaches to the analysis of sleep-associated neural codes (SANCs). We focus on two analysis paradigms for sleep-associated memory and propose a new unsupervised learning framework ('memory first, meaning later') for unbiased assessment of SANCs.

A hidden Markov model for decoding and the analysis of replay in spike trains

We present a hidden Markov model that describes variation in an animal's position associated with varying levels of activity in action potential spike trains of individual place cell neurons. The model incorporates a coarse-graining of position, which we find to be a more parsimonious description of the system than other models. We use a sequential Monte Carlo algorithm for Bayesian inference of model parameters, including the state space dimension, and we explain how to estimate position from spike train observations (decoding). We obtain greater accuracy over other methods in the conditions of high temporal resolution and small neuronal sample size. We also present a novel, model-based approach to the study of replay: the expression of spike train activity related to behaviour during times of motionlessness or sleep, thought to be integral to the consolidation of long-term memories. We demonstrate how we can detect the time, information content and compression rate of replay events in simulated and real hippocampal data recorded from rats in two different environments, and verify the correlation between the times of detected replay events and of sharp wave/ripples in the local field potential.

Gamma Synchronization Influences Map Formation Time in a Topological Model of Spatial Learning

The mammalian hippocampus plays a crucial role in producing a cognitive map of space-an internalized representation of the animal's environment. We have previously shown that it is possible to model this map formation using a topological framework, in which information about the environment is transmitted through the temporal organization of neuronal spiking activity, particularly those occasions in which the firing of different place cells overlaps. In this paper, we discuss how gamma rhythm, one of the main components of the extracellular electrical field potential affects the efficiency of place cell map formation. Using methods of algebraic topology and the maximal entropy principle, we demonstrate that gamma modulation synchronizes the spiking of dynamical cell assemblies, which enables learning a spatial map at faster timescales.

Uncovering representations of sleep-associated hippocampal ensemble spike activity

Pyramidal neurons in the rodent hippocampus exhibit spatial tuning during spatial navigation, and they are reactivated in specific temporal order during sharp-wave ripples observed in quiet wakefulness or slow wave sleep. However, analyzing representations of sleep-associated hippocampal ensemble spike activity remains a great challenge. In contrast to wake, during sleep there is a complete absence of animal behavior, and the ensemble spike activity is sparse (low occurrence) and fragmental in time. To examine important issues encountered in sleep data analysis, we constructed synthetic sleep-like hippocampal spike data (short epochs, sparse and sporadic firing, compressed timescale) for detailed investigations. Based upon two Bayesian population-decoding methods (one receptive field-based, and the other not), we systematically investigated their representation power and detection reliability. Notably, the receptive-field-free decoding method was found to be well-tuned for hippocampal ensemble spike data in slow wave sleep (SWS), even in the absence of prior behavioral measure or ground truth. Our results showed that in addition to the sample length, bin size, and firing rate, number of active hippocampal pyramidal neurons are critical for reliable representation of the space as well as for detection of spatiotemporal reactivated patterns in SWS or quiet wakefulness.

A model of topological mapping of space in bat hippocampus

The mammalian hippocampus plays a key role in spatial learning and memory, but the exact nature of the hippocampal representation of space is still being explored. Recently, there has been a fair amount of success in modeling hippocampal spatial maps in rats, assuming a topological perspective on spatial information processing. In this article, we use the topological approach to study the formation of a 3D spatial map in bats, which produces several insights into neurophysiological mechanisms of the hippocampal spatial leaning. First, we demonstrate that, in order to produce accurate maps of the environment, place cell should be organized into functional groups, which can be interpreted as cell assemblies. Second, the model suggests that the readout neurons in these cell assemblies should function as integrators of synaptic inputs, rather than detectors of place cells' coactivity, which allows estimating the integration time window. Lastly, the model suggests that, in contrast with relatively slow moving rats, suppressing θ-precession in bats improves the place cells capacity to encode spatial maps, which is consistent with the experimental observations. © 2016 Wiley Periodicals, Inc.

A Bayesian nonparametric approach for uncovering rat hippocampal population codes during spatial navigation

BACKGROUND

Rodent hippocampal population codes represent important spatial information about the environment during navigation. Computational methods have been developed to uncover the neural representation of spatial topology embedded in rodent hippocampal ensemble spike activity.

NEW METHOD

We extend our previous work and propose a novel Bayesian nonparametric approach to infer rat hippocampal population codes during spatial navigation. To tackle the model selection problem, we leverage a Bayesian nonparametric model. Specifically, we apply a hierarchical Dirichlet process-hidden Markov model (HDP-HMM) using two Bayesian inference methods, one based on Markov chain Monte Carlo (MCMC) and the other based on variational Bayes (VB).

RESULTS

The effectiveness of our Bayesian approaches is demonstrated on recordings from a freely behaving rat navigating in an open field environment.

COMPARISON WITH EXISTING METHODS

The HDP-HMM outperforms the finite-state HMM in both simulated and experimental data. For HPD-HMM, the MCMC-based inference with Hamiltonian Monte Carlo (HMC) hyperparameter sampling is flexible and efficient, and outperforms VB and MCMC approaches with hyperparameters set by empirical Bayes.

CONCLUSION

The Bayesian nonparametric HDP-HMM method can efficiently perform model selection and identify model parameters, which can used for modeling latent-state neuronal population dynamics.

Topological Schemas of Cognitive Maps and Spatial Learning

Spatial navigation in mammals is based on building a mental representation of their environment-a cognitive map. However, both the nature of this cognitive map and its underpinning in neural structures and activity remains vague. A key difficulty is that these maps are collective, emergent phenomena that cannot be reduced to a simple combination of inputs provided by individual neurons. In this paper we suggest computational frameworks for integrating the spiking signals of individual cells into a spatial map, which we call schemas. We provide examples of four schemas defined by different types of topological relations that may be neurophysiologically encoded in the brain and demonstrate that each schema provides its own large-scale characteristics of the environment-the schema integrals. Moreover, we find that, in all cases, these integrals are learned at a rate which is faster than the rate of complete training of neural networks. Thus, the proposed schema framework differentiates between the cognitive aspect of spatial learning and the physiological aspect at the neural network level.

Computational cognitive models of spatial memory in navigation space: a review

Spatial memory refers to the part of the memory system that encodes, stores, recognizes and recalls spatial information about the environment and the agent's orientation within it. Such information is required to be able to navigate to goal locations, and is vitally important for any embodied agent, or model thereof, for reaching goals in a spatially extended environment. In this paper, a number of computationally implemented cognitive models of spatial memory are reviewed and compared. Three categories of models are considered: symbolic models, neural network models, and models that are part of a systems-level cognitive architecture. Representative models from each category are described and compared in a number of dimensions along which simulation models can differ (level of modeling, types of representation, structural accuracy, generality and abstraction, environment complexity), including their possible mapping to the underlying neural substrate. Neural mappings are rarely explicated in the context of behaviorally validated models, but they could be useful to cognitive modeling research by providing a new approach for investigating a model's plausibility. Finally, suggested experimental neuroscience methods are described for verifying the biological plausibility of computational cognitive models of spatial memory, and open questions for the field of spatial memory modeling are outlined.

Neural representation of spatial topology in the rodent hippocampus

Pyramidal cells in the rodent hippocampus often exhibit clear spatial tuning in navigation. Although it has been long suggested that pyramidal cell activity may underlie a topological code rather than a topographic code, it remains unclear whether an abstract spatial topology can be encoded in the ensemble spiking activity of hippocampal place cells. Using a statistical approach developed previously, we investigate this question and related issues in greater detail. We recorded ensembles of hippocampal neurons as rodents freely foraged in one- and two-dimensional spatial environments and used a "decode-to-uncover" strategy to examine the temporally structured patterns embedded in the ensemble spiking activity in the absence of observed spatial correlates during periods of rodent navigation or awake immobility. Specifically, the spatial environment was represented by a finite discrete state space. Trajectories across spatial locations ("states") were associated with consistent hippocampal ensemble spiking patterns, which were characterized by a state transition matrix. From this state transition matrix, we inferred a topology graph that defined the connectivity in the state space. In both one- and two-dimensional environments, the extracted behavior patterns from the rodent hippocampal population codes were compared against randomly shuffled spike data. In contrast to a topographic code, our results support the efficiency of topological coding in the presence of sparse sample size and fuzzy space mapping. This computational approach allows us to quantify the variability of ensemble spiking activity, examine hippocampal population codes during off-line states, and quantify the topological complexity of the environment.

An overview of Bayesian methods for neural spike train analysis

Neural spike train analysis is an important task in computational neuroscience which aims to understand neural mechanisms and gain insights into neural circuits. With the advancement of multielectrode recording and imaging technologies, it has become increasingly demanding to develop statistical tools for analyzing large neuronal ensemble spike activity. Here we present a tutorial overview of Bayesian methods and their representative applications in neural spike train analysis, at both single neuron and population levels. On the theoretical side, we focus on various approximate Bayesian inference techniques as applied to latent state and parameter estimation. On the application side, the topics include spike sorting, tuning curve estimation, neural encoding and decoding, deconvolution of spike trains from calcium imaging signals, and inference of neuronal functional connectivity and synchrony. Some research challenges and opportunities for neural spike train analysis are discussed.

A variational nonparametric Bayesian approach for inferring rat hippocampal population codes

Rodent hippocampal population codes represent important spatial information of the environment during navigation. Several computational methods have been developed to uncover the neural representation of spatial topology embedded in rodent hippocampal ensemble spike activity. Here we extend our previous work and propose a nonparametric Bayesian approach to infer rat hippocampal population codes. Specifically, we develop an infinite hidden Markov model (iHMM) and variational Bayes (VB) inference method to analyze rat hippocampal ensemble spike activity. We demonstrate the effectiveness of our approach using an open field navigation example and discuss the significance/implications of our results.