Sensory feedback, error correction, and remapping in a multiple oscillator model of place-cell activity

Neurodynamical Computing at the Information Boundaries of Intelligent Systems

Artificial intelligence has not achieved defining features of biological intelligence despite models boasting more parameters than neurons in the human brain. In this perspective article, we synthesize historical approaches to understanding intelligent systems and argue that methodological and epistemic biases in these fields can be resolved by shifting away from cognitivist brain-as-computer theories and recognizing that brains exist within large, interdependent living systems. Integrating the dynamical systems view of cognition with the massive distributed feedback of perceptual control theory highlights a theoretical gap in our understanding of nonreductive neural mechanisms. Cell assemblies—properly conceived as reentrant dynamical flows and not merely as identified groups of neurons—may fill that gap by providing a minimal supraneuronal level of organization that establishes a neurodynamical base layer for computation. By considering information streams from physical embodiment and situational embedding, we discuss this computational base layer in terms of conserved oscillatory and structural properties of cortical-hippocampal networks. Our synthesis of embodied cognition, based in dynamical systems and perceptual control, aims to bypass the neurosymbolic stalemates that have arisen in artificial intelligence, cognitive science, and computational neuroscience.

Scalp recorded theta activity is modulated by reward, direction, and speed during virtual navigation in freely moving humans

Theta oscillations (~ 4–12 Hz) are dynamically modulated by speed and direction in freely moving animals. However, due to the paucity of electrophysiological recordings of freely moving humans, this mechanism remains poorly understood. Here, we combined mobile-EEG with fully immersive virtual-reality to investigate theta dynamics in 22 healthy adults (aged 18–29 years old) freely navigating a T-maze to find rewards. Our results revealed three dynamic periods of theta modulation: (1) theta power increases coincided with the participants’ decision-making period; (2) theta power increased for fast and leftward trials as subjects approached the goal location; and (3) feedback onset evoked two phase-locked theta bursts over the right temporal and frontal-midline channels. These results suggest that recording scalp EEG in freely moving humans navigating a simple virtual T-maze can be utilized as a powerful translational model by which to map theta dynamics during “real-life” goal-directed behavior in both health and disease.

Differences in reward biased spatial representations in the lateral septum and hippocampus

The lateral septum (LS), which is innervated by the hippocampus, is known to represent spatial information. However, the details of place representation in the LS, and whether this place information is combined with reward signaling, remains unknown. We simultaneously recorded from rat CA1 and caudodorsal lateral septum in rat during a rewarded navigation task and compared spatial firing in the two areas. While LS place cells are less numerous than in hippocampus, they are similar to the hippocampus in field size and number of fields per cell, but with field shape and center distributions that are more skewed toward reward. Spike cross-correlations between the hippocampus and LS are greatest for cells that have reward-proximate place fields, suggesting a role for the LS in relaying task-relevant hippocampal spatial information to downstream areas, such as the VTA.

Cognitive swarming in complex environments with attractor dynamics and oscillatory computing

Neurobiological theories of spatial cognition developed with respect to recording data from relatively small and/or simplistic environments compared to animals' natural habitats. It has been unclear how to extend theoretical models to large or complex spaces. Complementarily, in autonomous systems technology, applications have been growing for distributed control methods that scale to large numbers of low-footprint mobile platforms. Animals and many-robot groups must solve common problems of navigating complex and uncertain environments. Here, we introduce the NeuroSwarms control framework to investigate whether adaptive, autonomous swarm control of minimal artificial agents can be achieved by direct analogy to neural circuits of rodent spatial cognition. NeuroSwarms analogizes agents to neurons and swarming groups to recurrent networks. We implemented neuron-like agent interactions in which mutually visible agents operate as if they were reciprocally connected place cells in an attractor network. We attributed a phase state to agents to enable patterns of oscillatory synchronization similar to hippocampal models of theta-rhythmic (5-12 Hz) sequence generation. We demonstrate that multi-agent swarming and reward-approach dynamics can be expressed as a mobile form of Hebbian learning and that NeuroSwarms supports a single-entity paradigm that directly informs theoretical models of animal cognition. We present emergent behaviors including phase-organized rings and trajectory sequences that interact with environmental cues and geometry in large, fragmented mazes. Thus, NeuroSwarms is a model artificial spatial system that integrates autonomous control and theoretical neuroscience to potentially uncover common principles to advance both domains.

Representation of visual landmarks in retrosplenial cortex

The process by which visual information is incorporated into the brain’s spatial framework to represent landmarks is poorly understood. Studies in humans and rodents suggest that retrosplenial cortex (RSC) plays a key role in these computations. We developed an RSC-dependent behavioral task in which head-fixed mice learned the spatial relationship between visual landmark cues and hidden reward locations. Two-photon imaging revealed that these cues served as dominant reference points for most task-active neurons and anchored the spatial code in RSC. This encoding was more robust after task acquisition. Decoupling the virtual environment from mouse behavior degraded spatial representations and provided evidence that supralinear integration of visual and motor inputs contributes to landmark encoding. V1 axons recorded in RSC were less modulated by task engagement but showed surprisingly similar spatial tuning. Our data indicate that landmark representations in RSC are the result of local integration of visual, motor, and spatial information.

When moving through a city, people often use notable or familiar landmarks to help them navigate. Landmarks provide us with information about where we are and where we need to go next. But despite the ease with which we – and most other animals – use landmarks to find our way around, it remains unclear exactly how the brain makes this possible.

One area that seems to have a key role is the retrosplenial cortex, which is located deep within the back of the brain in humans. This area becomes more active when animals use visual landmarks to navigate. It is also one of the first brain regions to be affected in Alzheimer's disease, which may help to explain why patients with this condition can become lost and disoriented, even in places they have been many times before.

To find out how the retrosplenial cortex supports navigation, Fischer et al. measured its activity in mice exploring a virtual reality world. The mice ran through simulated corridors in which visual landmarks indicated where hidden rewards could be found. The activity of most neurons in the retrosplenial cortex was most strongly influenced by the mouse’s position relative to the landmark; for example, some neurons were always active 10 centimeters after the landmark.

In other experiments, when the landmarks were present but no longer indicated the location of a reward, the same neurons were much less active. Fischer et al. also measured the activity of the neurons when the mice were running with nothing shown on the virtual reality, and when they saw a landmark but did not run. Notably, the activity seen when the mice were using the landmarks to find rewards was greater than the sum of that recorded when the mice were just running or just seeing the landmark without a reward, making the “landmark response” an example of so-called supralinear processing.

Fischer et al. showed that visual centers of the brain send information about landmarks to retrosplenial cortex. But only the latter adjusts its activity depending on whether the mouse is using that landmark to navigate. These findings provide the first evidence for a “landmark code” at the level of neurons and lay the foundations for studying impaired navigation in patients with Alzheimer's disease. By showing that retrosplenial cortex neurons combine different types of input in a supralinear fashion, the results also point to general principles for how neurons in the brain perform complex calculations.

Hippocampal Global Remapping Can Occur without Input from the Medial Entorhinal Cortex

The high storage capacity of the episodic memory system relies on distinct representations for events that are separated in time and space. The spatial component of these computations includes the formation of independent maps by hippocampal place cells across environments, referred to as global re-mapping. Such remapping is thought to emerge by the switching of input patterns from specialized spatially selective cells in medial entorhinal cortex (mEC), such as grid and border cells. Although it has been shown that acute manipulations of mEC firing patterns are sufficient for inducing hippocampal remapping, it remains unknown whether specialized spatial mEC inputs are necessary for the reorganization of hippocampal spatial representations. Here, we examined remapping in rats without mEC input to the hippocampus and found that highly distinct spatial maps emerged rapidly in every individual rat. Our data suggest that hippocampal spatial computations do not depend on inputs from specialized cell types in mEC.

Origin and role of path integration in the cognitive representations of the hippocampus: computational insights into open questions

Path integration is a straightforward concept with varied connotations that are important to different disciplines concerned with navigation, such as ethology, cognitive science, robotics and neuroscience. In studying the hippocampal formation, it is fruitful to think of path integration as a computation that transforms a sense of motion into a sense of location, continuously integrated with landmark perception. Here, we review experimental evidence that path integration is intimately involved in fundamental properties of place cells and other spatial cells that are thought to support a cognitive abstraction of space in this brain system. We discuss hypotheses about the anatomical and computational origin of path integration in the well-characterized circuits of the rodent limbic system. We highlight how computational frameworks for map-building in robotics and cognitive science alike suggest an essential role for path integration in the creation of a new map in unfamiliar territory, and how this very role can help us make sense of differences in neurophysiological data from novel versus familiar and small versus large environments. Similar computational principles could be at work when the hippocampus builds certain non-spatial representations, such as time intervals or trajectories defined in a sensory stimulus space.

Spatial synchronization codes from coupled rate-phase neurons

During spatial navigation, the frequency and timing of spikes from spatial neurons including place cells in hippocampus and grid cells in medial entorhinal cortex are temporally organized by continuous theta oscillations (6-11 Hz). The theta rhythm is regulated by subcortical structures including the medial septum, but it is unclear how spatial information from place cells may reciprocally organize subcortical theta-rhythmic activity. Here we recorded single-unit spiking from a constellation of subcortical and hippocampal sites to study spatial modulation of rhythmic spike timing in rats freely exploring an open environment. Our analysis revealed a novel class of neurons that we termed 'phaser cells,' characterized by a symmetric coupling between firing rate and spike theta-phase. Phaser cells encoded space by assigning distinct phases to allocentric isocontour levels of each cell's spatial firing pattern. In our dataset, phaser cells were predominantly located in the lateral septum, but also the hippocampus, anteroventral thalamus, lateral hypothalamus, and nucleus accumbens. Unlike the unidirectional late-to-early phase precession of place cells, bidirectional phase modulation acted to return phaser cells to the same theta-phase along a given spatial isocontour, including cells that characteristically shifted to later phases at higher firing rates. Our dynamical models of intrinsic theta-bursting neurons demonstrated that experience-independent temporal coding mechanisms can qualitatively explain (1) the spatial rate-phase relationships of phaser cells and (2) the observed temporal segregation of phaser cells according to phase-shift direction. In open-field phaser cell simulations, competitive learning embedded phase-code entrainment maps into the weights of downstream targets, including path integration networks. Bayesian phase decoding revealed error correction capable of resetting path integration at subsecond timescales. Our findings suggest that phaser cells may instantiate a subcortical theta-rhythmic loop of spatial feedback. We outline a framework in which location-dependent synchrony reconciles internal idiothetic processes with the allothetic reference points of sensory experience.

Alterations of hippocampal place cells in foraging rats facing a "predatory" threat

Fear is an adaptive mechanism evolved to influence the primal decisions of foragers in "approach resource-avoid predator" conflicts. To survive and reproduce, animals must attain the basic needs (food, water, shelter, and mate) while avoiding the ultimate cost of predation. Consistent with this view, ecological studies have found that predatory threats cause animals to limit foraging to fewer places in their habitat and/or to restricted times. However, the neurophysiological basis through which animals alter their foraging boundaries when confronted with danger remains largely unknown. Here, we investigated place cells in the hippocampus, implicated in processing spatial information and memory, in male Long-Evans rats foraging for food under risky situations that would be common in nature. Specifically, place cells from dorsal cornu ammonis field 1 (CA1) were recorded while rats searched for food in a semi-naturalistic apparatus (consisting of a nest and a relatively large open area) before, during, and after encountering a "predatory" robot situated remotely from the nest. The looming robot induced remapping of place fields and increased the theta rhythm as the animals advanced toward the vicinity of threat, but not when they were around the safety of the nest. These neurophysiological effects on the hippocampus were prevented by lesioning of the amygdala. Based on these findings, we suggest that the amygdalar signaling of fear influences the stability of hippocampal place cells as a function of threat distance in rats foraging for food.

Mechanisms of hippocampal astrocytes mediation of spatial memory and theta rhythm by gliotransmitters and growth factors

Our knowledge about encoding and maintenance of spatial memory emphasizes the integrated functional role of the grid cells and the place cells of the hippocampus in the generation of theta rhythm in spatial memory formation. However, the role of astrocytes in these processes is often underestimated in their contribution to the required structural and functional characteristics of hippocampal neural network operative in spatial memory. We show that hippocampal astrocytes, by the secretion of gliotransmitters, such as glutamate, d-serine, and ATP and growth factors such as BDNF and by the expression of receptors and channels such as those of TNFα and aquaporin, have several diverse fuctions in spatial memory. We specifically focus on the role of astrocytes on five phases of spatial memory: (1) theta rhythm generation, (2) theta phase precession, (3) formation of spatial memory by mapping data of entorhinal grid cells into the place cells, (4) storage of spatial information, and (5) maintenance of spatial memory. Finally, by reviewing the literature, we propose specific mechanisms mentioned in the form of a hypothesis suggesting that astrocytes are important in spatial memory formation.

Controlling phase noise in oscillatory interference models of grid cell firing

Oscillatory interference models account for the spatial firing properties of grid cells in terms of neuronal oscillators with frequencies modulated by the animal's movement velocity. The phase of such a "velocity-controlled oscillator" (VCO) relative to a baseline (theta-band) oscillation tracks displacement along a preferred direction. Input from multiple VCOs with appropriate preferred directions causes a grid cell's grid-like firing pattern. However, accumulating phase noise causes the firing pattern to drift and become corrupted. Here we show how multiple redundant VCOs can automatically compensate for phase noise. By entraining the baseline frequency to the mean VCO frequency, VCO phases remain consistent, ensuring a coherent grid pattern and reducing its spatial drift. We show how the spatial stability of grid firing depends on the variability in VCO phases, e.g., a phase SD of 3 ms per 125 ms cycle results in stable grids for 1 min. Finally, coupling N VCOs with similar preferred directions as a ring attractor, so that their relative phases remain constant, produces grid cells with consistently offset grids, and reduces VCO phase variability of the order square root of N. The results suggest a viable functional organization of the grid cell network, and highlight the benefit of integrating displacement along multiple redundant directions for the purpose of path integration.

A hybrid oscillatory interference/continuous attractor network model of grid cell firing

Grid cells in the rodent medial entorhinal cortex exhibit remarkably regular spatial firing patterns that tessellate all environments visited by the animal. Two theoretical mechanisms that could generate this spatially periodic activity pattern have been proposed: oscillatory interference and continuous attractor dynamics. Although a variety of evidence has been cited in support of each, some aspects of the two mechanisms are complementary, suggesting that a combined model may best account for experimental data. The oscillatory interference model proposes that the grid pattern is formed from linear interference patterns or "periodic bands" in which velocity-controlled oscillators integrate self-motion to code displacement along preferred directions. However, it also allows the use of symmetric recurrent connectivity between grid cells to provide relative stability and continuous attractor dynamics. Here, we present simulations of this type of hybrid model, demonstrate that it generates intracellular membrane potential profiles that closely match those observed in vivo, addresses several criticisms aimed at pure oscillatory interference and continuous attractor models, and provides testable predictions for future empirical studies.

Bayesian integration of information in hippocampal place cells

Accurate spatial localization requires a mechanism that corrects for errors, which might arise from inaccurate sensory information or neuronal noise. In this paper, we propose that Hippocampal place cells might implement such an error correction mechanism by integrating different sources of information in an approximately Bayes-optimal fashion. We compare the predictions of our model with physiological data from rats. Our results suggest that useful predictions regarding the firing fields of place cells can be made based on a single underlying principle, Bayesian cue integration, and that such predictions are possible using a remarkably small number of model parameters.

Oscillatory neurocomputing with ring attractors: a network architecture for mapping locations in space onto patterns of neural synchrony

Theories of neural coding seek to explain how states of the world are mapped onto states of the brain. Here, we compare how an animal's location in space can be encoded by two different kinds of brain states: population vectors stored by patterns of neural firing rates, versus synchronization vectors stored by patterns of synchrony among neural oscillators. It has previously been shown that a population code stored by spatially tuned ‘grid cells’ can exhibit desirable properties such as high storage capacity and strong fault tolerance; here it is shown that similar properties are attainable with a synchronization code stored by rhythmically bursting ‘theta cells’ that lack spatial tuning. Simulations of a ring attractor network composed from theta cells suggest how a synchronization code might be implemented using fewer neurons and synapses than a population code with similar storage capacity. It is conjectured that reciprocal connections between grid and theta cells might control phase noise to correct two kinds of errors that can arise in the code: path integration and teleportation errors. Based upon these analyses, it is proposed that a primary function of spatially tuned neurons might be to couple the phases of neural oscillators in a manner that allows them to encode spatial locations as patterns of neural synchrony.

How environment geometry affects grid cell symmetry and what we can learn from it

The mammalian hippocampal formation provides neuronal representations of environmental location but the underlying mechanisms are unclear. The majority of cells in medial entorhinal cortex and parasubiculum show spatially periodic firing patterns. Grid cells exhibit hexagonal symmetry and form an important subset of this more general class. Occasional changes between hexagonal and non-hexagonal firing patterns imply a common underlying mechanism. Importantly, the symmetrical properties are strongly affected by the geometry of the environment. Here, we introduce a field–boundary interaction model where we demonstrate that the grid cell pattern can be formed from competing place-like and boundary inputs. We show that the modelling results can accurately capture our current experimental observations.

Does the entorhinal cortex use the Fourier transform?

Some neurons in the entorhinal cortex (EC) fire bursts when the animal occupies locations organized in a hexagonal grid pattern in their spatial environment. Place cells have also been observed, firing bursts only when the animal occupies a particular region of the environment. Both of these types of cells exhibit theta-cycle modulation, firing bursts in the 4–12 Hz range. Grid cells fire bursts of action potentials that precess with respect to the theta cycle, a phenomenon dubbed “theta precession.” Various models have been proposed to explain these phenomena, and how they relate to navigation. Among the most promising are the oscillator interference models. The bank-of-oscillators model proposed by Welday et al. (2011) exhibits all these features. However, their simulations are based on theoretical oscillators, and not implemented entirely with spiking neurons. We extend their work in a number of ways. First, we place the oscillators in a frequency domain and reformulate the model in terms of Fourier theory. Second, this perspective suggests a division of labor for implementing spatial maps: position vs. map layout. The animal's position is encoded in the phases of the oscillators, while the spatial map shape is encoded implicitly in the weights of the connections between the oscillators and the read-out nodes. Third, it reveals that the oscillator phases all need to conform to a linear relationship across the frequency domain. Fourth, we implement a partial model of the EC using spiking leaky integrate-and-fire (LIF) neurons. Fifth, we devise new coupling mechanisms, enlightened by the global phase constraint, and show they are capable of keeping spiking neural oscillators in consistent formation. Our model demonstrates place cells, grid cells, and phase precession. The Fourier model also gives direction for future investigations, such as integrating sensory feedback to combat drift, or explaining why grid cells exist at all.

Context prediction analysis and episodic memory

Events that happen at a particular place and time come to define our episodic memories. Extensive experimental and clinical research illustrate that the hippocampus is central to the processing of episodic memories, and this is in large part due to its analysis of context information according to spatial and temporal references. In this way, hippocampus defines ones expectations for a given context as well as detects errors in predicted contextual features. The detection of context prediction errors is hypothesized to distinguished events into meaningful epochs that come to be recalled as separate episodic memories. The nature of the spatial and temporal context information processed by hippocampus is described, as is a hypothesis that the apparently self-regulatory nature of hippocampal context processing may ultimately be mediated by natural homeostatic operations and plasticity. Context prediction errors by hippocampus are suggested to be valued by the midbrain dopamine system, the output of which is ultimately fed back to hippocampus to update memory-driven context expectations for future events. Thus, multiple network functions (both within and outside hippocampus) combine to result in adaptive episodic memories.

Lateral entorhinal cortex is necessary for associative but not nonassociative recognition memory

The lateral entorhinal cortex (LEC) provides one of the two major input pathways to the hippocampus and has been suggested to process the nonspatial contextual details of episodic memory. Combined with spatial information from the medial entorhinal cortex it is hypothesised that this contextual information is used to form an integrated spatially selective, context-specific response in the hippocampus that underlies episodic memory. Recently, we reported that the LEC is required for recognition of objects that have been experienced in a specific context (Wilson et al. (2013) Hippocampus 23:352-366). Here, we sought to extend this work to assess the role of the LEC in recognition of all associative combinations of objects, places and contexts within an episode. Unlike controls, rats with excitotoxic lesions of the LEC showed no evidence of recognizing familiar combinations of object in place, place in context, or object in place and context. However, LEC lesioned rats showed normal recognition of objects and places independently from each other (nonassociative recognition). Together with our previous findings, these data suggest that the LEC is critical for associative recognition memory and may bind together information relating to objects, places, and contexts needed for episodic memory formation.

Universal conditions for exact path integration in neural systems

Animals are capable of navigation even in the absence of prominent landmark cues. This behavioral demonstration of path integration is supported by the discovery of place cells and other neurons that show path-invariant response properties even in the dark. That is, under suitable conditions, the activity of these neurons depends primarily on the spatial location of the animal regardless of which trajectory it followed to reach that position. Although many models of path integration have been proposed, no known single theoretical framework can formally accommodate their diverse computational mechanisms. Here we derive a set of necessary and sufficient conditions for a general class of systems that performs exact path integration. These conditions include multiplicative modulation by velocity inputs and a path-invariance condition that limits the structure of connections in the underlying neural network. In particular, for a linear system to satisfy the path-invariance condition, the effective synaptic weight matrices under different velocities must commute. Our theory subsumes several existing exact path integration models as special cases. We use entorhinal grid cells as an example to demonstrate that our framework can provide useful guidance for finding unexpected solutions to the path integration problem. This framework may help constrain future experimental and modeling studies pertaining to a broad class of neural integration systems.

Attractor dynamics of spatially correlated neural activity in the limbic system

Attractor networks are a popular computational construct used to model different brain systems. These networks allow elegant computations that are thought to represent a number of aspects of brain function. Although there is good reason to believe that the brain displays attractor dynamics, it has proven difficult to test experimentally whether any particular attractor architecture resides in any particular brain circuit. We review models and experimental evidence for three systems in the rat brain that are presumed to be components of the rat's navigational and memory system. Head-direction cells have been modeled as a ring attractor, grid cells as a plane attractor, and place cells both as a plane attractor and as a point attractor. Whereas the models have proven to be extremely useful conceptual tools, the experimental evidence in their favor, although intriguing, is still mostly circumstantial.