A view model which accounts for the spatial fields of hippocampal primate spatial view cells and rat place cells

A Theory and Model of Scene Representations With Hippocampal Spatial View Cells

A theory and network model are presented of how scene representations are built by forming spatial view cells in the ventromedial visual cortical scene pathway to the hippocampus in primates including humans. Layer 1, corresponding to V1-V4, connects to Layer 2 in the retrosplenial scene area and uses competitive learning to form visual feature combination neurons for the part of the scene being fixated, a visual fixation scene patch. In Layer 3, corresponding to the parahippocampal scene area and hippocampus, the visual fixation scene patches are stitched together to form whole scene representations. This is performed with a continuous attractor network for a whole scene made from the overlapping Gaussian receptive fields of the neurons as the head rotates to view the whole scene. In addition, in Layer 3, gain modulation by gaze direction maps visual fixation scene patches to the correct part of the whole scene representation when saccades are made. Each neuron in Layer 3 is thus a spatial view cell that responds to a location in a viewed scene based on visual features in a part of the scene. The novel conceptual advances are that this theory shows how scene representations may be built in primates, including humans, based on features in spatial scenes that anchor the scene representation to the world being viewed (to allocentric, world-based, space); and how gaze direction contributes to this. This offers a revolutionary approach to understanding the spatial representations for navigation and episodic memory in primates, including humans.

Hippocampal Discoveries: Spatial View Cells, Connectivity, and Computations for Memory and Navigation, in Primates Including Humans

Two key series of discoveries about the hippocampus are described. One is the discovery of hippocampal spatial view cells in primates. This discovery opens the way to a much better understanding of human episodic memory, for episodic memory prototypically involves a memory of where people or objects or rewards have been seen in locations "out there" which could never be implemented by the place cells that encode the location of a rat or mouse. Further, spatial view cells are valuable for navigation using vision and viewed landmarks, and provide for much richer, vision-based, navigation than the place to place self-motion update performed by rats and mice who live in dark underground tunnels. Spatial view cells thus offer a revolution in our understanding of the functions of the hippocampus in memory and navigation in humans and other primates with well-developed foveate vision. The second discovery describes a computational theory of the hippocampal-neocortical memory system that includes the only quantitative theory of how information is recalled from the hippocampus to the neocortex. It is shown how foundations for this research were the discovery of reward neurons for food reward, and non-reward, in the primate orbitofrontal cortex, and representations of value including of monetary value in the human orbitofrontal cortex; and the discovery of face identity and face expression cells in the primate inferior temporal visual cortex and how they represent transform-invariant information. This research illustrates how in order to understand a brain computation, a whole series of integrated interdisciplinary discoveries is needed to build a theory of the operation of each neural system.

Hippocampal storage and recall of neocortical "What"-"Where" representations

A key question for understanding the function of the hippocampus in memory is how information is recalled from the hippocampus to the neocortex. This was investigated in a neuronal network model of the hippocampal system in which "What" and "Where" neuronal firing rate vectors were applied to separate neocortical modules, which then activated entorhinal cortex "What" and "Where" modules, then the dentate gyrus, then CA3, then CA1, then the entorhinal cortex, and then the backprojections to the neocortex. A rate model showed that the whole system could be trained to recall "Where" in the neocortex from "What" applied as a retrieval cue to the neocortex, and could in principle be trained up towards the theoretical capacity determined largely by the number of synapses onto any one neuron divided by the sparseness of the representation. The trained synaptic weights were then imported into an integrate-and-fire simulation of the same architecture, which showed that the time from presenting a retrieval cue to a neocortex module to recall the whole memory in the neocortex is approximately 100 ms. This is sufficiently fast for the backprojection synapses to be trained onto the still active neocortical neurons during storage of the episodic memory, and this is needed for recall to operate correctly to the neocortex. These simulations also showed that the long loop neocortex-hippocampus-neocortex that operates continuously in time may contribute to complete recall in the neocortex; but that this positive feedback long loop makes the whole dynamical system inherently liable to a pathological increase in neuronal activity. Important factors that contributed to stability included increased inhibition in CA3 and CA1 to keep the firing rates low; and temporal adaptation of the neuronal firing and of active synapses, which are proposed to make an important contribution to stabilizing runaway excitation in cortical circuits in the brain.

A ventromedial visual cortical 'Where' stream to the human hippocampus for spatial scenes revealed with magnetoencephalography

The primate including the human hippocampus implicated in episodic memory and navigation represents a spatial view, very different from the place representations in rodents. To understand this system in humans, and the computations performed, the pathway for this spatial view information to reach the hippocampus was analysed in humans. Whole-brain effective connectivity was measured with magnetoencephalography between 30 visual cortical regions and 150 other cortical regions using the HCP-MMP1 atlas in 21 participants while performing a 0-back scene memory task. In a ventromedial visual stream, V1-V4 connect to the ProStriate region where the retrosplenial scene area is located. The ProStriate region has connectivity to ventromedial visual regions VMV1-3 and VVC. These ventromedial regions connect to the medial parahippocampal region PHA1-3, which, with the VMV regions, include the parahippocampal scene area. The medial parahippocampal regions have effective connectivity to the entorhinal cortex, perirhinal cortex, and hippocampus. In contrast, when viewing faces, the effective connectivity was more through a ventrolateral visual cortical stream via the fusiform face cortex to the inferior temporal visual cortex regions TE2p and TE2a. A ventromedial visual cortical 'Where' stream to the hippocampus for spatial scenes was supported by diffusion topography in 171 HCP participants at 7 T.

Hippocampal spatial view cells for memory and navigation, and their underlying connectivity in humans

Hippocampal and parahippocampal gyrus spatial view neurons in primates respond to the spatial location being looked at. The representation is allocentric, in that the responses are to locations "out there" in the world, and are relatively invariant with respect to retinal position, eye position, head direction, and the place where the individual is located. The underlying connectivity in humans is from ventromedial visual cortical regions to the parahippocampal scene area, leading to the theory that spatial view cells are formed by combinations of overlapping feature inputs self-organized based on their closeness in space. Thus, although spatial view cells represent "where" for episodic memory and navigation, they are formed by ventral visual stream feature inputs in the parahippocampal gyrus in what is the parahippocampal scene area. A second "where" driver of spatial view cells are parietal inputs, which it is proposed provide the idiothetic update for spatial view cells, used for memory recall and navigation when the spatial view details are obscured. Inferior temporal object "what" inputs and orbitofrontal cortex reward inputs connect to the human hippocampal system, and in macaques can be associated in the hippocampus with spatial view cell "where" representations to implement episodic memory. Hippocampal spatial view cells also provide a basis for navigation to a series of viewed landmarks, with the orbitofrontal cortex reward inputs to the hippocampus providing the goals for navigation, which can then be implemented by hippocampal connectivity in humans to parietal cortex regions involved in visuomotor actions in space. The presence of foveate vision and the highly developed temporal lobe for object and scene processing in primates including humans provide a basis for hippocampal spatial view cells to be key to understanding episodic memory in the primate and human hippocampus, and the roles of this system in primate including human navigation.

View cells in the hippocampus and prefrontal cortex of macaques during virtual navigation

Cells selectively activated by a particular view of an environment have been found in the primate hippocampus (HPC). Whether view cells are present in other brain areas, and how view selectivity interacts with other variables such as object features and place remain unclear. Here, we explore these issues by recording the responses of neurons in the HPC and the lateral prefrontal cortex (LPFC) of rhesus macaques performing a task in which they learn new context-object associations while navigating a virtual environment using a joystick. We measured neuronal responses at different locations in a virtual maze where animals freely directed gaze to different regions of the visual scenes. We show that specific views containing task relevant objects selectively activated a proportion of HPC units, and an even higher proportion of LPFC units. Place selectivity was scarce and generally dependent on view. Many view cells were not affected by changing the object color or the context cue, two task relevant features. However, a small proportion of view cells showed selectivity for these two features. Our results show that during navigation in a virtual environment with complex and dynamic visual stimuli, view cells are found in both the HPC and the LPFC. View cells may have developed as a multiarea specialization in diurnal primates to encode the complexities and layouts of the environment through gaze exploration which ultimately enables building cognitive maps of space that guide navigation.

Spatial representability of neuronal activity

A common approach to interpreting spiking activity is based on identifying the firing fields—regions in physical or configuration spaces that elicit responses of neurons. Common examples include hippocampal place cells that fire at preferred locations in the navigated environment, head direction cells that fire at preferred orientations of the animal’s head, view cells that respond to preferred spots in the visual field, etc. In all these cases, firing fields were discovered empirically, by trial and error. We argue that the existence and a number of properties of the firing fields can be established theoretically, through topological analyses of the neuronal spiking activity. In particular, we use Leray criterion powered by persistent homology theory, Eckhoff conditions and Region Connection Calculus to verify consistency of neuronal responses with a single coherent representation of space.

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

First, neurophysiological evidence for the learning of invariant representations in the inferior temporal visual cortex is described. This includes object and face representations with invariance for position, size, lighting, view and morphological transforms in the temporal lobe visual cortex; global object motion in the cortex in the superior temporal sulcus; and spatial view representations in the hippocampus that are invariant with respect to eye position, head direction, and place. Second, computational mechanisms that enable the brain to learn these invariant representations are proposed. For the ventral visual system, one key adaptation is the use of information available in the statistics of the environment in slow unsupervised learning to learn transform-invariant representations of objects. This contrasts with deep supervised learning in artificial neural networks, which uses training with thousands of exemplars forced into different categories by neuronal teachers. Similar slow learning principles apply to the learning of global object motion in the dorsal visual system leading to the cortex in the superior temporal sulcus. The learning rule that has been explored in VisNet is an associative rule with a short-term memory trace. The feed-forward architecture has four stages, with convergence from stage to stage. This type of slow learning is implemented in the brain in hierarchically organized competitive neuronal networks with convergence from stage to stage, with only 4-5 stages in the hierarchy. Slow learning is also shown to help the learning of coordinate transforms using gain modulation in the dorsal visual system extending into the parietal cortex and retrosplenial cortex. Representations are learned that are in allocentric spatial view coordinates of locations in the world and that are independent of eye position, head direction, and the place where the individual is located. This enables hippocampal spatial view cells to use idiothetic, self-motion, signals for navigation when the view details are obscured for short periods.

Neurons including hippocampal spatial view cells, and navigation in primates including humans

A new theory is proposed of mechanisms of navigation in primates including humans in which spatial view cells found in the primate hippocampus and parahippocampal gyrus are used to guide the individual from landmark to landmark. The navigation involves approach to each landmark in turn (taxis), using spatial view cells to identify the next landmark in the sequence, and does not require a topological map. Two other cell types found in primates, whole body motion cells, and head direction cells, can be utilized in the spatial view cell navigational mechanism, but are not essential. If the landmarks become obscured, then the spatial view representations can be updated by self-motion (idiothetic) path integration using spatial coordinate transform mechanisms in the primate dorsal visual system to transform from egocentric to allocentric spatial view coordinates. A continuous attractor network or time cells or working memory is used in this approach to navigation to encode and recall the spatial view sequences involved. I also propose how navigation can be performed using a further type of neuron found in primates, allocentric-bearing-to-a-landmark neurons, in which changes of direction are made when a landmark reaches a particular allocentric bearing. This is useful if a landmark cannot be approached. The theories are made explicit in models of navigation, which are then illustrated by computer simulations. These types of navigation are contrasted with triangulation, which requires a topological map. It is proposed that the first strategy utilizing spatial view cells is used frequently in humans, and is relatively simple because primates have spatial view neurons that respond allocentrically to locations in spatial scenes. An advantage of this approach to navigation is that hippocampal spatial view neurons are also useful for episodic memory, and for imagery.

Goal-directed interaction of stimulus and task demand in the parahippocampal region

The hippocampus and parahippocampal region are essential for representing episodic memories involving various spatial locations and objects, and for using those memories for future adaptive behavior. The “dual‐stream model” was initially formulated based on anatomical characteristics of the medial temporal lobe, dividing the parahippocampal region into two streams that separately process and relay spatial and nonspatial information to the hippocampus. Despite its significance, the dual‐stream model in its original form cannot explain recent experimental results, and many researchers have recognized the need for a modification of the model. Here, we argue that dividing the parahippocampal region into spatial and nonspatial streams a priori may be too simplistic, particularly in light of ambiguous situations in which a sensory cue alone (e.g., visual scene) may not allow such a definitive categorization. Upon reviewing evidence, including our own, that reveals the importance of goal‐directed behavioral responses in determining the relative involvement of the parahippocampal processing streams, we propose the Goal‐directed Interaction of Stimulus and Task‐demand (GIST) model. In the GIST model, input stimuli such as visual scenes and objects are first processed by both the postrhinal and perirhinal cortices—the postrhinal cortex more heavily involved with visual scenes and perirhinal cortex with objects—with relatively little dependence on behavioral task demand. However, once perceptual ambiguities are resolved and the scenes and objects are identified and recognized, the information is then processed through the medial or lateral entorhinal cortex, depending on whether it is used to fulfill navigational or non‐navigational goals, respectively. As complex sensory stimuli are utilized for both navigational and non‐navigational purposes in an intermixed fashion in naturalistic settings, the hippocampus may be required to then put together these experiences into a coherent map to allow flexible cognitive operations for adaptive behavior to occur.

Spatial coordinate transforms linking the allocentric hippocampal and egocentric parietal primate brain systems for memory, action in space, and navigation

A theory and model of spatial coordinate transforms in the dorsal visual system through the parietal cortex that enable an interface via posterior cingulate and related retrosplenial cortex to allocentric spatial representations in the primate hippocampus is described. First, a new approach to coordinate transform learning in the brain is proposed, in which the traditional gain modulation is complemented by temporal trace rule competitive network learning. It is shown in a computational model that the new approach works much more precisely than gain modulation alone, by enabling neurons to represent the different combinations of signal and gain modulator more accurately. This understanding may have application to many brain areas where coordinate transforms are learned. Second, a set of coordinate transforms is proposed for the dorsal visual system/parietal areas that enables a representation to be formed in allocentric spatial view coordinates. The input stimulus is merely a stimulus at a given position in retinal space, and the gain modulation signals needed are eye position, head direction, and place, all of which are present in the primate brain. Neurons that encode the bearing to a landmark are involved in the coordinate transforms. Part of the importance here is that the coordinates of the allocentric view produced in this model are the same as those of spatial view cells that respond to allocentric view recorded in the primate hippocampus and parahippocampal cortex. The result is that information from the dorsal visual system can be used to update the spatial input to the hippocampus in the appropriate allocentric coordinate frame, including providing for idiothetic update to allow for self-motion. It is further shown how hippocampal spatial view cells could be useful for the transform from hippocampal allocentric coordinates to egocentric coordinates useful for actions in space and for navigation.

The storage and recall of memories in the hippocampo-cortical system

A quantitative computational theory of the operation of the hippocampus as an episodic memory system is described. The CA3 system operates as a single attractor or autoassociation network (1) to enable rapid one-trial associations between any spatial location (place in rodents or spatial view in primates) and an object or reward and (2) to provide for completion of the whole memory during recall from any part. The theory is extended to associations between time and object or reward to implement temporal order memory, which is also important in episodic memory. The dentate gyrus performs pattern separation by competitive learning to create sparse representations producing, for example, neurons with place-like fields from entorhinal cortex grid cells. The dentate granule cells generate, by the very small number of mossy fibre connections to CA3, a randomizing pattern separation effect that is important during learning but not recall and that separates out the patterns represented by CA3 firing as being very different from each other. This is optimal for an unstructured episodic memory system in which each memory must be kept distinct from other memories. The direct perforant path input to CA3 is quantitatively appropriate for providing the cue for recall in CA3 but not for learning. The CA1 recodes information from CA3 to set up associatively learned backprojections to the neocortex to allow the subsequent retrieval of information to the neocortex, giving a quantitative account of the large number of hippocampo-neocortical and neocortical-neocortical backprojections. Tests of the theory including hippocampal subregion analyses and hippocampal NMDA receptor knockouts are described and support the theory.

Memory System Neurons Represent Gaze Position and the Visual World

The entorhinal cortex, a brain area critical for memory, contains neurons that fire when a rodent is in a certain location (eg, grid cells), or when a monkey looks at certain locations. In rodents, these spatial representations align to visual objects in the environment by firing when the animal is in a preferred location defined by relative position of visual environmental features. Recently, our laboratory found that simultaneously recorded entorhinal neurons in monkeys can exhibit different spatial reference frames for gaze position, including a reference frame of visual environmental features. We also discovered that most of the neurons represent gaze position. These results suggest that gaze information in multiple spatial reference frames is a potent signal used in the primate memory system. Here, I describe how these findings support three underappreciated views of the hippocampal memory system.

Saccade direction encoding in the primate entorhinal cortex during visual exploration

We recently demonstrated that position in visual space is represented by grid cells in the primate entorhinal cortex (EC), suggesting that visual exploration of complex scenes in primates may employ signaling mechanisms similar to those used during exploration of physical space via movement in rodents. Here, we describe a group of saccade direction (SD) cells that encode eye movement information in the monkey EC during free-viewing of complex images. Significant saccade direction encoding was found in 20% of the cells recorded in the posterior EC. SD cells were generally broadly tuned and two largely separate populations of SD cells encoded future and previous saccade direction. Some properties of these cells resemble those of head-direction cells in rodent EC, suggesting that the same neural circuitry may be capable of performing homologous spatial computations under different exploratory contexts.

Activities of visual cortical and hippocampal neurons co-fluctuate in freely moving rats during spatial behavior

Visual cues exert a powerful control over hippocampal place cell activities that encode external spaces. The functional interaction of visual cortical neurons and hippocampal place cells during spatial navigation behavior has yet to be elucidated. Here we show that, like hippocampal place cells, many neurons in the primary visual cortex (V1) of freely moving rats selectively fire at specific locations as animals run repeatedly on a track. The V1 location-specific activity leads hippocampal place cell activity both spatially and temporally. The precise activities of individual V1 neurons fluctuate every time the animal travels through the track, in a correlated fashion with those of hippocampal place cells firing at overlapping locations. The results suggest the existence of visual cortical neurons that are functionally coupled with hippocampal place cells for spatial processing during natural behavior. These visual neurons may also participate in the formation and storage of hippocampal-dependent memories.

DOI:http://dx.doi.org/10.7554/eLife.08902.001

The brain is able to create and maintain a map of our surroundings as we go about our daily activities. It is thought that some of this spatial information is first processed in a region of the brain called the visual cortex, the information is then relayed to a region called the hippocampus, where the map is reliably stored. However, researchers do not fully understand how the brain transfers spatial information between these two regions.

To explore what happens to the information, Haggerty and Ji recorded electrical signals from the brains of rats that were being trained to run back and forth along a C-shaped track. As they ran on the track, the rats gradually developed spatial maps of there surroundings. Electrodes were used to record brain signals from both the visual cortex and the hippocampus as this development took place.

Similar to previous studies, analysis of the recordings showed that a specific population of neurons in the hippocampus, called CA1 neurons, produced an electrical signal whenever the rat ran past specific locations on the track, such as corners. Building on this Haggerty and Ji showed a population of neurons in the visual cortex called V1 neurons also produced location-specific electrical signals at the same time. Moreover, the electrical signals of these two populations of neurons fluctuate together in a coordinated fashion.

The results of Haggerty and Ji support the idea that there exists a specific population of visual cortical neurons that communicate with hippocampal neurons in the development of a particular spatial map.

DOI:http://dx.doi.org/10.7554/eLife.08902.002

Pattern separation, completion, and categorisation in the hippocampus and neocortex

The mechanisms for pattern completion and pattern separation are described in the context of a theory of hippocampal function in which the hippocampal CA3 system operates as a single attractor or autoassociation network to enable rapid, one-trial, associations between any spatial location (place in rodents, or spatial view in primates) and an object or reward, and to provide for completion of the whole memory during recall from any part. The factors important in the pattern completion in CA3 and also a large number of independent memories stored in CA3 include: a sparse distributed representation, representations that are independent due to the randomizing effect of the mossy fibres, heterosynaptic long-term depression as well as long-term potentiation in the recurrent collateral synapses, and diluted connectivity to minimize the number of multiple synapses between any pair of CA3 neurons which otherwise distort the basins of attraction. Recall of information from CA3 is implemented by the entorhinal cortex perforant path synapses to CA3 cells, which in acting as a pattern associator allow some pattern generalization. Pattern separation is performed in the dentate granule cells using competitive learning to convert grid-like entorhinal cortex firing to place-like fields, and in the dentate to CA3 connections that have diluted connectivity. Recall to the neocortex is achieved by a reverse hierarchical series of pattern association networks implemented by the hippocampo-cortical backprojections, each one of which performs some pattern generalization, to retrieve a complete pattern of cortical firing in higher-order cortical areas. New results on competitive networks show which factors contribute to their ability to perform pattern separation, pattern clustering, and pattern categorisation, and how these apply in different hippocampal and neocortical systems.

Primate spatial strategies and cognition: introduction to this special issue

Wild primates face significant challenges associated with locating resources that involve learning through exploration, encoding, and recalling travel routes, orienting to single landmarks or landmark arrays, monitoring food availability, and applying spatial strategies that reduce effort and increase efficiency. These foraging decisions are likely to involve tradeoffs between traveling to nearby or distant feeding sites based on expectations of resource productivity, predation risk, the availability of other nearby feeding sites, and individual requirements associated with nutrient balancing. Socioecological factors that affect primate foraging decisions include feeding competition, intergroup encounters, mate defense, and opportunities for food sharing. The nine research papers in this Special Issue, "Primate Spatial Strategies and Cognition," address a series of related questions examining how monkeys, apes, and humans encode, internally represent, and integrate spatial, temporal, and quantity information in efficiently locating and relocating productive feeding sites in both small-scale and large-scale space. The authors use a range of methods and approaches to study wild and captive primates, including computer and mathematical modeling, virtual reality, and detailed examinations of animal movement using GPS and GIS analyses to better understand primate cognitive ecology and species differences in decision-making. We conclude this Introduction by identifying a series of critical questions for future research designed to document species-specific differences in primate spatial cognition.

The mechanisms for pattern completion and pattern separation in the hippocampus

The mechanisms for pattern completion and pattern separation are described in the context of a theory of hippocampal function in which the hippocampal CA3 system operates as a single attractor or autoassociation network to enable rapid, one-trial, associations between any spatial location (place in rodents, or spatial view in primates) and an object or reward, and to provide for completion of the whole memory during recall from any part. The factors important in the pattern completion in CA3 together with a large number of independent memories stored in CA3 include a sparse distributed representation which is enhanced by the graded firing rates of CA3 neurons, representations that are independent due to the randomizing effect of the mossy fibers, heterosynaptic long-term depression as well as long-term potentiation in the recurrent collateral synapses, and diluted connectivity to minimize the number of multiple synapses between any pair of CA3 neurons which otherwise distort the basins of attraction. Recall of information from CA3 is implemented by the entorhinal cortex perforant path synapses to CA3 cells, which in acting as a pattern associator allow some pattern generalization. Pattern separation is performed in the dentate granule cells using competitive learning to convert grid-like entorhinal cortex firing to place-like fields. Pattern separation in CA3, which is important for completion of any one of the stored patterns from a fragment, is provided for by the randomizing effect of the mossy fiber synapses to which neurogenesis may contribute, by the large number of dentate granule cells each with a sparse representation, and by the sparse independent representations in CA3. Recall to the neocortex is achieved by a reverse hierarchical series of pattern association networks implemented by the hippocampo-cortical backprojections, each one of which performs some pattern generalization, to retrieve a complete pattern of cortical firing in higher-order cortical areas.

A quantitative theory of the functions of the hippocampal CA3 network in memory

A quantitative computational theory of the operation of the hippocampal CA3 system as an autoassociation or attractor network used in episodic memory system is described. In this theory, the CA3 system operates as a single attractor or autoassociation network to enable rapid, one-trial, associations between any spatial location (place in rodents, or spatial view in primates) and an object or reward, and to provide for completion of the whole memory during recall from any part. The theory is extended to associations between time and object or reward to implement temporal order memory, also important in episodic memory. The dentate gyrus (DG) performs pattern separation by competitive learning to produce sparse representations suitable for setting up new representations in CA3 during learning, producing for example neurons with place-like fields from entorhinal cortex grid cells. The dentate granule cells produce by the very small number of mossy fiber (MF) connections to CA3 a randomizing pattern separation effect important during learning but not recall that separates out the patterns represented by CA3 firing to be very different from each other, which is optimal for an unstructured episodic memory system in which each memory must be kept distinct from other memories. The direct perforant path (pp) input to CA3 is quantitatively appropriate to provide the cue for recall in CA3, but not for learning. Tests of the theory including hippocampal subregion analyses and hippocampal NMDA receptor knockouts are described, and support the theory.

Invariant Visual Object and Face Recognition: Neural and Computational Bases, and a Model, VisNet

Neurophysiological evidence for invariant representations of objects and faces in the primate inferior temporal visual cortex is described. Then a computational approach to how invariant representations are formed in the brain is described that builds on the neurophysiology. A feature hierarchy model in which invariant representations can be built by self-organizing learning based on the temporal and spatial statistics of the visual input produced by objects as they transform in the world is described. VisNet can use temporal continuity in an associative synaptic learning rule with a short-term memory trace, and/or it can use spatial continuity in continuous spatial transformation learning which does not require a temporal trace. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in, for example, spatial and object search tasks. The approach has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene. The approach has also been extended to provide, with an additional layer, for the development of representations of spatial scenes of the type found in the hippocampus.

Neural structures and mechanisms involved in scene recognition: a review and interpretation

Since the discovery in 1996 that a region within caudal parahippocampal cortex subserves learning and recall of topographical information, numerous studies aimed at elucidating the structures and pathways involved in scene recognition have been published. Neuroimaging studies, in particular, have revealed the locations and identities of some of the principal cortical structures that mediate these faculties. In the present study the detailed organization of the system is examined, based on a meta-analysis of neuroimaging studies of scene processing in human subjects, combined with reviews of the results of lesions on this type of processing, single neuron studies, and available hodological data in non-human primates. A cortical hierarchy of structures that mediate scene recognition is established based on these data, and an attempt is made to determine the function of the individual components of the system.

A computational theory of episodic memory formation in the hippocampus

A quantitative computational theory of the operation of the hippocampus as an episodic memory system is described. The CA3 system operates as a single attractor or autoassociation network to enable rapid, one-trial associations between any spatial location (place in rodents or spatial view in primates) and an object or reward and to provide for completion of the whole memory during recall from any part. The theory is extended to associations between time and object or reward to implement temporal order memory, also important in episodic memory. The dentate gyrus performs pattern separation by competitive learning to produce sparse representations, producing for example neurons with place-like fields from entorhinal cortex grid cells. The dentate granule cells produce by the very small number of mossy fibre connections to CA3 a randomizing pattern separation effect important during learning but not recall that separates out the patterns represented by CA3 firing to be very different from each other, which is optimal for an unstructured episodic memory system in which each memory must be kept distinct from other memories. The direct perforant path input to CA3 is quantitatively appropriate to provide the cue for recall in CA3, but not for learning. The CA1 recodes information from CA3 to set up associatively learned backprojections to neocortex to allow subsequent retrieval of information to neocortex, providing a quantitative account of the large number of hippocampo-neocortical and neocortical-neocortical backprojections. Tests of the theory including hippocampal subregion analyses and hippocampal NMDA receptor knockouts are described and support the theory.

Spatial scene representations formed by self-organizing learning in a hippocampal extension of the ventral visual system

We show in a unifying computational approach that representations of spatial scenes can be formed by adding an additional self-organizing layer of processing beyond the inferior temporal visual cortex in the ventral visual stream without the introduction of new computational principles. The invariant representations of objects by neurons in the inferior temporal visual cortex can be modelled by a multilayer feature hierarchy network with feedforward convergence from stage to stage, and an associative learning rule with a short-term memory trace to capture the invariant statistical properties of objects as they transform over short time periods in the world. If an additional layer is added to this architecture, training now with whole scenes that consist of a set of objects in a given fixed spatial relation to each other results in neurons in the added layer that respond to one of the trained whole scenes but do not respond if the objects in the scene are rearranged to make a new scene from the same objects. The formation of these scene-specific representations in the added layer is related to the fact that in the inferior temporal cortex and, we show, in the VisNet model, the receptive fields of inferior temporal cortex neurons shrink and become asymmetric when multiple objects are present simultaneously in a natural scene. This reduced size and asymmetry of the receptive fields of inferior temporal cortex neurons also provides a solution to the representation of multiple objects, and their relative spatial positions, in complex natural scenes.

Neurobiologically inspired mobile robot navigation and planning

After a short review of biologically inspired navigation architectures, mainly relying on modeling the hippocampal anatomy, or at least some of its functions, we present a navigation and planning model for mobile robots. This architecture is based on a model of the hippocampal and prefrontal interactions. In particular, the system relies on the definition of a new cell type "transition cells" that encompasses traditional "place cells".

Slowness and sparseness lead to place, head-direction, and spatial-view cells

We present a model for the self-organized formation of place cells, head-direction cells, and spatial-view cells in the hippocampal formation based on unsupervised learning on quasi-natural visual stimuli. The model comprises a hierarchy of Slow Feature Analysis (SFA) nodes, which were recently shown to reproduce many properties of complex cells in the early visual system []. The system extracts a distributed grid-like representation of position and orientation, which is transcoded into a localized place-field, head-direction, or view representation, by sparse coding. The type of cells that develops depends solely on the relevant input statistics, i.e., the movement pattern of the simulated animal. The numerical simulations are complemented by a mathematical analysis that allows us to accurately predict the output of the top SFA layer.

Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning

'Grid cells' in the dorsocaudal medial entorhinal cortex (dMEC) are activated when a rat is located at any of the vertices of a grid of equilateral triangles covering the environment. dMEC grid cells have different frequencies and phase offsets. However, cells in the dentate gyrus (DG) and hippocampal area CA3 of the rodent typically display place fields, where individual cells are active over only a single portion of the space. In a model of the hippocampus, we have shown that the connectivity from the entorhinal cortex to the dentate granule cells could allow the dentate granule cells to operate as a competitive network to recode their inputs to produce sparse orthogonal representations, and this includes spatial pattern separation. In this paper we show that the same computational hypothesis can account for the mapping of EC grid cells to dentate place cells. We show that the learning in the competitive network is an important part of the way in which the mapping can be achieved. We further show that incorporation of a short term memory trace into the associative learning can help to produce the relatively broad place fields found in the hippocampus.

Spatial view cells in the primate hippocampus and memory recall

Hippocampal spatial view neurons in primates provide allocentric representations of a view of space 'out there'. The responses depend on where the monkey is looking; and can be updated by idiothetic (self-motion) inputs provided by eye movements when the view is hidden. In a room-based object-place memory task, some hippocampal neurons respond to the objects shown, some to the places viewed, and some to combinations of the places viewed and the objects present in those locations. In an object-place recall task when the location in space at which an object has been seen is recalled by the presentation of the object, some primate hippocampal neurons maintain their responding to the object recall cue in a delay period without the object visible while the place is being recalled; and other neurons respond to the place being recalled. Other spatial view neurons form associations with the rewards present at particular locations in space. These findings, and computational models of the hippocampus, help to show how the primate including human hippocampus is involved in episodic memory.

A computational theory of hippocampal function, and empirical tests of the theory

The main aim of the paper is to present an up-to-date computational theory of hippocampal function and the predictions it makes about the different subregions (dentate gyrus, CA3 and CA1), and to examine behavioral and electrophysiological data that address the functions of the hippocampus and particularly its subregions. Based on the computational proposal that the dentate gyrus produces sparse representations by competitive learning and via the mossy fiber pathway forces new representations on the CA3 during learning (encoding), it has been shown behaviorally that the dentate gyrus supports spatial pattern separation during learning. Based on the computational proposal that CA3-CA3 autoassociative networks are important for episodic memory, it has been shown behaviorally that the CA3 supports spatial rapid one-trial learning, learning of arbitrary associations where space is a component, pattern completion, spatial short-term memory, and sequence learning by associations formed between successive items. The concept that the CA1 recodes information from CA3 and sets up associatively learned backprojections to neocortex to allow subsequent retrieval of information to neocortex, is consistent with findings on consolidation. Behaviorally, the CA1 is implicated in processing temporal information as shown by investigations requiring temporal order pattern separation and associations across time; computationally this could involve temporal decay memory, and temporal sequence memory which might also require CA3. The perforant path input to DG is implicated in learning, to CA3 in retrieval from CA3, and to CA1 in retrieval after longer time intervals ("intermediate-term memory").

Spatial view cells in the hippocampus, and their idiothetic update based on place and head direction

Single neuron recording studies have demonstrated the existence of spatial view neurons which encode information about the spatial location at which a primate is looking in the environment. These neurons are able to maintain their firing even in the absence of visual input. The standard neuronal network approach to model networks with memory that represent continuous spaces is that of continuous attractor neural networks. Stringer, Rolls and Trappenberg (2005) have recently shown how idiothetic (self-motion) inputs could update the activity packet of neuronal firing within a two-dimensional continuous attractor neural network of spatial view cells. However, this earlier study examined only the simplified situation in which the agent could rotate on the spot or move its eyes. In this paper we show how spatial view cells could be driven by head direction and place cells, themselves idiothetically updated. The head direction and place neurons are remapped by a competitive network with expansion recoding which self-organises so that different neurons represent different combinations of head direction and the place where the agent is located. The combination cells are then mapped by pattern association involving long-term synaptic potentiation but also long-term homosynaptic depression to spatial view cells, which during training are driven by the spatial view. After training, the spatial view cells are updated in the dark by the idiothetically driven head direction and place cells.

A Hierarchy of Associations in Hippocampo-Cortical Systems: Cognitive Maps and Navigation Strategies

In this letter we describe a hippocampo-cortical model of spatial processing and navigation based on a cascade of increasingly complex associative processes that are also relevant for other hippocampal functions such as episodic memory. Associative learning of different types and the related pattern encoding-recognition take place at three successive levels: (1) an object location level, which computes the landmarks from merged multimodal sensory inputs in the parahippocampal cortices; (2) a subject location level, which computes place fields by combination of local views and movement-related information in the entorhinal cortex; and (3) a spatiotemporal level, which computes place transitions from contiguous place fields in the CA3-CA1 region, which form building blocks for learning temporospatial sequences.

At the cell population level, superficial entorhinal place cells encode spatial, context-independent maps as landscapes of activity; populations of transition cells in the CA3-CA1 region encode context-dependent maps as sequences of transitions, which form graphs in prefrontal-parietal cortices. The model was tested on a robot moving in a real environment; these tests produced results that could help to interpret biological data.

Two different goal-oriented navigation strategies were displayed depend-ing on the type of map used by the system.

Thanks to its multilevel, multimodal integration and behavioral imple-mentation, the model suggests functional interpretations for largely un-accounted structural differences between hippocampo-cortical systems. Further, spatiotemporal information, a common denominator shared by several brain structures, could serve as a cognitive processing frame and a functional link, for example, during spatial navigation and episodic memory, as suggested by the applications of the model to other domains, temporal sequence learning and imitation in particular.