Eye position-sensitive units in hippocampal formation and in inferotemporal cortex of the macaque monkey

Computational cross-species views of the hippocampal formation

The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.

View cells in the hippocampus and prefrontal cortex of macaques during virtual navigation

Cells selectively activated by a particular view of an environment have been found in the primate hippocampus (HPC). Whether view cells are present in other brain areas, and how view selectivity interacts with other variables such as object features and place remain unclear. Here, we explore these issues by recording the responses of neurons in the HPC and the lateral prefrontal cortex (LPFC) of rhesus macaques performing a task in which they learn new context-object associations while navigating a virtual environment using a joystick. We measured neuronal responses at different locations in a virtual maze where animals freely directed gaze to different regions of the visual scenes. We show that specific views containing task relevant objects selectively activated a proportion of HPC units, and an even higher proportion of LPFC units. Place selectivity was scarce and generally dependent on view. Many view cells were not affected by changing the object color or the context cue, two task relevant features. However, a small proportion of view cells showed selectivity for these two features. Our results show that during navigation in a virtual environment with complex and dynamic visual stimuli, view cells are found in both the HPC and the LPFC. View cells may have developed as a multiarea specialization in diurnal primates to encode the complexities and layouts of the environment through gaze exploration which ultimately enables building cognitive maps of space that guide navigation.

Allocentric information represented by self-referenced spatial coding in the primate medial temporal lobe

For living organisms, the ability to acquire information regarding the external space around them is critical for future actions. While the information must be stored in an allocentric frame to facilitate its use in various spatial contexts, each case of use requires the information to be represented in a particular self-referenced frame. Previous studies have explored neural substrates responsible for the linkage between self-referenced and allocentric spatial representations based on findings in rodents. However, the behaviors of rodents are different from those of primates in several aspects; for example, rodents mainly explore their environments through locomotion, while primates use eye movements. In this review, we discuss the brain mechanisms responsible for the linkage in nonhuman primates. Based on recent physiological studies, we propose that two types of neural substrates link the first-person perspective with allocentric coding. The first is the view-center background signal, which represents an image of the background surrounding the current position of fixation on the retina. This perceptual signal is transmitted from the ventral visual pathway to the hippocampus (HPC) via the perirhinal cortex and parahippocampal cortex. Because images that share the same objective-position in the environment tend to appear similar when seen from different self-positions, the view-center background signals are easily associated with one another in the formation of allocentric position coding and storage. The second type of neural substrate is the HPC neurons' dynamic activity that translates the stored location memory to the first-person perspective depending on the current spatial context.

Neural Correlates of Spatial Navigation in Primate Hippocampus

The hippocampus has been extensively implicated in spatial navigation in rodents and more recently in bats. Numerous studies have revealed that various kinds of spatial information are encoded across hippocampal regions. In contrast, investigations of spatial behavioral correlates in the primate hippocampus are scarce and have been mostly limited to head-restrained subjects during virtual navigation. However, recent advances made in freely-moving primates suggest marked differences in spatial representations from rodents, albeit some similarities. Here, we review empirical studies examining the neural correlates of spatial navigation in the primate (including human) hippocampus at the levels of local field potentials and single units. The lower frequency theta oscillations are often intermittent. Single neuron responses are highly mixed and task-dependent. We also discuss neuronal selectivity in the eye and head coordinates. Finally, we propose that future studies should focus on investigating both intrinsic and extrinsic population activity and examining spatial coding properties in large-scale hippocampal-neocortical networks across tasks.

Reunification of Object and View-Center Background Information in the Primate Medial Temporal Lobe

Recent work has shown that the medial temporal lobe (MTL), including the hippocampus (HPC) and its surrounding limbic cortices, plays a role in scene perception in addition to episodic memory. The two basic factors of scene perception are the object (“what”) and location (“where”). In this review, we first summarize the anatomical knowledge related to visual inputs to the MTL and physiological studies examining object-related information processed along the ventral pathway briefly. Thereafter, we discuss the space-related information, the processing of which was unclear, presumably because of its multiple aspects and a lack of appropriate task paradigm in contrast to object-related information. Based on recent electrophysiological studies using non-human primates and the existing literature, we proposed the “reunification theory,” which explains brain mechanisms which construct object-location signals at each gaze. In this reunification theory, the ventral pathway signals a large-scale background image of the retina at each gaze position. This view-center background signal reflects the first person’s perspective and specifies the allocentric location in the environment by similarity matching between images. The spatially invariant object signal and view-center background signal, both of which are derived from the same retinal image, are integrated again (i.e., reunification) along the ventral pathway-MTL stream, particularly in the perirhinal cortex. The conjunctive signal, which represents a particular object at a particular location, may play a role in scene perception in the HPC as a key constituent element of an entire scene.

Modeling the Ventral and Dorsal Cortical Visual Pathways Using Artificial Neural Networks

Although in conventional models of cortical processing, object recognition and spatial properties are processed separately in ventral and dorsal cortical visual pathways respectively, some recent studies have shown that representations associated with both objects' identity (of shape) and space are present in both visual pathways. However, it is still unclear whether the presence of identity and spatial properties in both pathways have functional roles. In our study, we have tried to answer this question through computational modeling. Our simulation results show that both a model ventral and dorsal pathway, separately trained to do object and spatial recognition, respectively, each actively retained information about both identity and space. In addition, we show that these networks retained different amounts and kinds of identity and spatial information. As a result, our modeling suggests that two separate cortical visual pathways for identity and space (1) actively retain information about both identity and space (2) retain information about identity and space differently and (3) that this differently retained information about identity and space in the two pathways may be necessary to accurately and optimally recognize and localize objects. Further, modeling results suggests these findings are robust and do not strongly depend on the specific structures of the neural networks.

Spatial modulation of hippocampal activity in freely moving macaques

The hippocampal formation is linked to spatial navigation, but there is little corroboration from freely moving primates with concurrent monitoring of head and gaze stances. We recorded neural activity across hippocampal regions in rhesus macaques during free foraging in an open environment while tracking their head and eye. Theta activity was intermittently present at movement onset and modulated by saccades. Many neurons were phase-locked to theta, with few showing phase precession. Most neurons encoded a mixture of spatial variables beyond place and grid tuning. Spatial representations were dominated by facing location and allocentric direction, mostly in head, rather than gaze, coordinates. Importantly, eye movements strongly modulated neural activity in all regions. These findings reveal that the macaque hippocampal formation represents three-dimensional (3D) space using a multiplexed code, with head orientation and eye movement properties being dominant during free exploration.

Forward Processing of Object-Location Association from the Ventral Stream to Medial Temporal Lobe in Nonhuman Primates

While the hippocampus (HPC) is a prime candidate combining object identity and location due to its strong connections to the ventral and dorsal pathways via surrounding medial temporal lobe (MTL) areas, recent physiological studies have reported spatial information in the ventral pathway and its downstream target in MTL. However, it remains unknown whether the object-location association proceeds along the ventral MTL pathway before HPC. To address this question, we recorded neuronal activity from MTL and area anterior inferotemporal cortex (TE) of two macaques gazing at an object to retain its identity and location in each trial. The results showed significant effects of object-location association at a single-unit level in TE, perirhinal cortex (PRC), and HPC, but not in the parahippocampal cortex. Notably, a clear area difference emerged in the association form: 1) representations of object identity were added to those of subjects' viewing location in TE; 2) PRC signaled both the additive form and the conjunction of the two inputs; and 3) HPC signaled only the conjunction signal. These results suggest that the object and location signals are combined stepwise at TE and PRC each time primates view an object, and PRC may provide HPC with the conjunctional signal, which might be used for encoding episodic memory.

The intersection between the oculomotor and hippocampal memory systems: empirical developments and clinical implications

Decades of cognitive neuroscience research has shown that where we look is intimately connected to what we remember. In this article, we review findings from human and nonhuman animals, using behavioral, neuropsychological, neuroimaging, and computational modeling methods, to show that the oculomotor and hippocampal memory systems interact in a reciprocal manner, on a moment-to-moment basis, mediated by a vast structural and functional network. Visual exploration serves to efficiently gather information from the environment for the purpose of creating new memories, updating existing memories, and reconstructing the rich, vivid details from memory. Conversely, memory increases the efficiency of visual exploration. We call for models of oculomotor control to consider the influence of the hippocampal memory system on the cognitive control of eye movements, and for models of hippocampal and broader medial temporal lobe function to consider the influence of the oculomotor system on the development and expression of memory. We describe eye movement-based applications for the detection of neurodegeneration and delivery of therapeutic interventions for mental health disorders for which the hippocampus is implicated and memory dysfunctions are at the forefront.

A Stable Visual World in Primate Primary Visual Cortex

Humans and other primates rely on eye movements to explore visual scenes and to track moving objects. As a result, the image that is projected onto the retina-and propagated throughout the visual cortical hierarchy-is almost constantly changing and makes little sense without taking into account the momentary direction of gaze. How is this achieved in the visual system? Here, we show that in primary visual cortex (V1), the earliest stage of cortical vision, neural representations carry an embedded "eye tracker" that signals the direction of gaze associated with each image. Using chronically implanted multi-electrode arrays, we recorded the activity of neurons in area V1 of macaque monkeys during tasks requiring fast (exploratory) and slow (pursuit) eye movements. Neurons were stimulated with flickering, full-field luminance noise at all times. As in previous studies, we observed neurons that were sensitive to gaze direction during fixation, despite comparable stimulation of their receptive fields. We trained a decoder to translate neural activity into metric estimates of gaze direction. This decoded signal tracked the eye accurately not only during fixation but also during fast and slow eye movements. After a fast eye movement, the eye-position signal arrived in V1 at approximately the same time at which the new visual information arrived from the retina. Using simulations, we show that this V1 eye-position signal could be used to take into account the sensory consequences of eye movements and map the fleeting positions of objects on the retina onto their stable position in the world.

Spatial Responses, Immediate Experience, and Memory in the Monkey Hippocampus

Debate about the function of the hippocampus often pits theories advocating for spatial mapping against those that argue for a central role in memory. This review addresses whether research in the monkey supports the view that processing spatial information is fundamental to the function of the hippocampus. In support of spatial processing theories, neurons in the monkey hippocampal formation have striking spatial tuning, and an intact hippocampus is necessary to effectively utilize allocentric spatial relationships. However, the hippocampus also supports non-spatial processes, as its neurons acutely respond to distinct task events and hippocampal damage disrupts both expedient task acquisition and the monitoring of ongoing events in non-spatial paradigms. The features that are shared between spatial and non-spatial hippocampal-dependent tasks point toward a common mechanism underlying hippocampal function that is independent of processing spatial information. We suggest that spatial information is only one facet of immediate experience represented by the hippocampus. The current data support the idea that the hippocampus tracks many aspects of ongoing experience and the primary role of the hippocampus may be in linking experienced events into unitary episodes.

Getting directions from the hippocampus: The neural connection between looking and memory

Investigations into the neural basis of memory in human and non-human primates have focused on the hippocampus and associated medial temporal lobe (MTL) structures. However, how memory signals from the hippocampus affect motor actions is unknown. We propose that approaching this question through eye movement, especially by assessing the changes in looking behavior that occur with experience, is a promising method for exposing neural computations within the hippocampus. Here, we review how looking behavior is guided by memory in several ways, some of which have been shown to depend on the hippocampus, and how hippocampal neural signals are modulated by eye movements. Taken together, these findings highlight the need for future research on how MTL structures interact with the oculomotor system. Probing how the hippocampus reflects and impacts motor output during looking behavior renders a practical path to advance our understanding of the hippocampal memory system.

Navigational Style Influences Eye Movement Pattern during Exploration and Learning of an Environmental Map

During navigation people may adopt three different spatial styles (i.e., Landmark, Route, and Survey). Landmark style (LS) people are able to recall familiar landmarks but cannot combine them with directional information; Route style (RS) people connect landmarks to each other using egocentric information about direction; Survey style (SS) people use a map-like representation of the environment. SS individuals generally navigate better than LS and RS people. Fifty-one college students (20 LS; 17 RS, and 14 SS) took part in the experiment. The spatial cognitive style (SCS) was assessed by means of the SCS test; participants then had to learn a schematic map of a city, and after 5 min had to recall the path depicted on it. During the learning and delayed recall phases, eye-movements were recorded. Our intent was to investigate whether there is a peculiar way to explore an environmental map related to the individual’s spatial style. Results support the presence of differences in the strategy used by the three spatial styles for learning the path and its delayed recall. Specifically, LS individuals produced a greater number of fixations of short duration, while the opposite eye movement pattern characterized SS individuals. Moreover, SS individuals showed a more spread and comprehensive explorative pattern of the map, while LS individuals focused their exploration on the path and related targets. RS individuals showed a pattern of exploration at a level of proficiency between LS and SS individuals. We discuss the clinical and anatomical implications of our data.

Characteristics of Eye-Position Gain Field Populations Determine Geometry of Visual Space

We have previously demonstrated differences in eye-position spatial maps for anterior inferotemporal cortex (AIT) in the ventral stream and lateral intraparietal cortex (LIP) in the dorsal stream, based on population decoding of gaze angle modulations of neural visual responses (i.e., eye-position gain fields). Here we explore the basis of such spatial encoding differences through modeling of gain field characteristics. We created a population of model neurons, each having a different eye-position gain field. This population was used to reconstruct eye-position visual space using multidimensional scaling. As gain field shapes have never been well-established experimentally, we examined different functions, including planar, sigmoidal, elliptical, hyperbolic, and mixtures of those functions. All functions successfully recovered positions, indicating weak constraints on allowable gain field shapes. We then used a genetic algorithm to modify the characteristics of model gain field populations until the recovered spatial maps closely matched those derived from monkey neurophysiological data in AIT and LIP. The primary differences found between model AIT and LIP gain fields were that AIT gain fields were more foveally dominated. That is, gain fields in AIT operated on smaller spatial scales and smaller dispersions than in LIP. Thus, we show that the geometry of eye-position visual space depends on the population characteristics of gain fields, and that differences in gain field characteristics for different cortical areas may underlie differences in the representation of space.

Asymmetries in perception of 3D orientation

Visual scene interpretation depends on assumptions based on the statistical regularities of the world. People have some preference for seeing ambiguously oriented objects (Necker cubes) as if tilted down or viewed from above. This bias is a near certainty in the first instant (∼1 s) of viewing and declines over the course of many seconds. In addition, we found that there is modulation of perceived orientation that varies with position—for example objects on the left are more likely to be interpreted as viewed from the right. Therefore there is both a viewed-from-above prior and a scene position-dependent modulation of perceived 3-D orientation. These results are consistent with the idea that ambiguously oriented objects are initially assigned an orientation consistent with our experience of an asymmetric world in which objects most probably sit on surfaces below eye level.

Spatial modulation of primate inferotemporal responses by eye position

Background

A key aspect of representations for object recognition and scene analysis in the ventral visual stream is the spatial frame of reference, be it a viewer-centered, object-centered, or scene-based coordinate system. Coordinate transforms from retinocentric space to other reference frames involve combining neural visual responses with extraretinal postural information.

Methodology/Principal Findings

We examined whether such spatial information is available to anterior inferotemporal (AIT) neurons in the macaque monkey by measuring the effect of eye position on responses to a set of simple 2D shapes. We report, for the first time, a significant eye position effect in over 40% of recorded neurons with small gaze angle shifts from central fixation. Although eye position modulates responses, it does not change shape selectivity.

Conclusions/Significance

These data demonstrate that spatial information is available in AIT for the representation of objects and scenes within a non-retinocentric frame of reference. More generally, the availability of spatial information in AIT calls into questions the classic dichotomy in visual processing that associates object shape processing with ventral structures such as AIT but places spatial processing in a separate anatomical stream projecting to dorsal structures.

Alternate Task Inhibits Single-neuron Category-selective Responses in the Human Hippocampus while Preserving Selectivity in the Amygdala

One fifth of neurons in the medial-temporal lobe of human epilepsy patients respond selectively to categories of images, such as faces or cars. Here we show that responses of hippocampal neurons are rapidly modified as subjects alternate (over 60 sec) between two tasks (1) identifying images from a category, or (2) playing a simple video game superimposed on the same images. Category-selective responses, present when a subject identifies categories, are eliminated when the subject shifts to playing the game for 87% of category-selective hippocampal neurons. By contrast, responses in the amygdala are present during both tasks for 72% of category-selective amygdalar neurons. These results suggest that attention to images is required to evoke selective responses from single neurons in the hippocampus, but is not required by neurons in the amygdala.

Beyond Retinotopic Mapping: The Spatial Representation of Objects in the Human Lateral Occipital Complex

The spatial representation in the human ventral object-related areas (i.e., the lateral occipital complex [LOC]) is currently unknown. It seems plausible, however, that it would diverge from the strict retinotopic mapping (characteristic of V1) to a more invariant coordinate frame, thereby allowing for reliable object recognition in the face of eye, head, or body movement. To study this, we compared the fMRI activation in LOC when object displacement was limited to either the retina or the screen by manipulating eye position and object locations. We found clear adaptation in LOC when the object's screen position was fixed, regardless of the object's retinal position. Furthermore, we found significantly greater activation in LOC in the hemisphere contralateral to the object's screen position, although the visual task was constructed in a way that the objects were present equally often on each of the 2 retinal hemifields. Together, these results indicate that a sizeable fraction of the neurons in LOC may have head-based receptive fields. Such an extraretinal representation may be useful for maintenance of object coherence across saccadic eye movements, which are an integral part of natural vision.

Navigation in space--the role of the macaque ventral intraparietal area

Goal-directed self-motion through space is anything but a trivial task. What we take for granted in everyday life requires the complex interplay of different sensory and motor systems. On the sensory side most importantly a target of interest has to be localized relative to one's own position in space. On the motor side the most critical step in neural processing is to define and perform a movement towards the target as well as the avoidance of obstacles. Furthermore, the multisensory (visual, tactile and auditory) motion signals as induced by one's own movement have to be identified and differentiated from the real motion of visual, tactile or auditory objects in the outside world. In a number of experimental studies performed in recent years we and others have functionally characterized a subregion within monkey posterior parietal cortex (PPC) that appears to be well suited to contribute to such multisensory encoding of spatial and motion information. In this review I will summarize the most important experimental findings on the functional properties of this very region in monkey PPC, i.e. the ventral intraparietal area.

Evidence for implication of primate area V1 in neural 3-D spatial localization processing

We investigated the neural mechanisms underlying visual localization in 3-D space in area V1 of behaving monkeys. Three different sources of information, retinal disparity, viewing distance and gaze direction, that participate in these neural mechanisms are being reviewed. The way they interact with each other is studied by combining retinal and extraretinal signals. Interactions between retinal disparity and viewing distance have been shown in foveal V1; we have observed a strong modulation of the spontaneous activity and of the visual response of most V1 cells that was highly correlated with the vergence angle. As a consequence of these gain effects, neural horizontal disparity coding is favoured or refined for particular distances of fixation. Changing the gaze direction in the fronto-parallel plane also produces strong gains in the visual response of half of the cells in foveal V1. Cells tested for horizontal disparity and orientation selectivities show gain effects that occur coherently for the same spatial coordinates of the eyes. Shifts in preferred disparity also occurred in several neurons. Cells tested in calcarine V1 at retinal eccentricities larger than 10 degrees , show that horizontal disparity is encoded at least up to 20 degrees around both the horizontal and vertical meridians. At these large retinal eccentricities we found that vertical disparity is also encoded with tuning profiles similar to those of horizontal disparity coding. Combinations of horizontal and vertical disparity signals show that most cells encode both properties. In fact the expression of horizontal disparity coding depends on the vertical disparity signals that produce strong gain effects and frequent changes in peak selectivities. We conclude that the vertical disparity signal and the eye position signal serve to disambiguate the horizontal disparity signal to provide information on 3-D spatial coordinates in terms of distance, gaze direction and retinal eccentricity. We suggest that the relative weight among these different signals is the determining factor involved in the neural processing that gives information on 3-D spatial localization.

How parallel is visual processing in the ventral pathway?

Visual object perception is usually studied by presenting one object at a time at the fovea. However, the world around us is composed of multiple objects. The way our visual system deals with this complexity has remained controversial in the literature. Some models claim that the ventral pathway, a set of visual cortical areas responsible for object recognition, can process only one or very few objects at a time without ambiguity. Other models argue in favor of a massively parallel processing of objects in a scene. Recent experiments in monkeys have provided important data about this issue. The ventral pathway seems to be able to perform complex analyses on several objects simultaneously, but only during a short time period. Subsequently only one or very few objects are explicitly selected and consciously perceived. Here, we survey the implications of these new findings for our understanding of object processing.