Single-Neuron Representations of Spatial Targets in Humans

On the origin of memory neurons in the human hippocampus

The hippocampus is essential for episodic memory, yet its coding mechanism remains debated. In humans, two main theories have been proposed: one suggests that concept neurons represent specific elements of an episode, while another posits a conjunctive code, where index neurons code the entire episode. Here, we integrate new findings of index neurons in humans and other animals with the concept-specific memory framework, proposing that concept neurons evolve from index neurons through overlapping memories. This process is supported by engram literature, which posits that neurons are allocated to a memory trace based on excitability and that reactivation induces excitability. By integrating these insights, we connect two historically disparate fields of neuroscience: engram research and human single neuron episodic memory research.

A Theory and Model of Scene Representations With Hippocampal Spatial View Cells

A theory and network model are presented of how scene representations are built by forming spatial view cells in the ventromedial visual cortical scene pathway to the hippocampus in primates including humans. Layer 1, corresponding to V1-V4, connects to Layer 2 in the retrosplenial scene area and uses competitive learning to form visual feature combination neurons for the part of the scene being fixated, a visual fixation scene patch. In Layer 3, corresponding to the parahippocampal scene area and hippocampus, the visual fixation scene patches are stitched together to form whole scene representations. This is performed with a continuous attractor network for a whole scene made from the overlapping Gaussian receptive fields of the neurons as the head rotates to view the whole scene. In addition, in Layer 3, gain modulation by gaze direction maps visual fixation scene patches to the correct part of the whole scene representation when saccades are made. Each neuron in Layer 3 is thus a spatial view cell that responds to a location in a viewed scene based on visual features in a part of the scene. The novel conceptual advances are that this theory shows how scene representations may be built in primates, including humans, based on features in spatial scenes that anchor the scene representation to the world being viewed (to allocentric, world-based, space); and how gaze direction contributes to this. This offers a revolutionary approach to understanding the spatial representations for navigation and episodic memory in primates, including humans.

Visual cortical networks for "What" and "Where" to the human hippocampus revealed with dynamical graphs

Key questions for understanding hippocampal function in memory and navigation in humans are the type and source of visual information that reaches the human hippocampus. We measured bidirectional pairwise effective connectivity with functional magnetic resonance imaging between 360 cortical regions while 956 Human Connectome Project participants viewed scenes, faces, tools, or body parts. We developed a method using deterministic dynamical graphs to define whole cortical networks and the flow in both directions between their cortical regions over timesteps after signal is applied to V1. We revealed that a ventromedial cortical visual "Where" network from V1 via the retrosplenial and medial parahippocampal scene areas reaches the hippocampus when scenes are viewed. A ventrolateral "What" visual cortical network reaches the hippocampus from V1 via V2-V4, the fusiform face cortex, and lateral parahippocampal region TF when faces/objects are viewed. There are major implications for understanding the computations of the human vs rodent hippocampus in memory and navigation: primates with their fovea and highly developed cortical visual processing networks process information about the location of faces, objects, and landmarks in viewed scenes, whereas in rodents the representations in the hippocampal system are mainly about the place where the individual is located and self-motion between places.

Brain local structural connectomes and the subtypes of the medial temporal lobe parcellations

Objective

To investigate the quantitative characteristics and major subtypes of local structural connectomes for medial temporal lobe (MTL) parcellations.

Methods

The Q-Space Diffeomorphic Reconstruction (QSDR) method was used to track white matter fibers for the ROIs within MTL based on the integrating high-resolution T1 structural MR imaging and diffusion MR imaging of 100 adult Chinese individuals. Graph theoretical analysis was employed to construct the local structural connectome models for ROIs within MTL and acquire the network parameters. These connectivity matrices of these connectomes were classified into major subtypes undergoing hierarchical clustering.

Results

(1) In the local brain connectomes, the overall network features exhibited a low characteristic path length paired with moderate to high global efficiency, suggesting the effectiveness of the local brain connectome construction. The amygdala connectomes exhibited longer characteristic path length and weaker global efficiency than the ipsilateral hippocampus and parahippocampal connectomes. (2) The hubs of the amygdala connectomes were dispersed across the ventral frontal, olfactory area, limbic, parietal regions and subcortical nuclei, and the hubs the hippocampal connectomes were mainly situated within the limbic, parietal, and subcortical regions. The hubs distribution of the parahippocampal connectomes resembled the hippocampal structural connectomes, but lacking interhemispheric connections and connectivity with subcortical nuclei. (3) The subtypes of the brain local structural connectomes for each ROI were classified by hierarchical clustering, The subtypes of the bilateral amygdala connectomes were the amygdala-prefrontal connectome; the amygdala-ipsilateral or contralateral limbic connectome and the amygdala-posterior connectome. The subtypes of the bilateral hippocampal connectomes primarily included the hippocampus-ipsilateral or contralateral limbic connectome and the anterior temporal-hippocampus-ventral temporal-occipital connectome in the domain hemisphere. The subtypes of the parahippocampal connectomes exhibited resemblances to those of the hippocampus.

Conclusion

We have constructed the brain local connectomes of the MTL parcellations and acquired the network parameters to delineate the hubs distribution through graph theory analysis. The connectomes can be classified into different major subtypes, which were closely related to the functional connectivity.

Hippocampal Discoveries: Spatial View Cells, Connectivity, and Computations for Memory and Navigation, in Primates Including Humans

Two key series of discoveries about the hippocampus are described. One is the discovery of hippocampal spatial view cells in primates. This discovery opens the way to a much better understanding of human episodic memory, for episodic memory prototypically involves a memory of where people or objects or rewards have been seen in locations "out there" which could never be implemented by the place cells that encode the location of a rat or mouse. Further, spatial view cells are valuable for navigation using vision and viewed landmarks, and provide for much richer, vision-based, navigation than the place to place self-motion update performed by rats and mice who live in dark underground tunnels. Spatial view cells thus offer a revolution in our understanding of the functions of the hippocampus in memory and navigation in humans and other primates with well-developed foveate vision. The second discovery describes a computational theory of the hippocampal-neocortical memory system that includes the only quantitative theory of how information is recalled from the hippocampus to the neocortex. It is shown how foundations for this research were the discovery of reward neurons for food reward, and non-reward, in the primate orbitofrontal cortex, and representations of value including of monetary value in the human orbitofrontal cortex; and the discovery of face identity and face expression cells in the primate inferior temporal visual cortex and how they represent transform-invariant information. This research illustrates how in order to understand a brain computation, a whole series of integrated interdisciplinary discoveries is needed to build a theory of the operation of each neural system.

Neural circuits for goal-directed navigation across species

Across species, navigation is crucial for finding both resources and shelter. In vertebrates, the hippocampus supports memory-guided goal-directed navigation, whereas in arthropods the central complex supports similar functions. A growing literature is revealing similarities and differences in the organization and function of these brain regions. We review current knowledge about how each structure supports goal-directed navigation by building internal representations of the position or orientation of an animal in space, and of the location or direction of potential goals. We describe input pathways to each structure - medial and lateral entorhinal cortex in vertebrates, and columnar and tangential neurons in insects - that primarily encode spatial and non-spatial information, respectively. Finally, we highlight similarities and differences in spatial encoding across clades and suggest experimental approaches to compare coding principles and behavioral capabilities across species. Such a comparative approach can provide new insights into the neural basis of spatial navigation and neural computation.

Selective Brain Activations and Connectivities Related to the Storage and Recall of Human Object-Location, Reward-Location, and Word-Pair Episodic Memories

Different cortical systems to the hippocampus were activated using fMRI during different types of episodic memory task. For object with scene location episodic memory, the activations were high in cortical systems involved in spatial processing, including the ventromedial visual and medial parahippocampal system. These activations for the medial parahippocampal system were higher in the right hemisphere. The activations in the face and object processing ventrolateral visual cortical stream regions FFC, PIT, V8 and TE2p were higher in the object-location in scene task than the reward-location task, and were higher in the right hemisphere. For reward-location in scene episodic memory, activations were also high in the ventromedial visual cortical spatial stream to the hippocampus, but were also selectively high in storage in key reward cortical regions (ventromedial prefrontal 10r, 10v, 10d; pregenual anterior cingulate d32, p24, p32, s32; and medial orbitofrontal cortex reward-related pOFC, 11l, OFC). For word-pair episodic memory, activations were lower in the ventromedial visual and medial parahippocampal spatial cortical stream, and were higher in language-related regions in Broca's area (44, 45, 47l), and were higher in the left hemisphere for these regions and for the many highly connected inferior frontal gyrus regions in the left hemisphere. Further, effective connectivity analyses during the episodic memory tasks showed that the direction of connectivity for these systems was from early visual cortical regions V2-V4 to the ventromedial visual cortical regions VMV1-3 and VVC for spatial scene processing; was from the pregenual anterior cingulate and orbitofrontal cortex reward systems to the hippocampal system; and was from the FFC/V8/PIT system to TE2p in the visual inferior temporal visual cortex, which has connectivity to lateral parahippocampal TF, which in turn has forward effective connectivity to the hippocampus.

A preserved neural code for temporal order between memory formation and recall in the human medial temporal lobe

Temporal memory enables us to remember the temporal order of events happening in our life. The human medial temporal lobe (MTL) appears to contain neural representations supporting temporal memory formation, but the cellular mechanisms that preserve temporal order information for recall are largely unknown. Here, we examined whether human MTL neuronal activity represents the temporal position of events during memory formation and recall, using invasive single and multi-unit recordings in human epilepsy patients (n = 19). Participants freely navigated a virtual environment in order to explore and remember locations and temporal positions of objects. During each exploration period, they sequentially encountered two or three different objects, placed in different locations. This allowed us to examine single- and multi-unit neuronal firing rates (FR) as a function of the temporal position the objects were presented in. We found that a significant number of multi-units and single-units in various MTL regions including the hippocampus showed selectivity to the temporal position of objects during the exploration period. During recall, patients were asked to indicate which one of two objects from the same trial was found latter. Neural firing rates during recall showed a selectivity supporting recall of temporal positions. Interestingly, most of the selective single-units that stayed selective during encoding and recall preserved their temporal position preference. Our results thus suggest that neuronal activity in the human MTL contains a preserved neural code for temporal order in memory formation and recall.

Concept and location neurons in the human brain provide the 'what' and 'where' in memory formation

Our brains create new memories by capturing the 'who/what', 'where' and 'when' of everyday experiences. On a neuronal level, mechanisms facilitating a successful transfer into episodic memory are still unclear. We investigated this by measuring single neuron activity in the human medial temporal lobe during encoding of item-location associations. While previous research has found predictive effects in population activity in human MTL structures, we could attribute such effects to two specialized sub-groups of neurons: concept cells in the hippocampus, amygdala and entorhinal cortex (EC), and a second group of parahippocampal location-selective neurons. In both item- and location-selective populations, firing rates were significantly higher during successfully encoded trials. These findings are in line with theories of hippocampal indexing, since selective index neurons may act as pointers to neocortical representations. Overall, activation of distinct populations of neurons could directly support the connection of the 'what' and 'where' of episodic memory.

A ventromedial visual cortical 'Where' stream to the human hippocampus for spatial scenes revealed with magnetoencephalography

The primate including the human hippocampus implicated in episodic memory and navigation represents a spatial view, very different from the place representations in rodents. To understand this system in humans, and the computations performed, the pathway for this spatial view information to reach the hippocampus was analysed in humans. Whole-brain effective connectivity was measured with magnetoencephalography between 30 visual cortical regions and 150 other cortical regions using the HCP-MMP1 atlas in 21 participants while performing a 0-back scene memory task. In a ventromedial visual stream, V1-V4 connect to the ProStriate region where the retrosplenial scene area is located. The ProStriate region has connectivity to ventromedial visual regions VMV1-3 and VVC. These ventromedial regions connect to the medial parahippocampal region PHA1-3, which, with the VMV regions, include the parahippocampal scene area. The medial parahippocampal regions have effective connectivity to the entorhinal cortex, perirhinal cortex, and hippocampus. In contrast, when viewing faces, the effective connectivity was more through a ventrolateral visual cortical stream via the fusiform face cortex to the inferior temporal visual cortex regions TE2p and TE2a. A ventromedial visual cortical 'Where' stream to the hippocampus for spatial scenes was supported by diffusion topography in 171 HCP participants at 7 T.

A theory of hippocampal function: New developments

We develop further here the only quantitative theory of the storage of information in the hippocampal episodic memory system and its recall back to the neocortex. The theory is upgraded to account for a revolution in understanding of spatial representations in the primate, including human, hippocampus, that go beyond the place where the individual is located, to the location being viewed in a scene. This is fundamental to much primate episodic memory and navigation: functions supported in humans by pathways that build 'where' spatial view representations by feature combinations in a ventromedial visual cortical stream, separate from those for 'what' object and face information to the inferior temporal visual cortex, and for reward information from the orbitofrontal cortex. Key new computational developments include the capacity of the CA3 attractor network for storing whole charts of space; how the correlations inherent in self-organizing continuous spatial representations impact the storage capacity; how the CA3 network can combine continuous spatial and discrete object and reward representations; the roles of the rewards that reach the hippocampus in the later consolidation into long-term memory in part via cholinergic pathways from the orbitofrontal cortex; and new ways of analysing neocortical information storage using Potts networks.

Selective activations and functional connectivities to the sight of faces, scenes, body parts and tools in visual and non-visual cortical regions leading to the human hippocampus

Connectivity maps are now available for the 360 cortical regions in the Human Connectome Project Multimodal Parcellation atlas. Here we add function to these maps by measuring selective fMRI activations and functional connectivity increases to stationary visual stimuli of faces, scenes, body parts and tools from 956 HCP participants. Faces activate regions in the ventrolateral visual cortical stream (FFC), in the superior temporal sulcus (STS) visual stream for face and head motion; and inferior parietal visual (PGi) and somatosensory (PF) regions. Scenes activate ventromedial visual stream VMV and PHA regions in the parahippocampal scene area; medial (7m) and lateral parietal (PGp) regions; and the reward-related medial orbitofrontal cortex. Body parts activate the inferior temporal cortex object regions (TE1p, TE2p); but also visual motion regions (MT, MST, FST); and the inferior parietal visual (PGi, PGs) and somatosensory (PF) regions; and the unpleasant-related lateral orbitofrontal cortex. Tools activate an intermediate ventral stream area (VMV3, VVC, PHA3); visual motion regions (FST); somatosensory (1, 2); and auditory (A4, A5) cortical regions. The findings add function to cortical connectivity maps; and show how stationary visual stimuli activate other cortical regions related to their associations, including visual motion, somatosensory, auditory, semantic, and orbitofrontal cortex value-related, regions.

Theta-phase locking of single neurons during human spatial memory

The precise timing of single-neuron activity in relation to local field potentials may support various cognitive functions. Extensive research in rodents, along with some evidence in humans, suggests that single-neuron activity at specific phases of theta oscillations plays a crucial role in memory processes. Our fundamental understanding of such theta-phase locking in humans and its dependency on basic electrophysiological properties of the local field potential is still limited, however. Here, using single-neuron recordings in epilepsy patients performing a spatial memory task, we thus aimed at improving our understanding of factors modulating theta-phase locking in the human brain. Combining a generalized-phase approach for frequency-adaptive theta-phase estimation with time-resolved spectral parameterization, our results show that theta-phase locking is a strong and prevalent phenomenon across human medial temporal lobe regions, both during spatial memory encoding and retrieval. Neuronal theta-phase locking increased during periods of elevated theta power, when clear theta oscillations were present, and when aperiodic activity exhibited steeper slopes. Theta-phase locking was similarly strong during successful and unsuccessful memory, and most neurons activated at similar theta phases between encoding and retrieval. Some neurons changed their preferred theta phases between encoding and retrieval, in line with the idea that different memory processes are separated within the theta cycle. Together, these results help disentangle how different properties of local field potentials and memory states influence theta-phase locking of human single neurons. This contributes to a better understanding of how interactions between single neurons and local field potentials may support human spatial memory.

The memory systems of the human brain and generative artificial intelligence

Generative Artificial Intelligence foundation models (for example Generative Pre-trained Transformer - GPT - models) can generate the next token given a sequence of tokens. How can this 'generative AI' be compared with the 'real' intelligence of the human brain, when for example a human generates a whole memory in response to an incomplete retrieval cue, and then generates further prospective thoughts? Here these two types of generative intelligence, artificial in machines and real in the human brain are compared, and it is shown how when whole memories are generated by hippocampal recall in response to an incomplete retrieval cue, what the human brain computes, and how it computes it, are very different from generative AI. Key differences are the use of local associative learning rules in the hippocampal memory system, and of non-local backpropagation of error learning in AI. Indeed, it is argued that the whole operation of the human brain is performed computationally very differently to what is implemented in generative AI. Moreover, it is emphasized that the primate including human hippocampal system includes computations about spatial view and where objects and people are in scenes, whereas in rodents the emphasis is on place cells and path integration by movements between places. This comparison with generative memory and processing in the human brain has interesting implications for the further development of generative AI and for neuroscience research.

Two what, two where, visual cortical streams in humans

ROLLS, E. T. Two What, Two Where, Visual Cortical Streams in Humans. NEUROSCI BIOBEHAV REV 2024. Recent cortical connectivity investigations lead to new concepts about 'What' and 'Where' visual cortical streams in humans, and how they connect to other cortical systems. A ventrolateral 'What' visual stream leads to the inferior temporal visual cortex for object and face identity, and provides 'What' information to the hippocampal episodic memory system, the anterior temporal lobe semantic system, and the orbitofrontal cortex emotion system. A superior temporal sulcus (STS) 'What' visual stream utilising connectivity from the temporal and parietal visual cortex responds to moving objects and faces, and face expression, and connects to the orbitofrontal cortex for emotion and social behaviour. A ventromedial 'Where' visual stream builds feature combinations for scenes, and provides 'Where' inputs via the parahippocampal scene area to the hippocampal episodic memory system that are also useful for landmark-based navigation. The dorsal 'Where' visual pathway to the parietal cortex provides for actions in space, but also provides coordinate transforms to provide inputs to the parahippocampal scene area for self-motion update of locations in scenes in the dark or when the view is obscured.

Ripple-locked coactivity of stimulus-specific neurons and human associative memory

Associative memory enables the encoding and retrieval of relations between different stimuli. To better understand its neural basis, we investigated whether associative memory involves temporally correlated spiking of medial temporal lobe (MTL) neurons that exhibit stimulus-specific tuning. Using single-neuron recordings from patients with epilepsy performing an associative object-location memory task, we identified the object-specific and place-specific neurons that represented the separate elements of each memory. When patients encoded and retrieved particular memories, the relevant object-specific and place-specific neurons activated together during hippocampal ripples. This ripple-locked coactivity of stimulus-specific neurons emerged over time as the patients' associative learning progressed. Between encoding and retrieval, the ripple-locked timing of coactivity shifted, suggesting flexibility in the interaction between MTL neurons and hippocampal ripples according to behavioral demands. Our results are consistent with a cellular account of associative memory, in which hippocampal ripples coordinate the activity of specialized cellular populations to facilitate links between stimuli.

Simulation of background neuronal activity and noise in human intracranial microwire recordings

BACKGROUND

Human intracranial microwire recordings allow measurement of neuronal activity in human subjects at a fine temporal and spatial scale. The recorded extracellular potentials represent a mixture of action potentials from nearby neurons, local field potentials, and other noise sources. Signal processing of these recordings is used to separate the activity of putative single neurons from other background and noise. To better understand the separation of single neuron activity, one approach is to simulate the signals produced by neurons firing action potentials combined with background activity and noise.

NEW METHOD

This paper characterizes the background activity and noise in human intracranial microwire recordings and presents an accurate and efficient method of simulation using an infinite impulse response filter to color white noise.

RESULTS AND COMPARISON

This method reproduces the power spectral density of the background activity and noise over a frequency range of 1-5000 Hz and is over 200 times faster than previously used methods. It thus facilitates large scale studies of variation of noise sources, field potentials, and processing parameters. It performs equivalently in terms of spike sorting to simulation using white noise. Another advantage is that the simulated signals are known to arise from a pseudorandom number generator and cannot be the result of detecting simulated background spiking activity.

CONCLUSIONS

This approach provides a rapid and accurate method of simulating background noise and neural activity in human intracranial microwire recordings. It is suitable for use in large scale simulations to study spike sorting in this type of recording.

A map of spatial navigation for neuroscience

Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.

Computational cross-species views of the hippocampal formation

The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.

Single neurons in the human medial temporal lobe flexibly shift representations across spatial and memory tasks

Investigations into how individual neurons encode behavioral variables of interest have revealed specific representations in single neurons, such as place and object cells, as well as a wide range of cells with conjunctive encodings or mixed selectivity. However, as most experiments examine neural activity within individual tasks, it is currently unclear if and how neural representations change across different task contexts. Within this discussion, the medial temporal lobe is particularly salient, as it is known to be important for multiple behaviors including spatial navigation and memory, however the relationship between these functions is currently unclear. Here, to investigate how representations in single neurons vary across different task contexts in the medial temporal lobe, we collected and analyzed single-neuron activity from human participants as they completed a paired-task session consisting of a passive-viewing visual working memory and a spatial navigation and memory task. Five patients contributed 22 paired-task sessions, which were spike sorted together to allow for the same putative single neurons to be compared between the different tasks. Within each task, we replicated concept-related activations in the working memory task, as well as target-location and serial-position responsive cells in the navigation task. When comparing neuronal activity between tasks, we first established that a significant number of neurons maintained the same kind of representation, responding to stimuli presentations across tasks. Further, we found cells that changed the nature of their representation across tasks, including a significant number of cells that were stimulus responsive in the working memory task that responded to serial position in the spatial task. Overall, our results support a flexible encoding of multiple, distinct aspects of different tasks by single neurons in the human medial temporal lobe, whereby some individual neurons change the nature of their feature coding between task contexts.

Hippocampal spatial view cells for memory and navigation, and their underlying connectivity in humans

Hippocampal and parahippocampal gyrus spatial view neurons in primates respond to the spatial location being looked at. The representation is allocentric, in that the responses are to locations "out there" in the world, and are relatively invariant with respect to retinal position, eye position, head direction, and the place where the individual is located. The underlying connectivity in humans is from ventromedial visual cortical regions to the parahippocampal scene area, leading to the theory that spatial view cells are formed by combinations of overlapping feature inputs self-organized based on their closeness in space. Thus, although spatial view cells represent "where" for episodic memory and navigation, they are formed by ventral visual stream feature inputs in the parahippocampal gyrus in what is the parahippocampal scene area. A second "where" driver of spatial view cells are parietal inputs, which it is proposed provide the idiothetic update for spatial view cells, used for memory recall and navigation when the spatial view details are obscured. Inferior temporal object "what" inputs and orbitofrontal cortex reward inputs connect to the human hippocampal system, and in macaques can be associated in the hippocampus with spatial view cell "where" representations to implement episodic memory. Hippocampal spatial view cells also provide a basis for navigation to a series of viewed landmarks, with the orbitofrontal cortex reward inputs to the hippocampus providing the goals for navigation, which can then be implemented by hippocampal connectivity in humans to parietal cortex regions involved in visuomotor actions in space. The presence of foveate vision and the highly developed temporal lobe for object and scene processing in primates including humans provide a basis for hippocampal spatial view cells to be key to understanding episodic memory in the primate and human hippocampus, and the roles of this system in primate including human navigation.

Using multi-task experiments to test principles of hippocampal function

Investigations of hippocampal functions have revealed a dizzying array of findings, from lesion-based behavioral deficits, to a diverse range of characterized neural activations, to computational models of putative functionality. Across these findings, there remains an ongoing debate about the core function of the hippocampus and the generality of its representation. Researchers have debated whether the hippocampus's primary role relates to the representation of space, the neural basis of (episodic) memory, or some more general computation that generalizes across various cognitive domains. Within these different perspectives, there is much debate about the nature of feature encodings. Here, we suggest that in order to evaluate hippocampal responses-investigating, for example, whether neuronal representations are narrowly targeted to particular tasks or if they subserve domain-general purposes-a promising research strategy may be the use of multi-task experiments, or more generally switching between multiple task contexts while recording from the same neurons in a given session. We argue that this strategy-when combined with explicitly defined theoretical motivations that guide experiment design-could be a fruitful approach to better understand how hippocampal representations support different behaviors. In doing so, we briefly review key open questions in the field, as exemplified by articles in this special issue, as well as previous work using multi-task experiments, and extrapolate to consider how this strategy could be further applied to probe fundamental questions about hippocampal function.

The human posterior parietal cortex: effective connectome, and its relation to function

The effective connectivity between 21 regions in the human posterior parietal cortex, and 360 cortical regions was measured in 171 Human Connectome Project (HCP) participants using the HCP atlas, and complemented with functional connectivity and diffusion tractography. Intraparietal areas LIP, VIP, MIP, and AIP have connectivity from early cortical visual regions, and to visuomotor regions such as the frontal eye fields, consistent with functions in eye saccades and tracking. Five superior parietal area 7 regions receive from similar areas and from the intraparietal areas, but also receive somatosensory inputs and connect with premotor areas including area 6, consistent with functions in performing actions to reach for, grasp, and manipulate objects. In the anterior inferior parietal cortex, PFop, PFt, and PFcm are mainly somatosensory, and PF in addition receives visuo-motor and visual object information, and is implicated in multimodal shape and body image representations. In the posterior inferior parietal cortex, PFm and PGs combine visuo-motor, visual object, and reward input and connect with the hippocampal system. PGi in addition provides a route to motion-related superior temporal sulcus regions involved in social interactions. PGp has connectivity with intraparietal regions involved in coordinate transforms and may be involved in idiothetic update of hippocampal visual scene representations.

Single neurons in the human medial temporal lobe flexibly shift representations across spatial and memory tasks

Investigations into how individual neurons encode behavioral variables of interest have revealed specific representations in single neurons, such as place and object cells, as well as a wide range of cells with conjunctive encodings or mixed selectivity. However, as most experiments examine neural activity within individual tasks, it is currently unclear if and how neural representations change across different task contexts. Within this discussion, the medial temporal lobe is particularly salient, as it is known to be important for multiple behaviors including spatial navigation and memory, however the relationship between these functions is currently unclear. Here, to investigate how representations in single neurons vary across different task contexts in the MTL, we collected and analyzed single-neuron activity from human participants as they completed a paired-task session consisting of a passive-viewing visual working memory and a spatial navigation and memory task. Five patients contributed 22 paired-task sessions, which were spike sorted together to allow for the same putative single neurons to be compared between the different tasks. Within each task, we replicated concept-related activations in the working memory task, as well as target-location and serial-position responsive cells in the navigation task. When comparing neuronal activity between tasks, we first established that a significant number of neurons maintained the same kind of representation, responding to stimuli presentations across tasks. Further, we found cells that changed the nature of their representation across tasks, including a significant number of cells that were stimulus responsive in the working memory task that responded to serial position in the spatial task. Overall, our results support a flexible encoding of multiple, distinct aspects of different tasks by single neurons in the human MTL, whereby some individual neurons change the nature of their feature coding between task contexts.

The human posterior cingulate, retrosplenial, and medial parietal cortex effective connectome, and implications for memory and navigation

The human posterior cingulate, retrosplenial, and medial parietal cortex are involved in memory and navigation. The functional anatomy underlying these cognitive functions was investigated by measuring the effective connectivity of these Posterior Cingulate Division (PCD) regions in the Human Connectome Project-MMP1 atlas in 171 HCP participants, and complemented with functional connectivity and diffusion tractography. First, the postero-ventral parts of the PCD (31pd, 31pv, 7m, d23ab, and v23ab) have effective connectivity with the temporal pole, inferior temporal visual cortex, cortex in the superior temporal sulcus implicated in auditory and semantic processing, with the reward-related vmPFC and pregenual anterior cingulate cortex, with the inferior parietal cortex, and with the hippocampal system. This connectivity implicates it in hippocampal episodic memory, providing routes for "what," reward and semantic schema-related information to access the hippocampus. Second, the antero-dorsal parts of the PCD (especially 31a and 23d, PCV, and also RSC) have connectivity with early visual cortical areas including those that represent spatial scenes, with the superior parietal cortex, with the pregenual anterior cingulate cortex, and with the hippocampal system. This connectivity implicates it in the "where" component for hippocampal episodic memory and for spatial navigation. The dorsal-transitional-visual (DVT) and ProStriate regions where the retrosplenial scene area is located have connectivity from early visual cortical areas to the parahippocampal scene area, providing a ventromedial route for spatial scene information to reach the hippocampus. These connectivities provide important routes for "what," reward, and "where" scene-related information for human hippocampal episodic memory and navigation. The midcingulate cortex provides a route from the anterior dorsal parts of the PCD and the supracallosal part of the anterior cingulate cortex to premotor regions.

Rethinking retrosplenial cortex: Perspectives and predictions

The last decade has produced exciting new ideas about retrosplenial cortex (RSC) and its role in integrating diverse inputs. Here, we review the diversity in forms of spatial and directional tuning of RSC activity, temporal organization of RSC activity, and features of RSC interconnectivity with other brain structures. We find that RSC anatomy and dynamics are more consistent with roles in multiple sensorimotor and cognitive processes than with any isolated function. However, two more generalized categories of function may best characterize roles for RSC in complex cognitive processes: (1) shifting and relating perspectives for spatial cognition and (2) prediction and error correction for current sensory states with internal representations of the environment. Both functions likely take advantage of RSC's capacity to encode conjunctions among sensory, motor, and spatial mapping information streams. Together, these functions provide the scaffold for intelligent actions, such as navigation, perspective taking, interaction with others, and error detection.

The human orbitofrontal cortex, vmPFC, and anterior cingulate cortex effective connectome: emotion, memory, and action

The human orbitofrontal cortex, ventromedial prefrontal cortex (vmPFC), and anterior cingulate cortex are involved in reward processing and thereby in emotion but are also implicated in episodic memory. To understand these regions better, the effective connectivity between 360 cortical regions and 24 subcortical regions was measured in 172 humans from the Human Connectome Project and complemented with functional connectivity and diffusion tractography. The orbitofrontal cortex has effective connectivity from gustatory, olfactory, and temporal visual, auditory, and pole cortical areas. The orbitofrontal cortex has connectivity to the pregenual anterior and posterior cingulate cortex and hippocampal system and provides for rewards to be used in memory and navigation to goals. The orbitofrontal and pregenual anterior cortex have connectivity to the supracallosal anterior cingulate cortex, which projects to midcingulate and other premotor cortical areas and provides for action-outcome learning including limb withdrawal or flight or fight to aversive and nonreward stimuli. The lateral orbitofrontal cortex has outputs to language systems in the inferior frontal gyrus. The medial orbitofrontal cortex connects to the nucleus basalis of Meynert and the pregenual cingulate to the septum, and damage to these cortical regions may contribute to memory impairments by disrupting cholinergic influences on the neocortex and hippocampus.

Replicable patterns of causal information flow between hippocampus and prefrontal cortex during spatial navigation and spatial-verbal memory formation

Interactions between the hippocampus and prefrontal cortex (PFC) play an essential role in both human spatial navigation and episodic memory, but the underlying causal flow of information between these regions across task domains is poorly understood. Here we use intracranial EEG recordings and spectrally resolved phase transfer entropy to investigate information flow during two different virtual spatial navigation and memory encoding/recall tasks and examine replicability of information flow patterns across spatial and verbal memory domains. Information theoretic analysis revealed a higher causal information flow from hippocampus to lateral PFC than in the reverse direction. Crucially, an asymmetric pattern of information flow was observed during memory encoding and recall periods of both spatial navigation tasks. Further analyses revealed frequency specificity of interactions characterized by greater bottom-up information flow from hippocampus to PFC in delta-theta band (0.5-8 Hz); in contrast, top-down information flow from PFC to hippocampus was stronger in beta band (12-30 Hz). Bayesian analysis revealed a high degree of replicability between the two spatial navigation tasks (Bayes factor > 5.46e+3) and across tasks spanning the spatial and verbal memory domains (Bayes factor > 7.32e+8). Our findings identify a domain-independent and replicable frequency-dependent feedback loop engaged during memory formation in the human brain.

Neural Correlates of Spatial Navigation in Primate Hippocampus

The hippocampus has been extensively implicated in spatial navigation in rodents and more recently in bats. Numerous studies have revealed that various kinds of spatial information are encoded across hippocampal regions. In contrast, investigations of spatial behavioral correlates in the primate hippocampus are scarce and have been mostly limited to head-restrained subjects during virtual navigation. However, recent advances made in freely-moving primates suggest marked differences in spatial representations from rodents, albeit some similarities. Here, we review empirical studies examining the neural correlates of spatial navigation in the primate (including human) hippocampus at the levels of local field potentials and single units. The lower frequency theta oscillations are often intermittent. Single neuron responses are highly mixed and task-dependent. We also discuss neuronal selectivity in the eye and head coordinates. Finally, we propose that future studies should focus on investigating both intrinsic and extrinsic population activity and examining spatial coding properties in large-scale hippocampal-neocortical networks across tasks.

The hippocampus, ventromedial prefrontal cortex, and episodic and semantic memory

The human ventromedial prefrontal cortex (vmPFC)/anterior cingulate cortex is implicated in reward and emotion, but also in memory. It is shown how the human orbitofrontal cortex connecting with the vmPFC and anterior cingulate cortex provide a route to the hippocampus for reward and emotional value to be incorporated into episodic memory, enabling memory of where a reward was seen. It is proposed that this value component results in primarily episodic memories with some value component to be repeatedly recalled from the hippocampus so that they are more likely to become incorporated into neocortical semantic and autobiographical memories. The same orbitofrontal and anterior cingulate regions also connect in humans to the septal and basal forebrain cholinergic nuclei, thereby helping to consolidate memory, and helping to account for why damage to the vMPFC impairs memory. The human hippocampus and vmPFC thus contribute in complementary ways to forming episodic and semantic memories.

Multiple cortical visual streams in humans

The effective connectivity between 55 visual cortical regions and 360 cortical regions was measured in 171 HCP participants using the HCP-MMP atlas, and complemented with functional connectivity and diffusion tractography. A Ventrolateral Visual "What" Stream for object and face recognition projects hierarchically to the inferior temporal visual cortex, which projects to the orbitofrontal cortex for reward value and emotion, and to the hippocampal memory system. A Ventromedial Visual "Where" Stream for scene representations connects to the parahippocampal gyrus and hippocampus. An Inferior STS (superior temporal sulcus) cortex Semantic Stream receives from the Ventrolateral Visual Stream, from visual inferior parietal PGi, and from the ventromedial-prefrontal reward system and connects to language systems. A Dorsal Visual Stream connects via V2 and V3A to MT+ Complex regions (including MT and MST), which connect to intraparietal regions (including LIP, VIP and MIP) involved in visual motion and actions in space. It performs coordinate transforms for idiothetic update of Ventromedial Stream scene representations. A Superior STS cortex Semantic Stream receives visual inputs from the Inferior STS Visual Stream, PGi, and STV, and auditory inputs from A5, is activated by face expression, motion and vocalization, and is important in social behaviour, and connects to language systems.

The Effective Connectivity of the Human Hippocampal Memory System

Effective connectivity measurements in the human hippocampal memory system based on the resting-state blood oxygenation-level dependent signal were made in 172 participants in the Human Connectome Project to reveal the directionality and strength of the connectivity. A ventral "what" hippocampal stream involves the temporal lobe cortex, perirhinal and parahippocampal TF cortex, and entorhinal cortex. A dorsal "where" hippocampal stream connects parietal cortex with posterior and retrosplenial cingulate cortex, and with parahippocampal TH cortex, which, in turn, project to the presubiculum, which connects to the hippocampus. A third stream involves the orbitofrontal and ventromedial-prefrontal cortex with effective connectivity with the hippocampal, entorhinal, and perirhinal cortex. There is generally stronger forward connectivity to the hippocampus than backward. Thus separate "what," "where," and "reward" streams can converge in the hippocampus, from which back projections return to the sources. However, unlike the simple dual stream hippocampal model, there is a third stream related to reward value; there is some cross-connectivity between these systems before the hippocampus is reached; and the hippocampus has some effective connectivity with earlier stages of processing than the entorhinal cortex and presubiculum. These findings complement diffusion tractography and provide a foundation for new concepts on the operation of the human hippocampal memory system.

Predictive Representations in Hippocampal and Prefrontal Hierarchies

As we navigate the world, we use learned representations of relational structures to explore and to reach goals. Studies of how relational knowledge enables inference and planning are typically conducted in controlled small-scale settings. It remains unclear, however, how people use stored knowledge in continuously unfolding navigation (e.g., walking long distances in a city). We hypothesized that multiscale predictive representations guide naturalistic navigation in humans, and these scales are organized along posterior-anterior prefrontal and hippocampal hierarchies. We conducted model-based representational similarity analyses of neuroimaging data collected while male and female participants navigated realistically long paths in virtual reality. We tested the pattern similarity of each point, along each path, to a weighted sum of its successor points within predictive horizons of different scales. We found that anterior PFC showed the largest predictive horizons, posterior hippocampus the smallest, with the anterior hippocampus and orbitofrontal regions in between. Our findings offer novel insights into how cognitive maps support hierarchical planning at multiple scales.SIGNIFICANCE STATEMENT Whenever we navigate the world, we represent our journey at multiple horizons: from our immediate surroundings to our distal goal. How are such cognitive maps at different horizons simultaneously represented in the brain? Here, we applied a reinforcement learning-based analysis to neuroimaging data acquired while participants virtually navigated their hometown. We investigated neural patterns in the hippocampus and PFC, key cognitive map regions. We uncovered predictive representations with multiscale horizons in prefrontal and hippocampal gradients, with the longest predictive horizons in anterior PFC and the shortest in posterior hippocampus. These findings provide empirical support for the computational hypothesis that multiscale neural representations guide goal-directed navigation. This advances our understanding of hierarchical planning in everyday navigation of realistic distances.

Spatial goal coding in the hippocampal formation

The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.

Extensive cortical functional connectivity of the human hippocampal memory system

The cortical connections of the human hippocampal memory system are fundamental to understanding its operation in health and disease, especially in the context of the great development of the human cortex. The functional connectivity of the human hippocampal system was analyzed in 172 participants imaged at 7T in the Human Connectome Project. The human hippocampus has high functional connectivity not only with the entorhinal cortex, but also with areas that are more distant in the ventral 'what' stream including the perirhinal cortex and temporal cortical visual areas. Parahippocampal gyrus TF in humans has connectivity with this ventral 'what' subsystem. Correspondingly for the dorsal stream, the hippocampus has high functional connectivity not only with the presubiculum, but also with areas more distant, the medial parahippocampal cortex TH which includes the parahippocampal place or scene area, the posterior cingulate including retrosplenial cortex, and the parietal cortex. Further, there is considerable cross connectivity between the ventral and dorsal streams with the hippocampus. The findings are supported by anatomical connections, which together provide an unprecedented and quantitative overview of the extensive cortical connectivity of the human hippocampal system that goes beyond hierarchically organised and segregated pathways connecting the hippocampus and neocortex, and leads to new concepts on the operation of the hippocampal memory system in humans.

The Hippocampal Horizon: Constructing and Segmenting Experience for Episodic Memory

How do we recollect specific events that have occurred during continuous ongoing experience? There is converging evidence from non-human animals that spatially modulated cellular activity of the hippocampal formation supports the construction of ongoing events. On the other hand, recent human oriented event cognition models have outlined that our experience is segmented into discrete units, and that such segmentation can operate on shorter or longer timescales. Here, we describe a unification of how these dynamic physiological mechanisms of the hippocampus relate to ongoing externally and internally driven event segmentation, facilitating the demarcation of specific moments during experience. Our cross-species interdisciplinary approach offers a novel perspective in the way we construct and remember specific events, leading to the generation of many new hypotheses for future research.

Spatial modulation of hippocampal activity in freely moving macaques

The hippocampal formation is linked to spatial navigation, but there is little corroboration from freely moving primates with concurrent monitoring of head and gaze stances. We recorded neural activity across hippocampal regions in rhesus macaques during free foraging in an open environment while tracking their head and eye. Theta activity was intermittently present at movement onset and modulated by saccades. Many neurons were phase-locked to theta, with few showing phase precession. Most neurons encoded a mixture of spatial variables beyond place and grid tuning. Spatial representations were dominated by facing location and allocentric direction, mostly in head, rather than gaze, coordinates. Importantly, eye movements strongly modulated neural activity in all regions. These findings reveal that the macaque hippocampal formation represents three-dimensional (3D) space using a multiplexed code, with head orientation and eye movement properties being dominant during free exploration.

Neurons in the human medial temporal lobe track multiple temporal contexts during episodic memory processing

Episodic memory requires associating items with temporal context, a process for which the medial temporal lobe (MTL) is critical. This study uses recordings from 27 human subjects who were undergoing surgical intervention for intractable epilepsy. These same data were also utilized in Umbach et al. (2020). We identify 103 memory-sensitive neurons in the hippocampus and entorhinal cortex, whose firing rates predicted successful episodic memory encoding as subjects performed a verbal free recall task. These neurons exhibit important properties. First, as predicted from the temporal context model, they demonstrate reinstatement of firing patterns observed during encoding at the time of retrieval. The magnitude of reinstatement predicted the tendency of subjects to cluster retrieved memory items according to input serial position. Also, we found that spiking activity of these neurons was locked to the phase of hippocampal theta oscillations, but that the mean phase of spiking shifted between memory encoding versus retrieval. This unique observation is consistent with predictions of the “Separate Phases at Encoding And Retrieval (SPEAR)” model. Together, the properties we identify for memory-sensitive neurons characterize direct electrophysiological mechanisms for the representation of contextual information in the human MTL.

OpenMaze: An open-source toolbox for creating virtual navigation experiments

Incorporating 3D virtual environments into psychological experiments offers an innovative solution for balancing experimental control and ecological validity. Their flexible application to virtual navigation experiments, however, has been limited because accessible development tools best support only a subset of desirable task design features. We created OpenMaze, an open-source toolbox for the Unity game engine, to overcome this barrier. OpenMaze offers researchers the ability to conduct a wide range of first-person spatial navigation experiment paradigms in fully customized 3D environments. Crucially, because all experiments are defined using human-readable configuration files, our toolbox allows even those with no prior coding experience to build bespoke tasks. OpenMaze is also compatible with a variety of input devices and operating systems, broadening its possible applications. To demonstrate its advantages and limitations, we review and contrast other available software options before providing an overview of our design objectives and walking the reader through the process of building an experiment in OpenMaze.

A neural code for egocentric spatial maps in the human medial temporal lobe

Spatial navigation and memory rely on neural systems that encode places, distances, and directions in relation to the external world or relative to the navigating organism. Place, grid, and head-direction cells form key units of world-referenced, allocentric cognitive maps, but the neural basis of self-centered, egocentric representations remains poorly understood. Here, we used human single-neuron recordings during virtual spatial navigation tasks to identify neurons providing a neural code for egocentric spatial maps in the human brain. Consistent with previous observations in rodents, these neurons represented egocentric bearings toward reference points positioned throughout the environment. Egocentric bearing cells were abundant in the parahippocampal cortex and supported vectorial representations of egocentric space by also encoding distances toward reference points. Beyond navigation, the observed neurons showed activity increases during spatial and episodic memory recall, suggesting that egocentric bearing cells are not only relevant for navigation but also play a role in human memory.

A network linking scene perception and spatial memory systems in posterior cerebral cortex

The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.

Neurons including hippocampal spatial view cells, and navigation in primates including humans

A new theory is proposed of mechanisms of navigation in primates including humans in which spatial view cells found in the primate hippocampus and parahippocampal gyrus are used to guide the individual from landmark to landmark. The navigation involves approach to each landmark in turn (taxis), using spatial view cells to identify the next landmark in the sequence, and does not require a topological map. Two other cell types found in primates, whole body motion cells, and head direction cells, can be utilized in the spatial view cell navigational mechanism, but are not essential. If the landmarks become obscured, then the spatial view representations can be updated by self-motion (idiothetic) path integration using spatial coordinate transform mechanisms in the primate dorsal visual system to transform from egocentric to allocentric spatial view coordinates. A continuous attractor network or time cells or working memory is used in this approach to navigation to encode and recall the spatial view sequences involved. I also propose how navigation can be performed using a further type of neuron found in primates, allocentric-bearing-to-a-landmark neurons, in which changes of direction are made when a landmark reaches a particular allocentric bearing. This is useful if a landmark cannot be approached. The theories are made explicit in models of navigation, which are then illustrated by computer simulations. These types of navigation are contrasted with triangulation, which requires a topological map. It is proposed that the first strategy utilizing spatial view cells is used frequently in humans, and is relatively simple because primates have spatial view neurons that respond allocentrically to locations in spatial scenes. An advantage of this approach to navigation is that hippocampal spatial view neurons are also useful for episodic memory, and for imagery.

The Versatile Wayfinder: Prefrontal Contributions to Spatial Navigation

The prefrontal cortex (PFC) supports decision-making, goal tracking, and planning. Spatial navigation is a behavior that taxes these cognitive processes, yet the role of the PFC in models of navigation has been largely overlooked. In humans, activity in dorsolateral PFC (dlPFC) and ventrolateral PFC (vlPFC) during detours, reveal a role in inhibition and replanning. Dorsal anterior cingulate cortex (dACC) is implicated in planning and spontaneous internally-generated changes of route. Orbitofrontal cortex (OFC) integrates representations of the environment with the value of actions, providing a 'map' of possible decisions. In rodents, medial frontal areas interact with hippocampus during spatial decisions and switching between navigation strategies. In reviewing these advances, we provide a framework for how different prefrontal regions may contribute to different stages of navigation.