Solving the detour problem in navigation: a model of prefrontal and hippocampal interactions

London taxi drivers exploit neighbourhood boundaries for hierarchical route planning

Humans show an impressive ability to plan over complex situations and environments. A classic approach to explaining such planning has been tree-search algorithms which search through alternative state sequences for the most efficient path through states. However, this approach fails when the number of states is large due to the time to compute all possible sequences. Hierarchical route planning has been proposed as an alternative, offering a computationally efficient mechanism in which the representation of the environment is segregated into clusters. Current evidence for hierarchical planning comes from experimentally created environments which have clearly defined boundaries and far fewer states than the real-world. To test for real-world hierarchical planning we exploited the capacity of London licensed taxi drivers to use their memory to construct a street by street plan across London, UK (>26,000 streets). The time to recall each successive street name was treated as the response time, with a rapid average of 1.8 s between each street. In support of hierarchical planning we find that the clustered structure of London's regions impacts the response times, with minimal impact of the distance across the street network (as would be predicted by tree-search). We also find that changing direction during the plan (e.g. turning left or right) is associated with delayed response times. Thus, our results provide real-world evidence for how humans structure planning over a very large number of states, and give a measure of human expertise in planning.

Integration of Euclidean and path distances in hippocampal maps

The hippocampus is a key region for forming mental maps of our environment. These maps represent spatial information such as distances between landmarks. A cognitive map can allow for flexible inference of spatial relationships that have never been directly experienced before. Previous work has shown that the human hippocampus encodes distances between locations, but it is unclear how Euclidean and path distances are distinguished. In this study, participants performed an object-location task in a virtual environment. We combined functional magnetic resonance imaging with representational similarity analysis to test how Euclidean and path distances are represented in the hippocampus. We observe that hippocampal neural pattern similarity for objects scales with distance between object locations, and suggest that the hippocampus integrates Euclidean and path distances. One key characteristic of cognitive maps is their adaptive and flexible nature. We therefore subsequently modified path distances between objects using roadblocks in the environment. We found that hippocampal pattern similarity between objects adapted as a function of these changes in path distance, selectively in route learners but not in map learners. Taken together, our study supports the idea that the hippocampus creates integrative and flexible cognitive maps.

Predicting real-world navigation performance from a virtual navigation task in older adults

Virtual reality environments presented on tablets and smartphones offer a novel way of measuring navigation skill and predicting real-world navigation problems. The extent to which such virtual tests are effective at predicting navigation in older populations remains unclear. We compared the performance of 20 older participants (54-74 years old) in wayfinding tasks in a real-world environment in London, UK, and in similar tasks designed in a mobile app-based test of navigation (Sea Hero Quest). In a previous study with young participants (18-35 years old), we were able to predict navigation performance in real-world tasks in London and Paris using this mobile app. We find that for the older cohort, virtual navigation performance predicts real-world performance for medium difficulty, but not for the easy or difficult environments. Overall, our study supports the utility of using digital tests of spatial cognition in older age groups, while carefully adapting the task difficulty to the population.

Coding Dynamics of the Striatal Networks During Learning

The rat dorsomedial (DMS) and dorsolateral striatum (DLS), equivalent to caudate nucleus and putamen in primates, are required for goal-directed and habit behaviour, respectively. However, it is still unclear whether and how this functional dichotomy emerges in the course of learning. In this study, we investigated this issue by recording DMS and DLS single neuron activity in rats performing a continuous spatial alternation task, from the acquisition to optimized performance. We first applied a classical analytical approach to identify task-related activity based on the modifications of single neuron firing rate in relation to specific task events or maze trajectories. We then used an innovative approach based on Hawkes process to reconstruct a directed connectivity graph of simultaneously recorded neurons, that was used to decode animal behavior. This approach enabled us to better unravel the role of DMS and DLS neural networks across learning stages. We showed that DMS and DLS display different task-related activity throughout learning stages, and the proportion of coding neurons over time decreases in the DMS and increases in the DLS. Despite these major differences, the decoding power of both networks increases during learning. These results suggest that DMS and DLS neural networks gradually reorganize in different ways in order to progressively increase their control over the behavioral performance.

Rodent maze studies: from following simple rules to complex map learning

More than 100 years since the first maze designed for rodent research, researchers now have the choice of a variety of mazes that come in many different shapes and sizes. Still old designs get modified and new designs are introduced to fit new research questions. Yet, which maze is the most optimal to use or which training paradigm should be applied, remains up for debate. In this review, we not only provide a historical overview of maze designs and usages in rodent learning and memory research, but also discuss the possible navigational strategies the animals can use to solve each maze. Furthermore, we summarize the different phases of learning that take place when a maze is used as the experimental task. At last, we delve into how training and maze design can affect what the rodents are actually learning in a spatial task.

Individual differences in knowledge network navigation

With the rapid accumulation of online information, efficient web navigation has grown vital yet challenging. To create an easily navigable cyberspace catering to diverse demographics, understanding how people navigate differently is paramount. While previous research has unveiled individual differences in spatial navigation, such differences in knowledge space navigation remain sparse. To bridge this gap, we conducted an online experiment where participants played a navigation game on Wikipedia and completed personal information questionnaires. Our analysis shows that age negatively affects knowledge space navigation performance, while multilingualism enhances it. Under time pressure, participants' performance improves across trials and males outperform females, an effect not observed in games without time pressure. In our experiment, successful route-finding is usually not related to abilities of innovative exploration of routes. Our results underline the importance of age, multilingualism and time constraint in the knowledge space navigation.

Comparing episodic memory outcomes from walking augmented reality and stationary virtual reality encoding experiences

Episodic Memory (EM) is the neurocognitive capacity to consciously recollect personally experienced events in specific spatio-temporal contexts. Although the relevance of spatial and temporal information is widely acknowledged in the EM literature, it remains unclear whether and how EM performance and organisation is modulated by self-motion, and by motor- and visually- salient environmental features (EFs) of the encoding environment. This study examines whether and how EM is modulated by locomotion and the EFs encountered in a controlled lifelike learning route within a large-scale building. Twenty-eight healthy participants took part in a museum-tour encoding task implemented in walking Augmented Reality (AR) and stationary Virtual Reality (VR) conditions. EM performance and organisation were assessed immediately and 48-hours after trials using a Remember/Familiar recognition paradigm. Results showed a significant positive modulation effect of locomotion on distinctive EM aspects. Findings highlighted a significant performance enhancement effect of stairway-adjacent locations compared to dead-end and mid-route stimuli-presentation locations. The results of this study may serve as design criteria to facilitate neurocognitive rehabilitative interventions of EM. The underlying technological framework developed for this study represents a novel and ecologically sound method for evaluating EM processes in lifelike situations, allowing researchers a naturalistic perspective into the complex nature of EM.

Threat impairs flexible use of a cognitive map

Goal-directed behavior requires adaptive systems that respond to environmental demands. In the absence of threat (or presence of reward), individuals can explore many behavioral trajectories, effectively interrogating the environment across multiple dimensions. This leads to flexible, relational memory encoding and retrieval. In the presence of danger, motivation shifts to an imperative state characterized by a narrow focus of attention on threatening information. This impairs flexible, relational memory. We test how these motivational shifts (Murty & Adcock, 2017) affect behavioral flexibility in an ecologically valid setting. Participants learned the structure of maze-like environments and navigated to the location of objects in both safe and threatening contexts. The latter contained a predator that could 'capture' participants, leading to electric shock. After learning, the path to some objects was unpredictably blocked. forcing a detour for which one route was significantly shorter. We predicted that threat would push participants toward an imperative state, leading to less efficient and less flexible navigation. Threat caused participants to take longer paths to goal objects and less efficient detours when obstacles were encountered. Threat-related impairments in detour navigation persisted after controlling for non-detour navigation performance. and non-detour navigation was not a reliable predictor of detour navigation, This suggests a specific impairment in flexible navigation during detours, an impairment unlikely to be explained by more general processes like predator avoidance or divided attention that may be present during non-detour navigation. These results provide ecologically valid evidence that dynamic, observable threats reduce flexible use of cognitive maps to guide behavior.

Mice identify subgoal locations through an action-driven mapping process

Mammals form mental maps of the environments by exploring their surroundings. Here, we investigate which elements of exploration are important for this process. We studied mouse escape behavior, in which mice are known to memorize subgoal locations-obstacle edges-to execute efficient escape routes to shelter. To test the role of exploratory actions, we developed closed-loop neural-stimulation protocols for interrupting various actions while mice explored. We found that blocking running movements directed at obstacle edges prevented subgoal learning; however, blocking several control movements had no effect. Reinforcement learning simulations and analysis of spatial data show that artificial agents can match these results if they have a region-level spatial representation and explore with object-directed movements. We conclude that mice employ an action-driven process for integrating subgoals into a hierarchical cognitive map. These findings broaden our understanding of the cognitive toolkit that mammals use to acquire spatial knowledge.

From cognitive maps to spatial schemas

A schema refers to a structured body of prior knowledge that captures common patterns across related experiences. Schemas have been studied separately in the realms of episodic memory and spatial navigation across different species and have been grounded in theories of memory consolidation, but there has been little attempt to integrate our understanding across domains, particularly in humans. We propose that experiences during navigation with many similarly structured environments give rise to the formation of spatial schemas (for example, the expected layout of modern cities) that share properties with but are distinct from cognitive maps (for example, the memory of a modern city) and event schemas (such as expected events in a modern city) at both cognitive and neural levels. We describe earlier theoretical frameworks and empirical findings relevant to spatial schemas, along with more targeted investigations of spatial schemas in human and non-human animals. Consideration of architecture and urban analytics, including the influence of scale and regionalization, on different properties of spatial schemas may provide a powerful approach to advance our understanding of spatial schemas.

Toward an Understanding of Cognitive Mapping Ability Through Manipulations and Measurement of Schemas and Stress

Daily function depends on an ability to mentally map our environment. Environmental factors such as visibility and layout, and internal factors such as psychological stress, can challenge spatial memory and efficient navigation. Importantly, people vary dramatically in their ability to navigate flexibly and overcome such challenges. In this paper, we present an overview of "schema theory" and our view of its relevance to navigational memory research. We review several studies from our group and others, that integrate manipulations of environmental complexity and affective state in order to gain a richer understanding of the mechanisms that underlie individual differences in navigational memory. Our most recent data explicitly link such individual differences to ideas rooted in schema theory, and we discuss the potential for this work to advance our understanding of cognitive decline with aging. The data from this body of work highlight the powerful impacts of individual cognitive traits and affective states on the way people take advantage of environmental features and adopt navigational strategies.

Disarrangement and reorganization of the hippocampal functional connectivity during the spatial path adjustment of pigeons

Background

The hippocampus plays an important role to support path planning and adjustment in goal-directed spatial navigation. While we still only have limited knowledge about how do the hippocampal neural activities, especially the functional connectivity patterns, change during the spatial path adjustment. In this study, we measured the behavioural indicators and local field potentials of the pigeon (Columba livia, male and female) during a goal-directed navigational task with the detour paradigm, exploring the changing patterns of the hippocampal functional network connectivity of the bird during the spatial path learning and adjustment.

Results

Our study demonstrates that the pigeons progressively learned to solve the path adjustment task after the preferred path is blocked suddenly. Behavioural results show that both the total duration and the path lengths pigeons completed the task during the phase of adjustment are significantly longer than those during the acquisition and recovery phases. Furthermore, neural results show that hippocampal functional connectivity selectively changed during path adjustment. Specifically, we identified depressed connectivity in lower bands (delta and theta) and elevated connectivity in higher bands (slow-gamma and fast-gamma).

Conclusions

These results feature both the behavioural response and neural representation of the avian spatial cognitive learning process, suggesting that the functional disarrangement and reorganization of the connectivity in the avian hippocampus during different phases may contribute to our further understanding of the potential mechanism of path learning and adjustment.

Predictive maps in rats and humans for spatial navigation

Much of our understanding of navigation comes from the study of individual species, often with specific tasks tailored to those species. Here, we provide a novel experimental and analytic framework integrating across humans, rats, and simulated reinforcement learning (RL) agents to interrogate the dynamics of behavior during spatial navigation. We developed a novel open-field navigation task ("Tartarus maze") requiring dynamic adaptation (shortcuts and detours) to frequently changing obstructions on the path to a hidden goal. Humans and rats were remarkably similar in their trajectories. Both species showed the greatest similarity to RL agents utilizing a "successor representation," which creates a predictive map. Humans also displayed trajectory features similar to model-based RL agents, which implemented an optimal tree-search planning procedure. Our results help refine models seeking to explain mammalian navigation in dynamic environments and highlight the utility of modeling the behavior of different species to uncover the shared mechanisms that support behavior.

Predicting real world spatial disorientation in Alzheimer's disease patients using virtual reality navigation tests

Spatial navigation impairments in Alzheimer's disease (AD) have been suggested to underlie patients experiencing spatial disorientation. Though many studies have highlighted navigation impairments for AD patients in virtual reality (VR) environments, the extent to which these impairments predict a patient's risk for spatial disorientation in the real world is still poorly understood. The aims of this study were to (a) investigate the spatial navigation abilities of AD patients in VR environments as well as in a real world community setting and (b) explore whether we could predict patients at a high risk for spatial disorientation in the community based on their VR navigation. Sixteen community-dwelling AD patients and 21 age/gender matched controls were assessed on their egocentric and allocentric navigation abilities in VR environments using the Virtual Supermarket Test (VST) and Sea Hero Quest (SHQ) as well as in the community using the Detour Navigation Test (DNT). When compared to controls, AD patients exhibited impairments on the VST, SHQ, and DNT. For patients, only SHQ wayfinding distance and wayfinding duration significantly predicted composite disorientation score on the DNT (β = 0.422, p = 0.034, R² = 0.299 and β = 0.357, p = 0.046, R² = 0.27 respectively). However, these same VR measures could not reliably predict which patients were at highest risk of spatial disorientation in the community (p > 0.1). Future studies should focus on developing VR-based tests which can predict AD patients at high risk of getting spatially disorientated in the real world.

Neural network based successor representations to form cognitive maps of space and language

How does the mind organize thoughts? The hippocampal-entorhinal complex is thought to support domain-general representation and processing of structural knowledge of arbitrary state, feature and concept spaces. In particular, it enables the formation of cognitive maps, and navigation on these maps, thereby broadly contributing to cognition. It has been proposed that the concept of multi-scale successor representations provides an explanation of the underlying computations performed by place and grid cells. Here, we present a neural network based approach to learn such representations, and its application to different scenarios: a spatial exploration task based on supervised learning, a spatial navigation task based on reinforcement learning, and a non-spatial task where linguistic constructions have to be inferred by observing sample sentences. In all scenarios, the neural network correctly learns and approximates the underlying structure by building successor representations. Furthermore, the resulting neural firing patterns are strikingly similar to experimentally observed place and grid cell firing patterns. We conclude that cognitive maps and neural network-based successor representations of structured knowledge provide a promising way to overcome some of the short comings of deep learning towards artificial general intelligence.

Walking on a minefield: planning, remembering, and avoiding obstacles: preliminary findings

Travel planning (TP) is a kind of planning devoted to spatial orientation that is distinguishable from general planning (GP). It is crucial to reach a destination, since it allows to select the best route according to the environmental features (e.g., the one with little traffic or the safest). TP is also needed to avoid obstacles along the way and to put in place effective strategies to support navigation. TP involves several cognitive processes, such as visuo-spatial and topographic memory as well as other executive functions (i.e., general planning, cognitive flexibility, problem solving, and divergent thinking) and it is affected by internal factors (such as gender, cognitive strategies, age). Here, we focused on the effects of visuo-spatial (VSWM) and topographic (TWM) working memory on TP, using the Minefield Task (MFT), a new tool aimed at testing TP. We tested VSWM, TWM, GP, and TP in 44 college students. First, we checked for gender differences in all the tasks proposed and then assessed the relation among VSWM, TWM, GP, and TP. Results showed that even though gender difference could be found on TWM, GP, and TP, significative correlations emerged among TP, VSWM, and GP as well as a tendency to significance for VSWM and GP in the regression analyses. Though more evidence is needed, these results suggest that when a brand-new route is computed, GP and VSWM can be the most relevant processes, whereas topographic memory was less involved, probably because the MFT does not require to recall a route from memory. The implications of these results in clinical settings are discussed.

Environmental overlap influences goal-oriented coding of spatial sequences differently along the long-axis of hippocampus

When navigating our world we often first plan or retrieve a route to our goal, avoiding alternative paths to other destinations. Inspired by computational and animal models, we have recently demonstrated evidence that the human hippocampus supports prospective spatial coding, mediated by interactions with the prefrontal cortex. But the relationship between such signals and the need to discriminate possible routes based on their goal remains unclear. In the current study, we combined human fMRI, multi-voxel pattern analysis, and an established paradigm for contrasting memories of nonoverlapping routes with those of routes that cross paths and must be disambiguated. By classifying goal-oriented representations at the initiation of a navigational route, we demonstrate that environmental overlap modulates goal-oriented representations in the hippocampus. This modulation manifest through representational shifts from posterior to anterior components of the right hippocampus. Moreover, declines in goal-oriented decoding due to overlapping memories were predicted by the strength of the alternative memory, suggesting co-expression and competition between alternatives in the hippocampus during prospective thought. Moreover, exploratory whole-brain analyses revealed that a region of frontopolar cortex, which we have previously tied to prospective route planning, represented goal-states in new overlapping routes. Together, our findings provide insight into the influences of contextual overlap on the long-axis of the hippocampus and a broader memory and planning network that we have long-associated with such navigation tasks.

Predictive Representations in Hippocampal and Prefrontal Hierarchies

As we navigate the world, we use learned representations of relational structures to explore and to reach goals. Studies of how relational knowledge enables inference and planning are typically conducted in controlled small-scale settings. It remains unclear, however, how people use stored knowledge in continuously unfolding navigation (e.g., walking long distances in a city). We hypothesized that multiscale predictive representations guide naturalistic navigation in humans, and these scales are organized along posterior-anterior prefrontal and hippocampal hierarchies. We conducted model-based representational similarity analyses of neuroimaging data collected while male and female participants navigated realistically long paths in virtual reality. We tested the pattern similarity of each point, along each path, to a weighted sum of its successor points within predictive horizons of different scales. We found that anterior PFC showed the largest predictive horizons, posterior hippocampus the smallest, with the anterior hippocampus and orbitofrontal regions in between. Our findings offer novel insights into how cognitive maps support hierarchical planning at multiple scales.SIGNIFICANCE STATEMENT Whenever we navigate the world, we represent our journey at multiple horizons: from our immediate surroundings to our distal goal. How are such cognitive maps at different horizons simultaneously represented in the brain? Here, we applied a reinforcement learning-based analysis to neuroimaging data acquired while participants virtually navigated their hometown. We investigated neural patterns in the hippocampus and PFC, key cognitive map regions. We uncovered predictive representations with multiscale horizons in prefrontal and hippocampal gradients, with the longest predictive horizons in anterior PFC and the shortest in posterior hippocampus. These findings provide empirical support for the computational hypothesis that multiscale neural representations guide goal-directed navigation. This advances our understanding of hierarchical planning in everyday navigation of realistic distances.

Direct Approach or Detour: A Comparative Model of Inhibition and Neural Ensemble Size in Behavior Selection

Organisms must cope with different risk/reward landscapes in their ecological niche. Hence, species have evolved behavior and cognitive processes to optimally balance approach and avoidance. Navigation through space, including taking detours, appears also to be an essential element of consciousness. Such processes allow organisms to negotiate predation risk and natural geometry that obstruct foraging. One aspect of this is the ability to inhibit a direct approach toward a reward. Using an adaptation of the well-known detour paradigm in comparative psychology, but in a virtual world, we simulate how different neural configurations of inhibitive processes can yield behavior that approximates characteristics of different species. Results from simulations may help elucidate how evolutionary adaptation can shape inhibitive processing in particular and behavioral selection in general. More specifically, results indicate that both the level of inhibition that an organism can exert and the size of neural populations dedicated to inhibition contribute to successful detour navigation. According to our results, both factors help to facilitate detour behavior, but the latter (i.e., larger neural populations) appears to specifically reduce behavioral variation.

Partially overlapping spatial environments trigger reinstatement in hippocampus and schema representations in prefrontal cortex

When we remember a city that we have visited, we retrieve places related to finding our goal but also non-target locations within this environment. Yet, understanding how the human brain implements the neural computations underlying holistic retrieval remains unsolved, particularly for shared aspects of environments. Here, human participants learned and retrieved details from three partially overlapping environments while undergoing high-resolution functional magnetic resonance imaging (fMRI). Our findings show reinstatement of stores even when they are not related to a specific trial probe, providing evidence for holistic environmental retrieval. For stores shared between cities, we find evidence for pattern separation (representational orthogonalization) in hippocampal subfield CA2/3/DG and repulsion in CA1 (differentiation beyond orthogonalization). Additionally, our findings demonstrate that medial prefrontal cortex (mPFC) stores representations of the common spatial structure, termed schema, across environments. Together, our findings suggest how unique and common elements of multiple spatial environments are accessed computationally and neurally.

Mice learn multi-step routes by memorizing subgoal locations

The behavioral strategies that mammals use to learn multi-step routes are unknown. In this study, we investigated how mice navigate to shelter in response to threats when the direct path is blocked. Initially, they fled toward the shelter and negotiated obstacles using sensory cues. Within 20 min, they spontaneously adopted a subgoal strategy, initiating escapes by running directly to the obstacle's edge. Mice continued to escape in this manner even after the obstacle had been removed, indicating use of spatial memory. However, standard models of spatial learning-habitual movement repetition and internal map building-did not explain how subgoal memories formed. Instead, mice used a hybrid approach: memorizing salient locations encountered during spontaneous 'practice runs' to the shelter. This strategy was also used during a geometrically identical food-seeking task. These results suggest that subgoal memorization is a fundamental strategy by which rodents learn efficient multi-step routes in new environments.

Evidence for a gradient within the medial temporal lobes for flexible retrieval under hierarchical task rules

A fundamental question in memory research is how the hippocampus processes contextual cues to retrieve distinct mnemonic associations. Prior research has emphasized the importance of hippocampal-prefrontal interactions for context-dependent memory. Our fMRI study examined the human medial temporal lobes (MTL) and their prefrontal interactions when retrieving memories associated with hierarchically organized task contexts. Participants learned virtual object-location associations governed by subordinate and superordinate task rules, which could be independently cued to change. On each fMRI trial, participants retrieved the correct object for convergent rule and location contextual information. Results demonstrated that hippocampal activity and hippocampal-prefrontal functional interconnectivity distinguished retrieval under different levels of hierarchically organized task rules. In explicit contrast to the hippocampal tail, anterior (body and head) regions were recruited specifically for superordinate changes in the contextual hierarchy. The hippocampal body also differed in its functional connectivity with the prefrontal cortex for superordinate versus subordinate changes. Our findings demonstrate a gradient in MTL for associative retrieval under changing task rules, and advance understanding of hippocampal-prefrontal interactions that support flexible contextual memory.

Hippocampal place cells encode global location but not connectivity in a complex space

Flexible navigation relies on a cognitive map of space, thought to be implemented by hippocampal place cells: neurons that exhibit location-specific firing. In connected environments, optimal navigation requires keeping track of one's location and of the available connections between subspaces. We examined whether the dorsal CA1 place cells of rats encode environmental connectivity in four geometrically identical boxes arranged in a square. Rats moved between boxes by pushing saloon-type doors that could be locked in one or both directions. Although rats demonstrated knowledge of environmental connectivity, their place cells did not respond to connectivity changes, nor did they represent doorways differently from other locations. Place cells coded location in a global reference frame, with a different map for each box and minimal repetitive fields despite the repetitive geometry. These results suggest that CA1 place cells provide a spatial map that does not explicitly include connectivity.

Mobile brain/body imaging of landmark-based navigation with high-density EEG

Coupling behavioral measures and brain imaging in naturalistic, ecological conditions is key to comprehend the neural bases of spatial navigation. This highly integrative function encompasses sensorimotor, cognitive, and executive processes that jointly mediate active exploration and spatial learning. However, most neuroimaging approaches in humans are based on static, motion‐constrained paradigms and they do not account for all these processes, in particular multisensory integration. Following the Mobile Brain/Body Imaging approach, we aimed to explore the cortical correlates of landmark‐based navigation in actively behaving young adults, solving a Y‐maze task in immersive virtual reality. EEG analysis identified a set of brain areas matching state‐of‐the‐art brain imaging literature of landmark‐based navigation. Spatial behavior in mobile conditions additionally involved sensorimotor areas related to motor execution and proprioception usually overlooked in static fMRI paradigms. Expectedly, we located a cortical source in or near the posterior cingulate, in line with the engagement of the retrosplenial complex in spatial reorientation. Consistent with its role in visuo‐spatial processing and coding, we observed an alpha‐power desynchronization while participants gathered visual information. We also hypothesized behavior‐dependent modulations of the cortical signal during navigation. Despite finding few differences between the encoding and retrieval phases of the task, we identified transient time–frequency patterns attributed, for instance, to attentional demand, as reflected in the alpha/gamma range, or memory workload in the delta/theta range. We confirmed that combining mobile high‐density EEG and biometric measures can help unravel the brain structures and the neural modulations subtending ecological landmark‐based navigation.

The Versatile Wayfinder: Prefrontal Contributions to Spatial Navigation

The prefrontal cortex (PFC) supports decision-making, goal tracking, and planning. Spatial navigation is a behavior that taxes these cognitive processes, yet the role of the PFC in models of navigation has been largely overlooked. In humans, activity in dorsolateral PFC (dlPFC) and ventrolateral PFC (vlPFC) during detours, reveal a role in inhibition and replanning. Dorsal anterior cingulate cortex (dACC) is implicated in planning and spontaneous internally-generated changes of route. Orbitofrontal cortex (OFC) integrates representations of the environment with the value of actions, providing a 'map' of possible decisions. In rodents, medial frontal areas interact with hippocampus during spatial decisions and switching between navigation strategies. In reviewing these advances, we provide a framework for how different prefrontal regions may contribute to different stages of navigation.

Functional brain network architecture supporting the learning of social networks in humans

Most humans have the good fortune to live their lives embedded in richly structured social groups. Yet, it remains unclear how humans acquire knowledge about these social structures to successfully navigate social relationships. Here we address this knowledge gap with an interdisciplinary neuroimaging study drawing on recent advances in network science and statistical learning. Specifically, we collected BOLD MRI data while participants learned the community structure of both social and non-social networks, in order to examine whether the learning of these two types of networks was differentially associated with functional brain network topology. We found that participants learned the community structure of the networks, as evidenced by a slower reaction time when a trial moved between communities than when a trial moved within a community. Learning the community structure of social networks was also characterized by significantly greater functional connectivity of the hippocampus and temporoparietal junction when transitioning between communities than when transitioning within a community. Furthermore, temporoparietal regions of the default mode were more strongly connected to hippocampus, somatomotor, and visual regions for social networks than for non-social networks. Collectively, our results identify neurophysiological underpinnings of social versus non-social network learning, extending our knowledge about the impact of social context on learning processes. More broadly, this work offers an empirical approach to study the learning of social network structures, which could be fruitfully extended to other participant populations, various graph architectures, and a diversity of social contexts in future studies.

Stress Disrupts Human Hippocampal-Prefrontal Function during Prospective Spatial Navigation and Hinders Flexible Behavior

The ability to anticipate and flexibly plan for the future is critical for achieving goal-directed outcomes. Extant data suggest that neural and cognitive stress mechanisms may disrupt memory retrieval and restrict prospective planning, with deleterious impacts on behavior. Here, we examined whether and how acute psychological stress influences goal-directed navigational planning and efficient, flexible behavior. Our methods combined fMRI, neuroendocrinology, and machine learning with a virtual navigation planning task. Human participants were trained to navigate familiar paths in virtual environments and then (concurrent with fMRI) performed a planning and navigation task that could be most efficiently solved by taking novel shortcut paths. Strikingly, relative to non-stressed control participants, participants who performed the planning task under experimentally induced acute psychological stress demonstrated (1) disrupted neural activity critical for mnemonic retrieval and mental simulation and (2) reduced traversal of shortcuts and greater reliance on familiar paths. These neural and behavioral changes under psychological stress were tied to evidence for disrupted neural replay of memory for future locations in the spatial environment, providing mechanistic insight into why and how stress can alter planning and foster inefficient behavior.

Travel Planning Ability in Right Brain-Damaged Patients: Two Case Reports

Planning ability is fundamental for goal-directed spatial navigation. Preliminary findings from patients and healthy individuals suggest that travel planning (TP)-namely, navigational planning-can be considered a distinct process from visuospatial planning (VP) ability. To shed light on this distinction, two right brain-damaged patients without hemineglect were compared with a control group on two tasks aimed at testing VP (i.e., Tower of London-16, ToL-16) and TP (i.e., Minefield Task, MFT). The former requires planning the moves to reach the right configuration of three colored beads on three pegs, whereas the latter was opportunely developed to assess TP in the navigational environment when obstacles are present. Specifically, the MFT requires participants to plan a route on a large carpet avoiding some hidden obstacles previously observed. Patient 1 showed lesions encompassing the temporoparietal region and the insula; she performed poorer than the control group on the ToL-16 but showed no deficit on the MFT. Conversely, Patient 2 showed lesions mainly located in the occipitoparietal network of spatial navigation; she performed worse than the control group on the MFT but not on the ToL-16. In both cases performances satisfied the criteria for a classical dissociation, meeting criteria for a double dissociation. These results support the idea that TP is a distinct ability and that it is dissociated from VP skills.

Testing Navigation in Real Space: Contributions to Understanding the Physiology and Pathology of Human Navigation Control

Successful navigation relies on the flexible and appropriate use of metric representations of space or topological knowledge of the environment. Spatial dimensions (2D vs. 3D), spatial scales (vista-scale vs. large-scale environments) and the abundance of visual landmarks critically affect navigation performance and behavior in healthy human subjects. Virtual reality (VR)-based navigation paradigms in stationary position have given insight into the major navigational strategies, namely egocentric (body-centered) and allocentric (world-centered), and the cerebral control of navigation. However, VR approaches are biased towards optic flow and visual landmark processing. This major limitation can be overcome to some extent by increasingly immersive and realistic VR set-ups (including large-screen projections, eye tracking and use of head-mounted camera systems). However, the highly immersive VR settings are difficult to apply particularly to older subjects and patients with neurological disorders because of cybersickness and difficulties with learning and conducting the tasks. Therefore, a need for the development of novel spatial tasks in real space exists, which allows a synchronous analysis of navigational behavior, strategy, visual explorations and navigation-induced brain activation patterns. This review summarizes recent findings from real space navigation studies in healthy subjects and patients with different cognitive and sensory neurological disorders. Advantages and limitations of real space navigation testing and different VR-based navigation paradigms are discussed in view of potential future applications in clinical neurology.

Striatal and hippocampal contributions to flexible navigation in rats and humans

The hippocampus has been firmly established as playing a crucial role in flexible navigation. Recent evidence suggests that dorsal striatum may also play an important role in such goal-directed behaviour in both rodents and humans. Across recent studies, activity in the caudate nucleus has been linked to forward planning and adaptation to changes in the environment. In particular, several human neuroimaging studies have found the caudate nucleus tracks information traditionally associated with that by the hippocampus. In this brief review, we examine this evidence and argue the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour. We highlight that future research should explore the following: (1) Investigate neural responses during spatial navigation via a biophysically plausible framework explained by reinforcement learning models and (2) Observe the interaction between cortical areas and both the dorsal striatum and hippocampus during flexible navigation.

Age-Related Differences in Functional and Structural Connectivity in the Spatial Navigation Brain Network

Spatial navigation involves multiple cognitive processes including multisensory integration, visuospatial coding, memory, and decision-making. These functions are mediated by the interplay of cerebral structures that can be broadly separated into a posterior network (subserving visual and spatial processing) and an anterior network (dedicated to memory and navigation planning). Within these networks, areas such as the hippocampus (HC) are known to be affected by aging and to be associated with cognitive decline and navigation impairments. However, age-related changes in brain connectivity within the spatial navigation network remain to be investigated. For this purpose, we performed a neuroimaging study combining functional and structural connectivity analyses between cerebral regions involved in spatial navigation. Nineteen young (μ = 27 years, σ = 4.3; 10 F) and 22 older (μ = 73 years, σ = 4.1; 10 F) participants were examined in this study. Our analyses focused on the parahippocampal place area (PPA), the retrosplenial cortex (RSC), the occipital place area (OPA), and the projections into the visual cortex of central and peripheral visual fields, delineated from independent functional localizers. In addition, we segmented the HC and the medial prefrontal cortex (mPFC) from anatomical images. Our results show an age-related decrease in functional connectivity between low-visual areas and the HC, associated with an increase in functional connectivity between OPA and PPA in older participants compared to young subjects. Concerning the structural connectivity, we found age-related differences in white matter integrity within the navigation brain network, with the exception of the OPA. The OPA is known to be involved in egocentric navigation, as opposed to allocentric strategies which are more related to the hippocampal region. The increase in functional connectivity between the OPA and PPA may thus reflect a compensatory mechanism for the age-related alterations around the HC, favoring the use of the preserved structural network mediating egocentric navigation. Overall, these findings on age-related differences of functional and structural connectivity may help to elucidate the cerebral bases of spatial navigation deficits in healthy and pathological aging.

Prefrontal Dynamics Associated with Efficient Detours and Shortcuts: A Combined Functional Magnetic Resonance Imaging and Magnetoencenphalography Study

Central to the concept of the “cognitive map” is that it confers behavioral flexibility, allowing animals to take efficient detours, exploit shortcuts, and avoid alluring, but unhelpful, paths. The neural underpinnings of such naturalistic and flexible behavior remain unclear. In two neuroimaging experiments, we tested human participants on their ability to navigate to a set of goal locations in a virtual desert island riven by lava, which occasionally spread to block selected paths (necessitating detours) or receded to open new paths (affording real shortcuts or false shortcuts to be avoided). Detours activated a network of frontal regions compared with shortcuts. Activity in the right dorsolateral PFC specifically increased when participants encountered tempting false shortcuts that led along suboptimal paths that needed to be differentiated from real shortcuts. We also report modulation in event-related fields and theta power in these situations, providing insight to the temporal evolution of response to encountering detours and shortcuts. These results help inform current models as to how the brain supports navigation and planning in dynamic environments.

Does spatial cognitive style affect how navigational strategy is planned?

People orient themselves in the environment using three different, hierarchically organized, spatial cognitive styles: landmark, route, and survey. Landmark style is based on a representation encompassing only visual information (terrain features); route style is based on a representation that connects landmarks and routes using an egocentric (body-centred) frame of reference; survey style is based on a global map-like representation that mainly involves an allocentric (world-centred) frame of reference. This study was aimed at investigating whether individual spatial cognitive style affected the way to plan a path when searching for a lost object. Participants with landmark, route, and survey style were assessed with an ecological navigational planning task (the Key Search Task), which required planning a strategy to search for the lost key in a hypothetical wide squared field. Results showed that spatial cognitive styles were associated to different navigational planning strategies, although the time to complete the Key Search Task was comparable across the styles. As revealed by the Key Search Task score, survey style individuals were the best navigational planners, route style individuals were less efficient and landmark style individuals were the least efficient. These results suggest that spatial cognitive style has effects on navigational planning. Implications for clinical settings, such as for developmental topographical disorientation, are discussed.

Merging information in the entorhinal cortex: what can we learn from robotics experiments and modeling?

Place recognition is a complex process involving idiothetic and allothetic information. In mammals, evidence suggests that visual information stemming from the temporal and parietal cortical areas (‘what’ and ‘where’ information) is merged at the level of the entorhinal cortex (EC) to build a compact code of a place. Local views extracted from specific feature points can provide information important for view cells (in primates) and place cells (in rodents) even when the environment changes dramatically. Robotics experiments using conjunctive cells merging ‘what’ and ‘where’ information related to different local views show their important role for obtaining place cells with strong generalization capabilities. This convergence of information may also explain the formation of grid cells in the medial EC if we suppose that: (1) path integration information is computed outside the EC, (2) this information is compressed at the level of the EC owing to projection (which follows a modulo principle) of cortical activities associated with discretized vector fields representing angles and/or path integration, and (3) conjunctive cells merge the projections of different modalities to build grid cell activities. Applying modulo projection to visual information allows an interesting compression of information and could explain more recent results on grid cells related to visual exploration. In conclusion, the EC could be dedicated to the build-up of a robust yet compact code of cortical activity whereas the hippocampus proper recognizes these complex codes and learns to predict the transition from one state to another.

Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis

While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to model-based (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametrics and model-based reinforcement learning (MB-RL) to the functioning of the hippocampus (HC) and the ventral striatum (vStr)-a neuronal circuit that increasingly recognized to be an appropriate model system to understand goal-directed (spatial) decisions and planning mechanisms in the brain. We test the MB-RL agent in a contextual conditioning task that depends on intact hippocampus and ventral striatal (shell) function and show that it solves the task while showing key behavioral and neuronal signatures of the HC-vStr circuit. Our simulations also explore the benefits of biological forms of look-ahead prediction (forward sweeps) during both learning and control. This article thus contributes to fill the gap between our current understanding of computational algorithms and biological realizations of (model-based) reinforcement learning.

Spatial Representations in the Human Brain

While extensive research on the neurophysiology of spatial memory has been carried out in rodents, memory research in humans had traditionally focused on more abstract, language-based tasks. Recent studies have begun to address this gap using virtual navigation tasks in combination with electrophysiological recordings in humans. These studies suggest that the human medial temporal lobe (MTL) is equipped with a population of place and grid cells similar to that previously observed in the rodent brain. Furthermore, theta oscillations have been linked to spatial navigation and, more specifically, to the encoding and retrieval of spatial information. While some studies suggest a single navigational theta rhythm which is of lower frequency in humans than rodents, other studies advocate for the existence of two functionally distinct delta-theta frequency bands involved in both spatial and episodic memory. Despite the general consensus between rodent and human electrophysiology, behavioral work in humans does not unequivocally support the use of a metric Euclidean map for navigation. Formal models of navigational behavior, which specifically consider the spatial scale of the environment and complementary learning mechanisms, may help to better understand different navigational strategies and their neurophysiological mechanisms. Finally, the functional overlap of spatial and declarative memory in the MTL calls for a unified theory of MTL function. Such a theory will critically rely upon linking task-related phenomena at multiple temporal and spatial scales. Understanding how single cell responses relate to ongoing theta oscillations during both the encoding and retrieval of spatial and non-spatial associations appears to be key toward developing a more mechanistic understanding of memory processes in the MTL.

Coordinated prefrontal-hippocampal activity and navigation strategy-related prefrontal firing during spatial memory formation

Learning the location of relevant places in the environment is crucial for survival. Such capacity is supported by a distributed network comprising the prefrontal cortex and hippocampus, yet it is not fully understood how these structures cooperate during spatial reference memory formation. Hence, we examined neural activity in the prefrontal-hippocampal circuit in mice during acquisition of spatial reference memory. We found that interregional oscillatory coupling increased with learning, specifically in the slow-gamma frequency (20 to 40 Hz) band during spatial navigation. In addition, mice used both spatial and nonspatial strategies to navigate and solve the task, yet prefrontal neuronal spiking and oscillatory phase coupling were selectively enhanced in the spatial navigation strategy. Lastly, a representation of the behavioral goal emerged in prefrontal spiking patterns exclusively in the spatial navigation strategy. These results suggest that reference memory formation is supported by enhanced cortical connectivity and evolving prefrontal spiking representations of behavioral goals.

Neural mechanisms of navigation involving interactions of cortical and subcortical structures

Animals must perform spatial navigation for a range of different behaviors, including selection of trajectories toward goal locations and foraging for food sources. To serve this function, a number of different brain regions play a role in coding different dimensions of sensory input important for spatial behavior, including the entorhinal cortex, the retrosplenial cortex, the hippocampus, and the medial septum. This article will review data concerning the coding of the spatial aspects of animal behavior, including location of the animal within an environment, the speed of movement, the trajectory of movement, the direction of the head in the environment, and the position of barriers and objects both relative to the animal's head direction (egocentric) and relative to the layout of the environment (allocentric). The mechanisms for coding these important spatial representations are not yet fully understood but could involve mechanisms including integration of self-motion information or coding of location based on the angle of sensory features in the environment. We will review available data and theories about the mechanisms for coding of spatial representations. The computation of different aspects of spatial representation from available sensory input requires complex cortical processing mechanisms for transformation from egocentric to allocentric coordinates that will only be understood through a combination of neurophysiological studies and computational modeling.

Interacting networks of brain regions underlie human spatial navigation: a review and novel synthesis of the literature

Navigation is an inherently dynamic and multimodal process, making isolation of the unique cognitive components underlying it challenging. The assumptions of much of the literature on human spatial navigation are that 1) spatial navigation involves modality independent, discrete metric representations (i.e., egocentric vs. allocentric), 2) such representations can be further distilled to elemental cognitive processes, and 3) these cognitive processes can be ascribed to unique brain regions. We argue that modality-independent spatial representations, instead of providing exact metrics about our surrounding environment, more often involve heuristics for estimating spatial topology useful to the current task at hand. We also argue that egocentric (body centered) and allocentric (world centered) representations are better conceptualized as involving a continuum rather than as discrete. We propose a neural model to accommodate these ideas, arguing that such representations also involve a continuum of network interactions centered on retrosplenial and posterior parietal cortex, respectively. Our model thus helps explain both behavioral and neural findings otherwise difficult to account for with classic models of spatial navigation and memory, providing a testable framework for novel experiments.

Consistency and flexibility in solving spatial tasks: different horses show different cognitive styles

Individual animals vary in their behaviour and reactions to novel situations. These differences may extend to differences in cognition among individuals. We tested twenty-six horses for their ability to detour around symmetric and asymmetric obstacles. All of the animals were able to get around the barrier to reach a food target, but varied in their approach. Some horses moved slowly but were more accurate in choosing the shortest way. Other horses acted quickly, consistently detoured in the same direction, and did not reliably choose the shortest way. The remaining horses shifted from a faster, directionally consistent response with the symmetric barrier, to a slower but more accurate response with the asymmetric barrier. The asymmetric barrier induced a reduction in heart rate variability, suggesting that this is a more demanding task. The different approaches used to solve the asymmetric task may reflect distinct cognitive styles in horses, which vary among individuals, and could be linked to different personality traits. Understanding equine behaviour and cognition can inform horse welfare and management.

The cognitive map in humans: spatial navigation and beyond

The 'cognitive map' hypothesis proposes that brain builds a unified representation of the spatial environment to support memory and guide future action. Forty years of electrophysiological research in rodents suggest that cognitive maps are neurally instantiated by place, grid, border and head direction cells in the hippocampal formation and related structures. Here we review recent work that suggests a similar functional organization in the human brain and yields insights into how cognitive maps are used during spatial navigation. Specifically, these studies indicate that (i) the human hippocampus and entorhinal cortex support map-like spatial codes, (ii) posterior brain regions such as parahippocampal and retrosplenial cortices provide critical inputs that allow cognitive maps to be anchored to fixed environmental landmarks, and (iii) hippocampal and entorhinal spatial codes are used in conjunction with frontal lobe mechanisms to plan routes during navigation. We also discuss how these three basic elements of cognitive map based navigation-spatial coding, landmark anchoring and route planning-might be applied to nonspatial domains to provide the building blocks for many core elements of human thought.

Memory integration constructs maps of space, time, and concepts

Recent evidence demonstrates that new events are learned in the context of their relationships to existing memories. Within the hippocampus and medial prefrontal cortex, related memories are represented by integrated codes that connect events experienced at different times and places. Integrated codes form the basis of spatial, temporal, and conceptual maps of experience. These maps represent information that goes beyond direct experience and support generalization behaviors that require knowledge be used in new ways. The degree to which an individual memory is integrated into a coherent map is determined by its spatial, temporal, and conceptual proximity to existing knowledge. Integration is observed over a wide range of scales, suggesting that memories contain information about both broad and fine-grained contexts.

The successor representation in human reinforcement learning

Theories of reward learning in neuroscience have focused on two families of algorithms thought to capture deliberative versus habitual choice. 'Model-based' algorithms compute the value of candidate actions from scratch, whereas 'model-free' algorithms make choice more efficient but less flexible by storing pre-computed action values. We examine an intermediate algorithmic family, the successor representation, which balances flexibility and efficiency by storing partially computed action values: predictions about future events. These pre-computation strategies differ in how they update their choices following changes in a task. The successor representation's reliance on stored predictions about future states predicts a unique signature of insensitivity to changes in the task's sequence of events, but flexible adjustment following changes to rewards. We provide evidence for such differential sensitivity in two behavioural studies with humans. These results suggest that the successor representation is a computational substrate for semi-flexible choice in humans, introducing a subtler, more cognitive notion of habit.

Predictive representations can link model-based reinforcement learning to model-free mechanisms

Humans and animals are capable of evaluating actions by considering their long-run future rewards through a process described using model-based reinforcement learning (RL) algorithms. The mechanisms by which neural circuits perform the computations prescribed by model-based RL remain largely unknown; however, multiple lines of evidence suggest that neural circuits supporting model-based behavior are structurally homologous to and overlapping with those thought to carry out model-free temporal difference (TD) learning. Here, we lay out a family of approaches by which model-based computation may be built upon a core of TD learning. The foundation of this framework is the successor representation, a predictive state representation that, when combined with TD learning of value predictions, can produce a subset of the behaviors associated with model-based learning, while requiring less decision-time computation than dynamic programming. Using simulations, we delineate the precise behavioral capabilities enabled by evaluating actions using this approach, and compare them to those demonstrated by biological organisms. We then introduce two new algorithms that build upon the successor representation while progressively mitigating its limitations. Because this framework can account for the full range of observed putatively model-based behaviors while still utilizing a core TD framework, we suggest that it represents a neurally plausible family of mechanisms for model-based evaluation.

Models of spatial and temporal dimensions of memory

Episodic memory involves coding of the spatial location and time of individual events. Coding of space and time is also relevant to working memory, spatial navigation, and the disambiguation of overlapping memory representations. Neurophysiological data demonstrate that neuronal activity codes the current, past and future location of an animal as well as temporal intervals within a task. Models have addressed how neural coding of space and time for memory function could arise, with both dimensions coded by the same neurons. Neural coding could depend upon network oscillatory and attractor dynamics as well as modulation of neuronal intrinsic properties. These models are relevant to the coding of space and time involving structures including the hippocampus, entorhinal cortex, retrosplenial cortex, striatum and parahippocampal gyrus, which have been implicated in both animal and human studies.

Hippocampal and prefrontal processing of network topology to simulate the future

Topological networks lie at the heart of our cities and social milieu. However, it remains unclear how and when the brain processes topological structures to guide future behaviour during everyday life. Using fMRI in humans and a simulation of London (UK), here we show that, specifically when new streets are entered during navigation of the city, right posterior hippocampal activity indexes the change in the number of local topological connections available for future travel and right anterior hippocampal activity reflects global properties of the street entered. When forced detours require re-planning of the route to the goal, bilateral inferior lateral prefrontal activity scales with the planning demands of a breadth-first search of future paths. These results help shape models of how hippocampal and prefrontal regions support navigation, planning and future simulation.

The hippocampus is known to support navigation, but how it processes possible paths to aid navigation is unknown. Here Javadi et al. show that entering streets drives hippocampal activity corresponding to the number of future paths, and that prefrontal activity corresponds to path-planning demands.