Goal-oriented representations in the human hippocampus during planning and navigation

Empowering safer socially sensitive autonomous vehicles using human-plausible cognitive encoding

Autonomous vehicles (AVs) will soon cruise our roads as a global undertaking. Beyond completing driving tasks, AVs are expected to incorporate ethical considerations into their operation. However, a critical challenge remains. When multiple road users are involved, their impacts on AV ethical decision-making are distinct yet interrelated. Current AVs lack social sensitivity in ethical decisions, failing to enable both differentiated consideration of road users and a holistic view of their collective impact. Drawing on research in AV ethics and neuroscience, we propose a scheme based on social concern and human-plausible cognitive encoding. Specifically, we first assess the individual impact that each road user poses to the AV based on risk. Then, social concern can differentiate these impacts by weighting the risks according to road user categories. Through cognitive encoding, these independent impacts are holistically encoded into a behavioral belief, which in turn supports ethical decisions that consider the collective impact of all involved parties. A total of two thousand benchmark scenarios from CommonRoad are used for evaluation. Empirical results show that our scheme enables safer and more ethical decisions, reducing overall risk by 26.3%, with a notable 22.9% decrease for vulnerable road users. In accidents, we enhance self-protection by 8.3%, improve protection for all road users by 17.6%, and significantly boost protection for vulnerable road users by 51.7%. As a human-inspired practice, this work renders AVs socially sensitive to overcome future ethical challenges in everyday driving.

Real-world goal-directed behavior reveals aberrant functional brain connectivity in children with ADHD

Functional connectomics is a popular approach to investigate the neural underpinnings of developmental disorders of which attention deficit hyperactivity disorder (ADHD) is one of the most prevalent. Nonetheless, neuronal mechanisms driving the aberrant functional connectivity resulting in ADHD symptoms remain largely unclear. Whereas resting state activity reflecting intrinsic tonic background activity is only vaguely connected to behavioral effects, naturalistic neuroscience has provided means to measure phasic brain dynamics associated with overt manifestation of the symptoms. Here we collected functional magnetic resonance imaging (fMRI) data in three experimental conditions, an active virtual reality (VR) task where the participants execute goal-directed behaviors, a passive naturalistic Video Viewing task, and a standard Resting State condition. Thirty-nine children with ADHD and thirty-seven typically developing (TD) children participated in this preregistered study. Functional connectivity was examined with network-based statistics (NBS) and graph theoretical metrics. During the naturalistic VR task, the ADHD group showed weaker task performance and stronger functional connectivity than the TD group. Group differences in functional connectivity were observed in widespread brain networks: particularly subcortical areas showed hyperconnectivity in ADHD. More restricted group differences in functional connectivity were observed during the Video Viewing, and there were no group differences in functional connectivity in the Resting State condition. These observations were consistent across NBS and graph theoretical analyses, although NBS revealed more pronounced group differences. Furthermore, during the VR task and Video Viewing, functional connectivity in TD controls was associated with task performance during the measurement, while Resting State activity in TD controls was correlated with ADHD symptoms rated over six months. We conclude that overt expression of the symptoms is correlated with aberrant brain connectivity in ADHD. Furthermore, naturalistic paradigms where clinical markers can be coupled with simultaneously occurring brain activity may further increase the interpretability of psychiatric neuroimaging findings.

Schemas, reinforcement learning and the medial prefrontal cortex

Schemas are rich and complex knowledge structures about the typical unfolding of events in a context; for example, a schema of a dinner at a restaurant. In this Perspective, we suggest that reinforcement learning (RL), a computational theory of learning the structure of the world and relevant goal-oriented behaviour, underlies schema learning. We synthesize literature about schemas and RL to offer that three RL principles might govern the learning of schemas: learning via prediction errors, constructing hierarchical knowledge using hierarchical RL, and dimensionality reduction through learning a simplified and abstract representation of the world. We then suggest that the orbitomedial prefrontal cortex is involved in both schemas and RL due to its involvement in dimensionality reduction and in guiding memory reactivation through interactions with posterior brain regions. Last, we hypothesize that the amount of dimensionality reduction might underlie gradients of involvement along the ventral-dorsal and posterior-anterior axes of the orbitomedial prefrontal cortex. More specific and detailed representations might engage the ventral and posterior parts, whereas abstraction might shift representations towards the dorsal and anterior parts of the medial prefrontal cortex.

Neural codes track prior events in a narrative and predict subsequent memory for details

Throughout our lives, we learn schemas that specify what types of events to expect in particular contexts and the temporal order in which these events usually occur. Here, our first goal was to investigate how such context-dependent temporal structures are represented in the brain during processing of temporally extended events. To accomplish this, we ran a 2-day fMRI study (N = 40) in which we exposed participants to many unique animated videos of weddings composed of sequences of rituals; each sequence originated from one of two fictional cultures (North and South), where rituals were shared across cultures, but the transition structure between these rituals differed across cultures. The results, obtained using representational similarity analysis, revealed that context-dependent temporal structure is represented in multiple ways in parallel, including distinct neural representations for the culture, for particular sequences, and for past and current events within the sequence. Our second goal was to test the hypothesis that neural schema representations scaffold memory for specific details. In keeping with this hypothesis, we found that the strength of the neural representation of the North/South schema for a particular wedding predicted subsequent episodic memory for the details of that wedding.

An experimental framework for conjoint measures of olfaction, navigation, and motion as pre-clinical biomarkers of Alzheimer's disease

Elucidating Alzheimer's disease (AD) prodromal symptoms can resolve the outstanding challenge of early diagnosis. Based on intrinsically related substrates of olfaction and spatial navigation, we propose a novel experimental framework for their conjoint study. Artificial intelligence-driven multimodal study combining self-collected olfactory and motion data with available big clinical datasets can potentially promote high-precision early clinical screenings to facilitate timely interventions targeting neurodegenerative progression.

Differences and similarities between human hippocampal low-frequency oscillations during navigation and mental simulation

Low frequency oscillations in the hippocampus emerge during by both spatial navigation and episodic memory function in humans. We have recently shown that in humans, memory-related processing is a stronger driver of low frequency oscillations than navigation. These findings and others support the idea that low-frequency oscillations are more strongly associated with a general memory function than with a specific role in spatial navigation. However, whether the low-frequency oscillations that support episodic memory and those during navigation could still share some similar functional roles remains unclear. In this study, patients undergoing intracranial electroencephalography (iEEG) monitoring performed a navigation task in which they navigated and performed internally directed route replay, similar to episodic memory. We trained a random forest classification model to use patterns in low-frequency power (2-12 Hz) to learn the position during navigation and subsequently used the same model to successfully decode position during mental simulation. We show that removal of background differences in power between navigation and mental simulation is critical to detecting the overlapping patterns. These results suggest that the low-frequency oscillations that emerge during navigation are more associated with a role in memory than specifically with a navigation related function.

Context changes retrieval of prospective outcomes during decision deliberation

Foreseeing the future outcomes is the art of decision-making. Substantial evidence shows that, during choice deliberation, the brain can retrieve prospective decision outcomes. However, decisions are seldom made in a vacuum. Context carries information that can radically affect the outcomes of a choice. Nevertheless, most investigations of retrieval processes examined decisions in isolation, disregarding the context in which they occur. Here, we studied how context shapes prospective outcome retrieval during deliberation. We designed a decision-making task where participants were presented with object-context pairs and made decisions which led to a certain outcome. We show during deliberation, likely outcomes were retrieved in transient patterns of neural activity, as early as 3 s before participants decided. The strength of prospective outcome retrieval explains participants' behavioral efficiency, but only when context affects the decision outcome. Our results suggest context imparts strong constraints on retrieval processes and how neural representations are shaped during decision-making.

Human hippocampus and dorsomedial prefrontal cortex infer and update latent causes during social interaction

Latent-cause inference is the process of identifying features of the environment that have caused an outcome. This problem is especially important in social settings where individuals may not make equal contributions to the outcomes they achieve together. Here, we designed a novel task in which participants inferred which of two characters was more likely to have been responsible for outcomes achieved by working together. Using computational modeling, univariate and multivariate analysis of human fMRI, and continuous theta-burst stimulation, we identified two brain regions that solved the task. Notably, as each outcome occurred, it was possible to decode the inference of its cause (the responsible character) from hippocampal activity. Activity in dorsomedial prefrontal cortex (dmPFC) updated estimates of association between cause-responsible character-and the outcome. Disruption of dmPFC activity impaired participants' ability to update their estimate as a function of inferred responsibility but spared their ability to infer responsibility.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

The brain hierarchically represents the past and future during multistep anticipation

Memory for temporal structure enables both planning of future events and retrospection of past events. We investigated how the brain flexibly represents extended temporal sequences into the past and future during anticipation. Participants learned sequences of environments in immersive virtual reality. Pairs of sequences had the same environments in a different order, enabling context-specific learning. During fMRI, participants anticipated upcoming environments multiple steps into the future in a given sequence. Temporal structure was represented in the hippocampus and across higher-order visual regions (1) bidirectionally, with graded representations into the past and future and (2) hierarchically, with further events into the past and future represented in successively more anterior brain regions. In hippocampus, these bidirectional representations were context-specific, and suppression of far-away environments predicted response time costs in anticipation. Together, this work sheds light on how we flexibly represent sequential structure to enable planning over multiple timescales.

A spatial transformation-based CAN model for information integration within grid cell modules

The hippocampal-entorhinal circuit is considered to play an important role in the spatial cognition of animals. However, the mechanism of the information flow within the circuit and its contribution to the function of the grid-cell module are still topics of discussion. Prevailing theories suggest that grid cells are primarily influenced by self-motion inputs from the Medial Entorhinal Cortex, with place cells serving a secondary role by contributing to the visual calibration of grid cells. However, recent evidence suggests that both self-motion inputs and visual cues may collaboratively contribute to the formation of grid-like patterns. In this paper, we introduce a novel Continuous Attractor Network model based on a spatial transformation mechanism. This mechanism enables the integration of self-motion inputs and visual cues within grid-cell modules, synergistically driving the formation of grid-like patterns. From the perspective of individual neurons within the network, our model successfully replicates grid firing patterns. From the view of neural population activity within the network, the network can form and drive the activated bump, which describes the characteristic feature of grid-cell modules, namely, path integration. Through further exploration and experimentation, our model can exhibit significant performance in path integration. This study provides a new insight into understanding the mechanism of how the self-motion and visual inputs contribute to the neural activity within grid-cell modules. Furthermore, it provides theoretical support for achieving accurate path integration, which holds substantial implications for various applications requiring spatial navigation and mapping.

Forming cognitive maps for abstract spaces: the roles of the human hippocampus and orbitofrontal cortex

How does the human brain construct cognitive maps for decision-making and inference? Here, we conduct an fMRI study on a navigation task in multidimensional abstract spaces. Using a deep neural network model, we assess learning levels and categorized paths into exploration and exploitation stages. Univariate analyses show higher activation in the bilateral hippocampus and lateral prefrontal cortex during exploration, positively associated with learning level and response accuracy. Conversely, the bilateral orbitofrontal cortex (OFC) and retrosplenial cortex show higher activation during exploitation, negatively associated with learning level and response accuracy. Representational similarity analysis show that the hippocampus, entorhinal cortex, and OFC more accurately represent destinations in exploitation than exploration stages. These findings highlight the collaboration between the medial temporal lobe and prefrontal cortex in learning abstract space structures. The hippocampus may be involved in spatial memory formation and representation, while the OFC integrates sensory information for decision-making in multidimensional abstract spaces.

A goal-centric outlook on learning

Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.