Behavior-dependent directional tuning in the human visual-navigation network

Relating scene memory and perception activity to functional properties, networks, and landmarks of posterior cerebral cortex - a probabilistic atlas

Adaptive behavior in complex environments requires integrating visual perception with memory of our spatial environment. Recent work has implicated three brain areas in posterior cerebral cortex - the place memory areas (PMAs) that are anterior to the three visual scene perception areas (SPAs) - in this function. However, PMAs' relationship to the broader cortical hierarchy remains unclear due to limited group-level characterization. Here, we examined the PMA and SPA locations across three fMRI datasets (44 participants, 29 female). SPAs were identified using a standard visual localizer where participants viewed scenes versus faces. PMAs were identified by contrasting activity when participants recalled personally familiar places versus familiar faces (Datasets 1-2) or places versus multiple categories (familiar faces, bodies, and objects, and famous faces; Dataset 3). Across datasets, the PMAs were located anterior to the SPAs on the ventral and lateral cortical surfaces. The anterior displacement between PMAs and SPAs was highly reproducible. Compared to public atlases, the PMAs fell at the boundary between externally-oriented networks (dorsal attention) and internally-oriented networks (default mode). Additionally, while SPAs overlapped with retinotopic maps, the PMAs were consistently located anterior to mapped visual cortex. These results establish the anatomical position of the PMAs at inflection points along the cortical hierarchy between unimodal sensory and transmodal, apical regions, which informs broader theories of how the brain integrates perception and memory for scenes. We have released probabilistic parcels of these regions to facilitate future research into their roles in spatial cognition.

Centering cognitive neuroscience on task demands and generalization

Cognitive neuroscience seeks generalizable theories explaining the relationship between behavioral, physiological and mental states. In pursuit of such theories, we propose a theoretical and empirical framework that centers on understanding task demands and the mutual constraints they impose on behavior and neural activity. Task demands emerge from the interaction between an agent's sensory impressions, goals and behavior, which jointly shape the activity and structure of the nervous system on multiple spatiotemporal scales. Understanding this interaction requires multitask studies that vary more than one experimental component (for example, stimuli and instructions) combined with dense behavioral and neural sampling and explicit testing for generalization across tasks and data modalities. By centering task demands rather than mental processes that tasks are assumed to engage, this framework paves the way for the discovery of new generalizable concepts unconstrained by existing taxonomies, and moves cognitive neuroscience toward an action-oriented, dynamic and integrated view of the brain.

A neural compass in the human brain during naturalistic virtual navigation

A central component of wayfinding is the ability to maintain a consistent representation of one's facing direction when moving about the world. In rodents, head direction cells are believed to support this "neural compass", but identifying a similar mechanism in humans during dynamic naturalistic navigation has been challenging. To address this issue, we acquired fMRI data while participants freely navigated through a virtual reality city. Encoding model analyses revealed voxel clusters in retrosplenial complex and superior parietal lobule that exhibited reliable tuning as a function of facing direction. Crucially, these directional tunings were consistent across perceptually different versions of the city, spatially separated locations within the city, and motivationally distinct phases of the behavioral task. Analysis of the model weights indicated that these regions may represent facing direction relative to the principal axis of the environment. These findings reveal specific mechanisms in the human brain that allow us to maintain a sense of direction during naturalistic, dynamic navigation.

Electrophysiological signatures of veridical head direction in humans

Information about heading direction is critical for navigation as it provides the means to orient ourselves in space. However, given that veridical head-direction signals require physical rotation of the head and most human neuroimaging experiments depend upon fixing the head in position, little is known about how the human brain is tuned to such heading signals. Here we adress this by asking 52 healthy participants undergoing simultaneous electroencephalography and motion tracking recordings (split into two experiments) and 10 patients undergoing simultaneous intracranial electroencephalography and motion tracking recordings to complete a series of orientation tasks in which they made physical head rotations to target positions. We then used a series of forward encoding models and linear mixed-effects models to isolate electrophysiological activity that was specifically tuned to heading direction. We identified a robust posterior central signature that predicts changes in veridical head orientation after regressing out confounds including sensory input and muscular activity. Both source localization and intracranial analysis implicated the medial temporal lobe as the origin of this effect. Subsequent analyses disentangled head-direction signatures from signals relating to head rotation and those reflecting location-specific effects. Lastly, when directly comparing head direction and eye-gaze-related tuning, we found that the brain maintains both codes while actively navigating, with stronger tuning to head direction in the medial temporal lobe. Together, these results reveal a taxonomy of population-level head-direction signals within the human brain that is reminiscent of those reported in the single units of rodents.

Effects of estrogen on spatial navigation and memory

RATIONALE

Animal studies suggest that the so-called "female" hormone estrogen enhances spatial navigation and memory. This contradicts the observation that males generally out-perform females in spatial navigation and tasks involving spatial memory. A closer look at the vast number of studies actually reveals that performance differences are not so clear.

OBJECTIVES

To help clarify the unclear performance differences between men and women and the role of estrogen, we attempted to isolate organizational from activational effects of estrogen on spatial navigation and memory.

METHODS

In a double-blind, placebo-controlled study, we tested the effects of orally administered estradiol valerate (E2V) in healthy, young women in their low-hormone menstrual cycle phase, compared to healthy, young men. Participants performed several first-person, environmentally rich, 3-D computer games inspired by spatial navigation and memory paradigms in animal research.

RESULTS

We found navigation behavior suggesting that sex effects dominated any E2 effects with men performing better with allocentric strategies and women with egocentric strategies. Increased E2 levels did not lead to general improvements in spatial ability in either sex but to behavioral changes reflecting navigation flexibility.

CONCLUSION

Estrogen-driven differences in spatial cognition might be better characterized on a spectrum of navigation flexibility rather than by categorical performance measures or skills.

A different perspective on avian hippocampus function: Visual-spatial perception

The behavioral and neural mechanisms that support spatial cognition have been an enduring interest of psychologists, and much of that enduring interest is attributable to the groundbreaking research of Ken Cheng. One manifestation of this interest, inspired by the idea of studying spatial cognition under natural field conditions, has been research carried out to understand the role of the avian hippocampal formation (HF) in supporting homing pigeon navigation. Emerging from that research has been the conclusion that the role of HF in homing pigeon navigation aligns well with the canonical narrative of a hippocampus important for spatial memory and the implementation of such memories to support navigation. However, recently an accumulation of disparate observations has prompted a rethinking of the avian HF as a structure also important in shaping visual-spatial perception or attention antecedent to any memory processing. In this perspective paper, we summarize field observations contrasting the behavior of intact and HF-lesioned homing pigeons from several studies, based primarily on GPS-recorded flight paths, that support a recharacterization of HF's functional profile to include visual-spatial perception. Although admittedly still speculative, we hope the offered perspective will motivate controlled, experimental-laboratory studies to further test the hypothesis of a HF important for visual-perceptual integration, or scene construction, of landscape elements in support of navigation.

Scene Perception and Visuospatial Memory Converge at the Anterior Edge of Visually Responsive Cortex

To fluidly engage with the world, our brains must simultaneously represent both the scene in front of us and our memory of the immediate surrounding environment (i.e., local visuospatial context). How does the brain's functional architecture enable sensory and mnemonic representations to closely interface while also avoiding sensory-mnemonic interference? Here, we asked this question using first-person, head-mounted virtual reality and fMRI. Using virtual reality, human participants of both sexes learned a set of immersive, real-world visuospatial environments in which we systematically manipulated the extent of visuospatial context associated with a scene image in memory across three learning conditions, spanning from a single FOV to a city street. We used individualized, within-subject fMRI to determine which brain areas support memory of the visuospatial context associated with a scene during recall (Experiment 1) and recognition (Experiment 2). Across the whole brain, activity in three patches of cortex was modulated by the amount of known visuospatial context, each located immediately anterior to one of the three scene perception areas of high-level visual cortex. Individual subject analyses revealed that these anterior patches corresponded to three functionally defined place memory areas, which selectively respond when visually recalling personally familiar places. In addition to showing activity levels that were modulated by the amount of visuospatial context, multivariate analyses showed that these anterior areas represented the identity of the specific environment being recalled. Together, these results suggest a convergence zone for scene perception and memory of the local visuospatial context at the anterior edge of high-level visual cortex.SIGNIFICANCE STATEMENT As we move through the world, the visual scene around us is integrated with our memory of the wider visuospatial context. Here, we sought to understand how the functional architecture of the brain enables coexisting representations of the current visual scene and memory of the surrounding environment. Using a combination of immersive virtual reality and fMRI, we show that memory of visuospatial context outside the current FOV is represented in a distinct set of brain areas immediately anterior and adjacent to the perceptually oriented scene-selective areas of high-level visual cortex. This functional architecture would allow efficient interaction between immediately adjacent mnemonic and perceptual areas while also minimizing interference between mnemonic and perceptual representations.

Entorhinal grid-like codes and time-locked network dynamics track others navigating through space

Navigating through crowded, dynamically changing environments requires the ability to keep track of other individuals. Grid cells in the entorhinal cortex are a central component of self-related navigation but whether they also track others' movement is unclear. Here, we propose that entorhinal grid-like codes make an essential contribution to socio-spatial navigation. Sixty human participants underwent functional magnetic resonance imaging (fMRI) while observing and re-tracing different paths of a demonstrator that navigated a virtual reality environment. Results revealed that grid-like codes in the entorhinal cortex tracked the other individual navigating through space. The activity of grid-like codes was time-locked to increases in co-activation and entorhinal-cortical connectivity that included the striatum, the hippocampus, parahippocampal and right posterior parietal cortices. Surprisingly, the grid-related effects during observation were stronger the worse participants performed when subsequently re-tracing the demonstrator's paths. Our findings suggests that network dynamics time-locked to entorhinal grid-cell-related activity might serve to distribute information about the location of others throughout the brain.

Rethinking retrosplenial cortex: Perspectives and predictions

The last decade has produced exciting new ideas about retrosplenial cortex (RSC) and its role in integrating diverse inputs. Here, we review the diversity in forms of spatial and directional tuning of RSC activity, temporal organization of RSC activity, and features of RSC interconnectivity with other brain structures. We find that RSC anatomy and dynamics are more consistent with roles in multiple sensorimotor and cognitive processes than with any isolated function. However, two more generalized categories of function may best characterize roles for RSC in complex cognitive processes: (1) shifting and relating perspectives for spatial cognition and (2) prediction and error correction for current sensory states with internal representations of the environment. Both functions likely take advantage of RSC's capacity to encode conjunctions among sensory, motor, and spatial mapping information streams. Together, these functions provide the scaffold for intelligent actions, such as navigation, perspective taking, interaction with others, and error detection.

VR for Cognition and Memory

This chapter will provide a review of research into human cognition through the lens of VR-based paradigms for studying memory. Emphasis is placed on why VR increases the ecological validity of memory research and the implications of such enhancements.

Integrative Serum Metabolic Fingerprints Based Multi-Modal Platforms for Lung Adenocarcinoma Early Detection and Pulmonary Nodule Classification

Identification of novel non-invasive biomarkers is critical for the early diagnosis of lung adenocarcinoma (LUAD), especially for the accurate classification of pulmonary nodule. Here, a multiplexed assay is developed on an optimized nanoparticle-based laser desorption/ionization mass spectrometry platform for the sensitive and selective detection of serum metabolic fingerprints (SMFs). Integrative SMFs based multi-modal platforms are constructed for the early detection of LUAD and the classification of pulmonary nodule. The dual modal model, metabolic fingerprints with protein tumor marker neural network (MP-NN), integrating SMFs with protein tumor marker carcinoembryonic antigen (CEA) via deep learning, shows superior performance compared with the single modal model Met-NN (p < 0.001). Based on MP-NN, the tri modal model MPI-RF integrating SMFs, tumor marker CEA, and image features via random forest demonstrates significantly higher performance than the clinical models (Mayo Clinic and Veterans Affairs) and the image artificial intelligence in pulmonary nodule classification (p < 0.001). The developed platforms would be promising tools for LUAD screening and pulmonary nodule management, paving the conceptual and practical foundation for the clinical application of omics tools.

Selective neural coding of object, feature, and geometry spatial cues in humans

Orienting in space requires the processing of visual spatial cues. The dominant hypothesis about the brain structures mediating the coding of spatial cues stipulates the existence of a hippocampal-dependent system for the representation of geometry and a striatal-dependent system for the representation of landmarks. However, this dual-system hypothesis is based on paradigms that presented spatial cues conveying either conflicting or ambiguous spatial information and that used the term landmark to refer to both discrete three-dimensional objects and wall features. Here, we test the hypothesis of complex activation patterns in the hippocampus and the striatum during visual coding. We also postulate that object-based and feature-based navigation are not equivalent instances of landmark-based navigation. We examined how the neural networks associated with geometry-, object-, and feature-based spatial navigation compared with a control condition in a two-choice behavioral paradigm using fMRI. We showed that the hippocampus was involved in all three types of cue-based navigation, whereas the striatum was more strongly recruited in the presence of geometric cues than object or feature cues. We also found that unique, specific neural signatures were associated with each spatial cue. Object-based navigation elicited a widespread pattern of activity in temporal and occipital regions relative to feature-based navigation. These findings extend the current view of a dual, juxtaposed hippocampal-striatal system for visual spatial coding in humans. They also provide novel insights into the neural networks mediating object versus feature spatial coding, suggesting a need to distinguish these two types of landmarks in the context of human navigation.

Microcircuits for spatial coding in the medial entorhinal cortex

The hippocampal formation is critically involved in learning and memory and contains a large proportion of neurons encoding aspects of the organism's spatial surroundings. In the medial entorhinal cortex (MEC), this includes grid cells with their distinctive hexagonal firing fields as well as a host of other functionally defined cell types including head direction cells, speed cells, border cells, and object-vector cells. Such spatial coding emerges from the processing of external inputs by local microcircuits. However, it remains unclear exactly how local microcircuits and their dynamics within the MEC contribute to spatial discharge patterns. In this review we focus on recent investigations of intrinsic MEC connectivity, which have started to describe and quantify both excitatory and inhibitory wiring in the superficial layers of the MEC. Although the picture is far from complete, it appears that these layers contain robust recurrent connectivity that could sustain the attractor dynamics posited to underlie grid pattern formation. These findings pave the way to a deeper understanding of the mechanisms underlying spatial navigation and memory.

Task-related connectivity of decision points during spatial navigation in a schematic map

Successful navigation is largely dependent on the ability to make correct decisions at navigational decision points. However, the interaction between the brain regions associated with the navigational decision point in a schematic map is unclear. In this study, we adopted a 2D subway paradigm to study the neural basis underlying decision points. Twenty-eight subjects performed a spatial navigation task using a subway map during fMRI scanning. We adopted a voxel-wise general linear model (GLM) approach and found four brain regions, the left hippocampus (HIP), left parahippocampal gyrus (PHG), left ventromedial prefrontal cortex (vmPFC), and right retrosplenial cortex (RSC), activated at a navigational decision point in a schematic map. Using a psychophysiological interactions (PPI) method, we found that (1) both the left vmPFC and right HIP interacted cooperatively with the right RSC, and (2) the left HIP and the left vmPFC interacted cooperatively at the decision point. These findings may be helpful for revealing the neural mechanisms underlying decision points in a schematic map during spatial navigation.

BrainIAK: The Brain Imaging Analysis Kit

Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEMs), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high-performance compute (HPC) clusters, and the same code can be se amlessly transferred from a laptop to a cluster. For each of the aforementioned techniques, we describe the data analysis problem that the technique is meant to solve and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.

The neural representation of absolute direction during mental navigation in conceptual spaces

When humans mentally “navigate” bidimensional uniform conceptual spaces, they recruit the same grid-like and distance codes typically evoked when exploring the physical environment. Here, using fMRI, we show evidence that conceptual navigation also elicits another kind of spatial code: that of absolute direction. This code is mostly localized in the medial parietal cortex, where its strength predicts participants’ comparative semantic judgments. It may provide a complementary mechanism for conceptual navigation outside the hippocampal formation.

Viganò et al. use fMRI in healthy human participants to show that conceptual navigation elicits a spatial code for absolute direction in the medial parietal cortex. Their findings are suggestive of a complementary mechanism for conceptual navigation outside the hippocampal formation.

Ultra-Fast Label-Free Serum Metabolic Diagnosis of Coronary Heart Disease via a Deep Stabilizer

Although mass spectrometry (MS) of metabolites has the potential to provide real-time monitoring of patient status for diagnostic purposes, the diagnostic application of MS is limited due to sample treatment and data quality/reproducibility. Here, the generation of a deep stabilizer for ultra-fast, label-free MS detection and the application of this method for serum metabolic diagnosis of coronary heart disease (CHD) are reported. Nanoparticle-assisted laser desorption/ionization-MS is used to achieve direct metabolic analysis of trace unprocessed serum in seconds. Furthermore, a deep stabilizer is constructed to map native MS results to high-quality results obtained by established methods. Finally, using the newly developed protocol and diagnosis variation characteristic surface to characterize sensitivity/specificity and variation, CHD is diagnosed with advanced accuracy in a high-throughput/speed manner. This work advances design of metabolic analysis tools for disease detection as it provides a direct label-free, ultra-fast, and stabilized platform for future protocol development in clinics.

Emotional representations of space vary as a function of peoples' affect and interoceptive sensibility

Most research on people's representation of space has focused on spatial appraisal and navigation. But there is more to space besides navigation and assessment: people have different emotional experiences at different places, which create emotionally tinged representations of space. Little is known about the emotional representation of space and the factors that shape it. The purpose of this study was to develop a graphic methodology to study the emotional representation of space and some of the environmental features (non-natural vs. natural) and personal features (affective state and interoceptive sensibility) that modulate it. We gave participants blank maps of the region where they lived and asked them to apply shade where they had happy/sad memories, and where they wanted to go after Covid-19 lockdown. Participants also completed self-reports on affective state and interoceptive sensibility. By adapting methods for analyzing neuroimaging data, we examined shaded pixels to quantify where and how strong emotions are represented in space. The results revealed that happy memories were consistently associated with similar spatial locations. Yet, this mapping response varied as a function of participants' affective state and interoceptive sensibility. Certain regions were associated with happier memories in participants whose affective state was more positive and interoceptive sensibility was higher. The maps of happy memories, desired locations to visit after lockdown, and regions where participants recalled happier memories as a function of positive affect and interoceptive sensibility overlayed significantly with natural environments. These results suggest that people's emotional representations of their environment are shaped by the naturalness of places, and by their affective state and interoceptive sensibility.

A network linking scene perception and spatial memory systems in posterior cerebral cortex

The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.

Plasmonic Alloys Reveal a Distinct Metabolic Phenotype of Early Gastric Cancer

Gastric cancer (GC) is a multifactorial process, accompanied by alterations in metabolic pathways. Non-invasive metabolic profiling facilitates GC diagnosis at early stage leading to an improved prognostic outcome. Herein, mesoporous PdPtAu alloys are designed to characterize the metabolic profiles in human blood. The elemental composition is optimized with heterogeneous surface plasmonic resonance, offering preferred charge transfer for photoinduced desorption/ionization and enhanced photothermal conversion for thermally driven desorption. The surface structure of PdPtAu is further tuned with controlled mesopores, accommodating metabolites only, rather than large interfering compounds. Consequently, the optimized PdPtAu alloy yields direct metabolic fingerprints by laser desorption/ionization mass spectrometry in seconds, consuming 500 nL of native plasma. A distinct metabolic phenotype is revealed for early GC by sparse learning, resulting in precise GC diagnosis with an area under the curve of 0.942. It is envisioned that the plasmonic alloy will open up a new era of minimally invasive blood analysis to improve the surveillance of cancer patients in the clinical setting.

Allocentric representation in the human amygdala and ventral visual stream

The hippocampus and the entorhinal cortex are considered the main brain structures for allocentric representation of the external environment. Here, we show that the amygdala and the ventral visual stream are involved in allocentric representation. Thirty-one young men explored 35 virtual environments during high-resolution functional magnetic resonance imaging (fMRI) of the medial temporal lobe (MTL) and were subsequently tested on recall of the allocentric pattern of the objects in each environment-in other words, the positions of the objects relative to each other and to the outer perimeter. We find increasingly unique brain activation patterns associated with increasing allocentric accuracy in distinct neural populations in the perirhinal cortex, parahippocampal cortex, fusiform cortex, amygdala, hippocampus, and entorhinal cortex. In contrast to the traditional view of a hierarchical MTL network with the hippocampus at the top, we demonstrate, using recently developed graph analyses, a hierarchical allocentric MTL network without a main connector hub.

Rapid Computer-Aided Diagnosis of Stroke by Serum Metabolic Fingerprint Based Multi-Modal Recognition

Stroke is a leading cause of mortality and disability worldwide, expected to result in 61 million disability-adjusted life-years in 2020. Rapid diagnostics is the core of stroke management for early prevention and medical treatment. Serum metabolic fingerprints (SMFs) reflect underlying disease progression, predictive of patient phenotypes. Deep learning (DL) encoding SMFs with clinical indexes outperforms single biomarkers, while posing challenges with poor prediction to interpret by feature selection. Herein, rapid computer-aided diagnosis of stroke is performed using SMF based multi-modal recognition by DL, to combine adaptive machine learning with a novel feature selection approach. SMFs are extracted by nano-assisted laser desorption/ionization mass spectrometry (LDI MS), consuming 100 nL of serum in seconds. A multi-modal recognition is constructed by integrating SMFs and clinical indexes with an enhanced area under curve (AUC) up to 0.845 for stroke screening, compared to single-modal diagnosis by only SMFs or clinical indexes. The prediction of DL is addressed by selecting 20 key metabolite features with differential regulation through a saliency map approach, shedding light on the molecular mechanisms in stroke. The approach highlights the emerging role of DL in precision medicine and suggests an expanding utility for computational analysis of SMFs in stroke screening.

Current Promises and Limitations of Combined Virtual Reality and Functional Magnetic Resonance Imaging Research in Humans: A Commentary on Huffman and Ekstrom (2019)

Real-world navigation requires movement of the body through space, producing a continuous stream of visual and self-motion signals, including proprioceptive, vestibular, and motor efference cues. These multimodal cues are integrated to form a spatial cognitive map, an abstract, amodal representation of the environment. How the brain combines these disparate inputs and the relative importance of these inputs to cognitive map formation and recall are key unresolved questions in cognitive neuroscience. Recent advances in virtual reality technology allow participants to experience body-based cues when virtually navigating, and thus it is now possible to consider these issues in new detail. Here, we discuss a recent publication that addresses some of these issues (D. J. Huffman and A. D. Ekstrom. A modality-independent network underlies the retrieval of large-scale spatial environments in the human brain. Neuron, 104, 611-622, 2019). In doing so, we also review recent progress in the study of human spatial cognition and raise several questions that might be addressed in future studies.