Multidimensional memory topography in the medial parietal cortex identified from neuroimaging of thousands of daily memory videos

Visual imagery of familiar people and places in category selective cortex

Visual imagery is a dynamic process recruiting a network of brain regions. We used electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) fusion to investigate the dynamics of category selective imagery in medial parietal cortex (MPC), ventral temporal cortex (VTC), and primary visual cortex (V1). Subjects attended separate EEG and fMRI sessions where they created mental images of personally familiar people and place stimuli. The fMRI contrast comparing people and place imagery replicated previous findings of category-selectivity in the medial parietal cortex. In addition, greater activity for places was found in the ventral and lateral place memory areas, the frontal eye fields, the inferior temporal sulcus, and the intraparietal sulcus. In contrast, greater activity for people was found in the fusiform face area and the right posterior superior temporal sulcus. Using multivariate decoding analysis in fMRI, we could decode individual stimuli within the preferred category in VTC. A more complex pattern emerged in MPC, which represented information that was not restricted to the preferred category. We were also able to decode category and individual stimuli in the EEG data. EEG-fMRI fusion indicated similar timings in MPC and VTC activity during imagery. However, in the VTC, fusion was higher in place selective regions during an early time window, and higher in face selective regions in a later time window. In contrast, fusion correlations in V1 occurred later during the imagery period, possibly reflecting the top-down progression of mental imagery from category-selective regions to primary visual cortex.

Neural dynamics of semantic control underlying generative storytelling

Storytelling has been pivotal for the transmission of knowledge across human history, yet the role of semantic control and its associated neural dynamics has been poorly investigated. Here, human participants generated stories that were either appropriate (ordinary), novel (random), or balanced (creative), while recording functional magnetic resonance imaging (fMRI). Deep language models confirmed participants adherence to task instructions. At the neural level, linguistic and visual areas exhibited neural synchrony across participants regardless of the semantic control level, with parietal and frontal regions being more synchronized during random ideation. Importantly, creative stories were differentiated by a multivariate pattern of neural activity in frontal and fronto-temporo-parietal cortices compared to ordinary and random stories. Crucially, similar brain regions were also encoding the features that distinguished the stories. Moreover, we found specific spatial frequency patterns underlying the modulation of semantic control during story generation, while functional coupling in default, salience, and control networks differentiated creative stories with their controls. Remarkably, the temporal irreversibility between visual and high-level areas was higher during creative ideation, suggesting the enhanced hierarchical structure of causal interactions as a neural signature of creative storytelling. Together, our findings highlight the neural mechanisms underlying the regulation of semantic exploration during narrative ideation.

A core set of neural states underlying naturalistic memory reactivation in the posterior medial cortex

In the posterior midline default mode network, spatial activity patterns similar to those during the initial experience are reactivated during the successful recall of past events. Prior studies have shown that these event-specific activity patterns are consistent across individuals recalling a shared experience, suggesting that common functional responses underlying episodic recall do exist. However, the spatial organization of function during episodic encoding and subsequent recall, especially in the absence of external stimuli, remains poorly understood. To address this, we leverage fMRI data collected during the encoding and recall of naturalistic movies to identify a core set of neural states in the posterior medial cortex. These states are stimulus-locked, reactivated during recall, and have a shared spatial organization across brains (i.e., individuals). We show that a surprisingly small number of these states (16 states across hemispheres) is sufficient to achieve the same levels of reactivation in the posterior medial cortex as when using the standard methods of the field. Additionally, these states are linked to actions and social-affective features of events in the movies. Our findings elucidate the properties of a common, spatially organized code within the posterior default mode network which appears during natural recollection of memories.

Synergistic effects of plasma S100B and MRI measures of cerebrovascular disease on cognition in older adults

There is growing interest in studying vascular contributions to cognitive impairment and dementia (VCID) and developing biomarkers to identify at-risk individuals. A combination of biofluid and neuroimaging markers may better profile early stage VCID than individual measures. Here, we tested this possibility focusing on plasma levels of S100 calcium-binding protein B (S100B), which has been linked with blood-brain-barrier (BBB) integrity, and neuroimaging measures assessing BBB function (water exchange rate across the BBB (kw)) and cerebral small vessel disease (white matter hyperintensities (WMHs)). A total of 74 older adults without dementia had plasma samples collected and underwent cognitive assessment. A subsample had neuroimaging data including diffusion prepared pseudo-continuous arterial spin labeling (DP-pCASL) for assessment of BBB kw and T2-weighted fluid-attenuated inversion recovery (FLAIR) for quantification of WMHs. Results indicated that higher plasma S100B levels were associated with poorer episodic memory performance (β = - .031, SE = .008, p < .001). Moreover, significant interactions were observed between plasma S100B levels and parietal lobe BBB kw (interaction β = .095, SE = .042, p = .028) and between plasma S100B levels and deep WMH volume (interaction β = - .025, SE = .009, p = .007) for episodic memory. Individuals with the poorest memory performance showed both high plasma S100B and either low BBB kw in the parietal lobe or increased deep WMH burden. Taken together, our results provide support for the combined use of biofluid and neuroimaging markers in the study of VCID.

Tell me why: the missing w in episodic memory's what, where, and when

Endel Tulving defined episodic memory as consisting of a spatiotemporal context. It enables us to recollect personal experiences of people, things, places, and situations. In other words, it is made up of what, where, and when components. However, this definition does not include arguably the most important aspect of episodic memory: the why. Understanding why we remember has important implications to better understand how our memory system works and as a potential target of intervention for memory impairment. The intrinsic and extrinsic factors related to why some experiences are better remembered than others have been widely investigated but largely independently studied. How these factors interact with one another to drive an event to become a lasting memory is still unknown. This review summarizes research examining the why of episodic memory, where we aim to uncover the factors that drive core features of our memory. We discuss the concept of episodic memory examining the what, where, and when, and how the why is essential to each of these key components of episodic memory. Furthermore, we discuss the neural mechanisms known to support our rich episodic memories and how a why signal may provide critical modulatory impact on neural activity and communication. Finally, we discuss the individual differences that may further drive why we remember certain experiences over others. A better understanding of these elements, and how we experience memory in daily life, can elucidate why we remember what we remember, providing important insight into the overarching goal of our memory system.

Decoding episodic autobiographical memory in naturalistic virtual reality

Episodic autobiographical memory (EAM) is a long-term memory system of personally experienced events with their context - what, where, when - and subjective elements, e.g., emotions, thoughts, or self-reference. EAM formation has rarely been studied in a controlled, real-life-like paradigm, and there is no predictive model of long-term retrieval from self-rated subjective experience at encoding. The present longitudinal study, with three surprise free recall memory tests immediately, one-week and one-month after encoding, investigated incidental encoding of EAM in an immersive virtual environment where 30 participants either interacted with or observed specific events of varying emotional valences with simultaneous physiological recordings. The predictive analyses highlight the temporal dynamics of the predictors of EAM from subjective ratings at encoding: common characteristics related to sense of remembering and infrequency of real-life encounter of the event were identified over time, but different variables become relevant at different time points, such as the emotion and mental imagery or prospective aspects. This dynamic and time-dependent role of memory predictors challenges traditional views of a uniform influence of encoding factors over time. Current evidence for the multiphasic nature of memory formation points to the role of different mechanisms at play during encoding but also consolidation and subsequent retrieval.

Towards an ecologically valid naturalistic cognitive neuroscience of memory and event cognition

The landscape of human memory and event cognition research has witnessed a transformative journey toward the use of naturalistic contexts and tasks. In this review, we track this progression from abrupt, artificial stimuli used in extensively controlled laboratory experiments to more naturalistic tasks and stimuli that present a more faithful representation of the real world. We argue that in order to improve ecological validity, naturalistic study designs must consider the complexity of the cognitive phenomenon being studied. Then, we review the current state of "naturalistic" event segmentation studies and critically assess frequently employed movie stimuli. We evaluate recently developed tools like lifelogging and other extended reality technologies to help address the challenges we identified with existing naturalistic approaches. We conclude by offering some guidelines that can be used to design ecologically valid cognitive neuroscience studies of memory and event cognition.

The Neural Underpinnings of Aphantasia: A Case Study of Identical Twins

Aphantasia is a condition characterized by reduced voluntary mental imagery. As this lack of mental imagery disrupts visual memory, understanding the nature of this condition can provide important insight into memory, perception, and imagery. Here, we leveraged the power of case studies to better characterize this condition by running a pair of identical twins, one with aphantasia and one without, through mental imagery tasks in an fMRI scanner. We identified objective, neural measures of aphantasia, finding less visual information in their memories which may be due to lower connectivity between frontoparietal and occipitotemporal lobes of the brain. However, despite this difference, we surprisingly found more visual information in the aphantasic twin's memory than anticipated, suggesting that aphantasia is a spectrum rather than a discrete condition.

Personalized Neuroimaging Reveals the Impact of Children's Interests on Language Processing in the Brain

Cognition is shaped by individual experiences and interests. However, to study cognition in the brain, researchers typically use generic stimuli that are the same across all individuals. Language, in particular, is animated and motivated by several highly personal factors that are typically not accounted for in neuroimaging study designs, such as "interest" in a topic. Due to its inherently personal and idiosyncratic nature, it is unknown how interest in a topic modulates language processing in the brain. We conducted functional magnetic resonance imaging (fMRI) in 20 children (ages 6.98-12.01 years, mean(SD)=9.35(1.52), 5 female/15 male) as they listened to personalized narratives about a topic of specific interest, as well as to non-personalized generic narratives. We found that personalized narratives about a topic of interest increased activation in canonical language areas, as well as in reward and self-reference regions. Strikingly, we found that activation patterns elicited by topics of personal interest were more consistent across children, despite their idiosyncratic nature, than activation patterns elicited by narratives about an identical generic topic. These results reinforce the critical role that personal interests play in language processing in the human brain, and demonstrate the feasibility of using a personalized neuroimaging approach to study the effects of individually-varying factors such as interest in the brain.

Modeling short visual events through the BOLD moments video fMRI dataset and metadata

Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1000 short (3 s) naturalistic video clips of visual events across ten human subjects. We use the videos' extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception.

How does the "default mode" network contribute to semantic cognition?

This review examines whether and how the "default mode" network (DMN) contributes to semantic processing. We review evidence implicating the DMN in the processing of individual word meanings and in sentence- and discourse-level semantics. Next, we argue that the areas comprising the DMN contribute to semantic processing by coordinating and integrating the simultaneous activity of local neuronal ensembles across multiple unimodal and multimodal cortical regions, creating a transient, global neuronal ensemble. The resulting ensemble implements an integrated simulation of phenomenological experience - that is, an embodied situation model - constructed from various modalities of experiential memory traces. These situation models, we argue, are necessary not only for semantic processing but also for aspects of cognition that are not traditionally considered semantic. Although many aspects of this proposal remain provisional, we believe it provides new insights into the relationships between semantic and non-semantic cognition and into the functions of the DMN.

Individual differences in internal models explain idiosyncrasies in scene perception

According to predictive processing theories, vision is facilitated by predictions derived from our internal models of what the world should look like. However, the contents of these models and how they vary across people remains unclear. Here, we use drawing as a behavioral readout of the contents of the internal models in individual participants. Participants were first asked to draw typical versions of scene categories, as descriptors of their internal models. These drawings were converted into standardized 3d renders, which we used as stimuli in subsequent scene categorization experiments. Across two experiments, participants' scene categorization was more accurate for renders tailored to their own drawings compared to renders based on others' drawings or copies of scene photographs, suggesting that scene perception is determined by a match with idiosyncratic internal models. Using a deep neural network to computationally evaluate similarities between scene renders, we further demonstrate that graded similarity to the render based on participants' own typical drawings (and thus to their internal model) predicts categorization performance across a range of candidate scenes. Together, our results showcase the potential of a new method for understanding individual differences - starting from participants' personal expectations about the structure of real-world scenes.

Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding

Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.