Dynamic activity patterns in the anterior temporal lobe represents object semantics

Reconstructing Spatiotemporal Trajectories of Visual Object Memories in the Human Brain

How the human brain reconstructs, step-by-step, the core elements of past experiences is still unclear. Here, we map the spatiotemporal trajectories along which visual object memories are reconstructed during associative recall. Specifically, we inquire whether retrieval reinstates feature representations in a copy-like but reversed direction with respect to the initial perceptual experience, or alternatively, this reconstruction involves format transformations and regions beyond initial perception. Participants from two cohorts studied new associations between verbs and randomly paired object images, and subsequently recalled the objects when presented with the corresponding verb cue. We first analyze multivariate fMRI patterns to map where in the brain high- and low-level object features can be decoded during perception and retrieval, showing that retrieval is dominated by conceptual features, represented in comparatively late visual and parietal areas. A separately acquired EEG dataset is then used to track the temporal evolution of the reactivated patterns using similarity-based EEG-fMRI fusion. This fusion suggests that memory reconstruction proceeds from anterior frontotemporal to posterior occipital and parietal regions, in line with a conceptual-to-perceptual gradient but only partly following the same trajectories as during perception. Specifically, a linear regression statistically confirms that the sequential activation of ventral visual stream regions is reversed between image perception and retrieval. The fusion analysis also suggests an information relay to frontoparietal areas late during retrieval. Together, the results shed light onto the temporal dynamics of memory recall and the transformations that the information undergoes between the initial experience and its later reconstruction from memory.

Semantic structures facilitate threat memory integration throughout the medial temporal lobe and medial prefrontal cortex

Emotional experiences can profoundly impact our conceptual model of the world, modifying how we represent and remember a host of information even indirectly associated with that experienced in the past. Yet, how a new emotional experience infiltrates and spreads across pre-existing semantic knowledge structures (e.g., categories) is unknown. We used a modified aversive sensory preconditioning paradigm in fMRI (n = 35) to investigate whether threat memories integrate with a pre-established category to alter the representation of the entire category. We observed selective but transient changes in the representation of conceptually related items in the amygdala, medial prefrontal cortex, and occipitotemporal cortex following threat conditioning to a simple cue (geometric shape) pre-associated with a different, but related, set of category exemplars. These representational changes persisted beyond 24 h in the hippocampus and perirhinal cortex. Reactivation of the semantic category during threat conditioning, combined with activation of the hippocampus or medial prefrontal cortex, was predictive of subsequent amygdala reactivity toward novel category members at test. This provides evidence for online integration of emotional experiences into semantic categories, which then promotes threat generalization. Behaviorally, threat conditioning by proxy selectively and retroactively enhanced recognition memory and increased the perceived typicality of the semantic category indirectly associated with threat. These findings detail a complex route through which new emotional learning generalizes by modifying semantic structures built up over time and stored in memory as conceptual knowledge.

Hierarchical organization of objects in scenes is reflected in mental representations of objects

The arrangement of objects in scenes follows certain rules ("Scene Grammar"), which we exploit to perceive and interact efficiently with our environment. We have proposed that Scene Grammar is hierarchically organized: scenes are divided into clusters of objects ("phrases", e.g., the sink phrase); within every phrase, one object ("anchor", e.g., the sink) holds strong predictions about identity and position of other objects ("local objects", e.g., a toothbrush). To investigate if this hierarchy is reflected in the mental representations of objects, we collected pairwise similarity judgments for everyday object pictures and for the corresponding words. Similarity judgments were stronger not only for object pairs appearing in the same scene, but also object pairs appearing within the same phrase of the same scene as opposed to appearing in different phrases of the same scene. Besides, object pairs with the same status in the scenes (i.e., being both anchors or both local objects) were judged as more similar than pairs of different status. Comparing effects between pictures and words, we found similar, significant impact of scene hierarchy on the organization of mental representation of objects, independent of stimulus modality. We conclude that the hierarchical structure of visual environment is incorporated into abstract, domain general mental representations of the world.

100 years after Liepmann-Lesion correlates of diminished selection and application of familiar versus novel tools

100 years ago, Liepmann highlighted the role of left ventro-dorsal lesions for impairments in conceptual (rather ventral) and motor (more dorsal) related aspects of apraxia. Many studies thereafter attributed to an extended left fronto-temporo-parietal network. Yet, to date there are only few studies that looked at apraxic performance in the selection and application of familiar versus novel tools. In the current study we applied modern voxel-based lesion-symptom mapping (VLSM) to analyze neural correlates of impaired selection and application of familiar versus novel tools. 58 left (LBD) and 51 right brain damaged (RBD) stroke patients participated in the Novel Tools Test (NTT) and the Familiar Tools Test (FTT) of the Diagnostic Instrument for Limb Apraxia (DILA-S). We further assessed performance in control tasks, namely semantic knowledge (BOSU), visuo-spatial working memory (Corsi Block Tapping) and meaningless imitation of gestures (IML). Impaired tool use was most pronounced after LBD. Our VLSM results in the LBD group suggested that selection- versus application-related aspects of praxis and semantics of familiar versus novel tool use can be behaviorally and neuro-anatomically differentiated. For impairments in familiar tool tasks, the major focus of lesion maps was rather ventral while deficiencies in novel tool tasks went along with rather dorsal lesions. Affected selection processes were linked to rather anterior lesions, while impacted application processes went along with rather posterior lesion maps. In our study, particular tool selection processes were rather specific for familiar versus novel tools. Foci for lesion overlaps of experimental and control tasks were noticed ventrally for semantic knowledge and FTT, in fronto-parietal regions for working memory and NTT, and ventro-dorsally for imitation of meaningless gestures and the application of NTT and FTT. We visualized our current interpretation within a neuroanatomical model for apraxia of tool use.

Supracategorical fear information revealed by aversively conditioning multiple categories

Fear-generalization is a critical function for survival, in which an organism extracts information from a specific instantiation of a threat (e.g., the western diamondback rattlesnake in my front yard on Sunday) and learns to fear - and accordingly respond to - pertinent higher-order information (e.g., snakes live in my yard). Previous work investigating fear-conditioning in humans has used functional magnetic resonance imaging (fMRI) to demonstrate that activity patterns representing stimuli from an aversively-conditioned category (CS+) are more similar to each other than those of a neutral category (CS-). Here we used fMRI and multiple aversively-conditioned categories to ask whether we would find only similarity increases within the CS+ categories or also similarity increases between the CS+ categories. Using representational similarity analysis, we correlated several models to activity patterns underlying different brain regions and found that, following fear-conditioning, between-category and within-category similarity increased for the CS+ categories in the insula, superior frontal gyrus (SFG), and the right temporal pole. When specifically investigating fear-generalization, these between- and within-category effects were detected in the SFG. These results advance prior pattern-based neuroimaging work by exploring the effect of aversively-conditioning multiple categories and indicate an extended role for such regions in potentially representing supracategorical information during fear-learning.