Creatures great and small: Real-world size of animals predicts visual cortex representations beyond taxonomic category

Novel approaches to functional lateralization: Assessing information in activity patterns across hemispheres and more accurately identifying structural homologues

Functional lateralization is typically measured by comparing activation levels across the right and left hemispheres of the brain. Significant additional information, however, exists within distributed multi-voxel patterns of activity - a format not detectable by traditional activation-based analysis of functional magnetic resonance imaging (fMRI) data. We introduce and test two methods -one anatomical, one functional- that allow hemispheric information asymmetries to be detected. We first introduce and apply a novel tool that draws on brain 'surface fingerprints' to pair every location in one hemisphere with its hemispheric homologue. We use anatomical data to show that this approach is more accurate than the common distance-from-midline method for comparing bilateral regions. Next, we introduce a complementary analysis method that quantifies multivariate laterality in functional data. This new 'multivariate Laterality Index' (mLI) reflects both quantitative and qualitative information-differences across homologous activity patterns. We apply the technique here to functional data collected as participants viewed faces and non-faces. Using the previously generated surface fingerprints to pair-up homologous searchlights in each hemisphere, we use the novel multivariate laterality technique to identify face-information asymmetries across right and left counterparts of the fusiform gyrus, inferior temporal gyrus, superior parietal lobule, and early visual areas. The typical location of the fusiform face area has greater information asymmetry for faces than for shapes. More generally, we argue that the field should consider an information-based approach to lateralization.

The structure of prior knowledge enhances memory in experts by reducing interference

The influence of prior knowledge on memory is ubiquitous, making the specific mechanisms of this relationship difficult to disentangle. Here, we show that expert knowledge produces a fundamental shift in the way that interitem similarity (i.e., the perceived resemblance between items in a set) biases episodic recognition. Within a group of expert birdwatchers and matched controls, we characterized the psychological similarity space for a set of well-known local species and a set of less familiar, nonlocal species. In experts, interitem similarity was influenced most strongly by taxonomic features, whereas in controls, similarity judgments reflected bird color. In controls, perceived episodic oldness during a recognition memory task increased along with measures of global similarity between items, consistent with classic models of episodic recognition. Surprisingly, for experts, high global similarity did not drive oldness signals. Instead, for local birds memory tracked the availability of species-level name knowledge, whereas for nonlocal birds, it was mediated by the organization of generalized conceptual space. These findings demonstrate that episodic memory in experts can benefit from detailed subcategory knowledge, or, lacking that, from the overall relational structure of concepts. Expertise reshapes psychological similarity space, helping to resolve mnemonic separation challenges arising from high interitem overlap. Thus, even in the absence of knowledge about item-specific details or labels, the presence of generalized knowledge appears to support episodic recognition in domains of expertise by altering the typical relationship between psychological similarity and memory.

Grid-like and distance codes for representing word meaning in the human brain

Relational information about items in memory is thought to be represented in our brain thanks to an internal comprehensive model, also referred to as a "cognitive map". In the human neuroimaging literature, two signatures of bi-dimensional cognitive maps have been reported: the grid-like code and the distance-dependent code. While these kinds of representation were previously observed during spatial navigation and, more recently, during processing of perceptual stimuli, it is still an open question whether they also underlie the representation of the most basic items of language: words. Here we taught human participants the meaning of novel words as arbitrary labels for a set of audiovisual objects varying orthogonally in size and sound. The novel words were therefore conceivable as points in a navigable 2D map of meaning. While subjects performed a word comparison task, we recorded their brain activity using functional magnetic resonance imaging (fMRI). By applying a combination of representational similarity and fMRI-adaptation analyses, we found evidence of (i) a grid-like code, in the right postero-medial entorhinal cortex, representing the relative angular positions of words in the word space, and (ii) a distance-dependent code, in medial prefrontal, orbitofrontal, and mid-cingulate cortices, representing the Euclidean distance between words. Additionally, we found evidence that the brain also separately represents the single dimensions of word meaning: their implied size, encoded in visual areas, and their implied sound, in Heschl's gyrus/Insula. These results support the idea that the meaning of words, when they are organized along two dimensions, is represented in the human brain across multiple maps of different dimensionality. SIGNIFICANT STATEMENT: How do we represent the meaning of words and perform comparative judgements on them in our brain? According to influential theories, concepts are conceivable as points of an internal map (where distance represents similarity) that, as the physical space, can be mentally navigated. Here we use fMRI to show that when humans compare newly learnt words, they recruit a grid-like and a distance code, the same types of neural codes that, in mammals, represent relations between locations in the environment and support physical navigation between them.

Neural pattern similarity across concept exemplars predicts memory after a long delay

The irregularities of the world ensure that each interaction we have with a concept is unique. In order to generalize across these unique encounters to form a high-level representation of a concept, we must draw on similarities between exemplars to form new conceptual knowledge that is maintained over a long time. Two neural similarity measures - pattern robustness and encoding-retrieval similarity - are particularly important for predicting memory outcomes. In this study, we used fMRI to measure activity patterns while people encoded and retrieved novel pairings between unfamiliar (Dutch) words and visually presented animal species. We address two underexplored questions: 1) whether neural similarity measures can predict memory outcomes, despite perceptual variability between presentations of a concept and 2) if pattern similarity measures can predict subsequent memory over a long delay (i.e., one month). Our findings indicate that pattern robustness during encoding in brain regions that include parietal and medial temporal areas is an important predictor of subsequent memory. In addition, we found significant encoding-retrieval similarity in the left ventrolateral prefrontal cortex after a month's delay. These findings demonstrate that pattern similarity is an important predictor of memory for novel word-animal pairings even when the concept includes multiple exemplars. Importantly, we show that established predictive relationships between pattern similarity and subsequent memory do not require visually identical stimuli (i.e., are not simply due to low-level visual overlap between stimulus presentations) and are maintained over a month.

Influences on memory for naturalistic visual episodes: sleep, familiarity, and traits differentially affect forms of recall

The memories we form are composed of information that we extract from multifaceted episodes. Static stimuli and paired associations have proven invaluable stimuli for understanding memory, but real-life events feature spatial and temporal dimensions that help form new retrieval paths. We ask how the ability to recall semantic, temporal, and spatial aspects (the "what, when, and where") of naturalistic episodes is affected by three influences-prior familiarity, postencoding sleep, and individual differences-by testing their influence on three forms of recall: cued recall, free recall, and the extent that recalled details are recombined for a novel prompt. Naturalistic videos of events with rare animals were presented to 115 participants, randomly assigned to receive a 12- or 24-h delay with sleep and/or wakefulness. Participants' immediate and delayed recall was tested and coded by its spatial, temporal, and semantic content. We find that prior familiarity with items featured in events improved cued recall, but not free recall, particularly for temporal and spatial details. In contrast, postencoding sleep, relative to wakefulness, improved free recall, but not cued recall, of all forms of content. Finally, individuals with higher trait scores in the Survey of Autobiographical Memory spontaneously incorporated more spatial details during free recall, and more event details (at a trend level) in a novel recombination recall task. These findings show that prior familiarity, postencoding sleep, and memory traits can each enhance a different form of recall. More broadly, this work highlights that recall is heterogeneous in response to different influences on memory.

Neural Patterns are More Similar across Individuals during Successful Memory Encoding than during Failed Memory Encoding

After experiencing the same episode, some people can recall certain details about it, whereas others cannot. We investigate how common (intersubject) neural patterns during memory encoding influence whether an episode will be subsequently remembered, and how divergence from a common organization is associated with encoding failure. Using functional magnetic resonance imaging with intersubject multivariate analyses, we measured brain activity as people viewed episodes within wildlife videos and then assessed their memory for these episodes. During encoding, greater neural similarity was observed between the people who later remembered an episode (compared with those who did not) within the regions of the declarative memory network (hippocampus, posterior medial cortex [PMC], and dorsal Default Mode Network [dDMN]). The intersubject similarity of the PMC and dDMN was episode-specific. Hippocampal encoding patterns were also more similar between subjects for memory success that was defined after one day, compared with immediately after retrieval. The neural encoding patterns were sufficiently robust and generalizable to train machine learning classifiers to predict future recall success in held-out subjects, and a subset of decodable regions formed a network of shared classifier predictions of subsequent memory success. This work suggests that common neural patterns reflect successful, rather than unsuccessful, encoding across individuals.

Screen size matches of familiar images are biased by canonical size, rather than showing a memory size effect

Being confronted with the depiction of a familiar object activates a number of properties of the object that are stored in memory. Memory properties such as color and size have been shown to interfere with the processing of the color and of the size of the depiction, so that that reaction times are longer when the color or size of the depiction are incongruent with the stored knowledge about the object. In the case of color, it is known that the memorized information also affects the appearance of the depiction, for example when a gray banana appears slightly yellow, a phenomenon known as memory color effect. Here, I tested whether a memory size effect also occurs. To this aim, I conducted one experiment where observers matched either the screen size or the real-world size of pairs of animals or vehicles. The results indicate that the screen matches are biased in the same direction as the real-world size matches, opposite of what would be predicted by a memory color effect. This result was replicated in a second experiment using a different and larger set of animal images. Overall, I confirm that observers cannot ignore the real-world size information when they attempt to match the screen size of two items, although this results in a bias towards the canonical size of the items, rather than in a memory size effect.

Neural activity in human visual cortex is transformed by learning real world size

The way that our brain processes visual information is directly affected by our experience. Repeated exposure to a visual stimulus triggers experience-dependent plasticity in the visual cortex of many species. Humans also have the unique ability to acquire visual knowledge through instruction. We introduced human participants to the real-world size of previously unfamiliar species, and to the functional motion of novel tools, during a functional magnetic resonance imaging scan. Using machine learning, we compared activity patterns evoked by images of the new items, before and after participants learned the animals' real-world size or tools' motion. We found that, after acquiring size information, participants' visual activity patterns for the new animals became more confusable with activity patterns evoked by similar-sized known animals in early visual cortex, but not in ventral temporal cortex, reflecting an influence of new size knowledge on posterior, but not anterior, components of the ventral stream. In contrast, learning the functional motion of new tools did not lead to an equivalent change in recorded activity. Finally, the time-points marked by evidence of new size information in early visual cortex were more likely to show size information and greater activation in the right angular gyrus, a key hub of semantic knowledge and spatial cognition. Overall, these findings suggest that learning an item's real-world size by instruction influences subsequent activity in visual cortex and in a region that is central to semantic and spatial brain systems.