Brain mechanisms underlying cue-based memorizing during free viewing of movie Memento

Specialization for different memory dimensions in brain activity evoked by cued recollection

Cued recollection involves the retrieval of different features of the encoded event. Previous research has shown that the recollection of complex events jointly recruits the Default Mode and the Frontoparietal Control networks, but the degree to which activity within these networks varies as a function of the particular memory dimension (e.g., the "when-what-where" information) remains largely unknown. In the present functional Magnetic Resonance Imaging (fMRI) study, human participants retrieved specific information about a previously encoded TV show to assess the veracity of detailed sentences along four memory dimensions (i.e., object and character details, spatial layouts, temporal sequences, verbal dialogues). A common activity for all dimensions was observed in a left-lateralized network of regions that largely overlaps with the Frontoparietal Control Network (FPCN), including the lateral prefrontal, lateral superior parietal, and lateral temporal cortex. Instead, a larger degree of specialization for different memory dimensions was observed within the Default Mode Network (DMN), particularly in its posterior nodes. Dimension-related specificity in both networks was associated with memory performance across subjects. Finally, a clear leftward asymmetry was observed in the DMN for all dimensions except for the temporal one, whereas the FPCN showed a bilateral activation across dimensions. The present results generally support the view that specific memory information is processed by a mosaic of regions within large portions of the associative cortex involved in higher-order mnemonic functions.

Stimulus Selection Influences Prediction of Individual Phenotypes in Naturalistic Conditions

While the use of naturalistic stimuli such as movie clips for understanding individual differences and brain-behaviour relationships attracts increasing interest, the influence of stimulus selection remains largely unclear. By using machine learning to predict individual traits (phenotypes) from brain activity evoked during various movie clips, we show that different movie stimuli can result in distinct prediction performances. In brain regions related to lower-level processing of the stimulus, prediction to a certain degree benefits from stronger synchronisation of brain activity across subjects. By contrast, better predictions in frontoparietal brain regions are mainly associated with larger inter-subject variability. Furthermore, we demonstrate that while movie clips with rich social content in general achieve better predictions, the importance of specific movie features for prediction highly depends on the phenotype under investigation. Overall, our findings underscore the importance of careful stimulus selection and provide novel insights into stimulus selection for phenotype prediction in naturalistic conditions, opening new avenues for future research.

Causal and Chronological Relationships Predict Memory Organization for Nonlinear Narratives

While recounting an experience, one can employ multiple strategies to transition from one part to the next. For instance, if the event was learned out of linear order, one can recall events according to the time they were learned (temporal), similar events (semantic), events occurring nearby in time (chronological), or events produced by the current event (causal). To disentangle the importance of these factors, we had participants watch the nonlinear narrative, Memento, under different task instructions and presentation orders. For each scene of the film, we also separately computed semantic and causal networks. We then contrasted the evidence for temporal, semantic, chronological, or causal strategies during recall. Critically, there was stronger evidence for the causal and chronological strategies than semantic or temporal strategies. Moreover, the causal and chronological strategies outperformed the temporal one even when we asked participants to recall the film in the presented order, underscoring the fundamental nature of causal structure in scaffolding understanding and organizing recall. Nevertheless, time still marginally predicted recall transitions, suggesting it operates as a weak signal in the presence of more salient forms of structure. In addition, semantic and causal network properties predicted scene memorability, including a stronger role for incoming causes to an event than its outgoing effects. In summary, these findings highlight the importance of accounting for complex, causal networks in knowledge building and memory.

Intersubject Dynamic Conditional Correlation: A Novel Method to Track the Framewise Network Implication during Naturalistic Stimuli

Background: Naturalistic stimuli have become increasingly popular in modern cognitive neuroscience. These stimuli have high ecological validity due to their rich and multilayered features. However, their complexity also presents methodological challenges for uncovering neural network reconfiguration. Dynamic functional connectivity using the sliding-window technique is commonly used but has several limitations. In this study, we introduce a new method called intersubject dynamic conditional correlation (ISDCC). Method: ISDCC uses intersubject analysis to remove intrinsic and non-neuronal signals, retaining only intersubject-consistent stimuli-induced signals. It then applies dynamic conditional correlation (DCC) based on the generalized autoregressive conditional heteroskedasticity to calculate the framewise functional connectivity. To validate ISDCC, we analyzed simulation data with known network reconfiguration patterns and two publicly available narrative functional Magnetic Resonance Imaging (fMRI) datasets. Results: (1) ISDCC accurately unveiled the underlying network reconfiguration patterns in simulation data, demonstrating greater sensitivity than DCC; (2) ISDCC identified synchronized network reconfiguration patterns across listeners; (3) ISDCC effectively differentiated between stimulus types with varying temporal coherence; and (4) network reconfigurations unveiled by ISDCC were significantly correlated with listener engagement during narrative comprehension. Conclusion: ISDCC is a precise and dynamic method for tracking network implications in response to naturalistic stimuli.

Searching for Linearity: Reconstructive Processes Reverse Temporal Scrambling in Memory for Movie Scenes

Meaning-making and temporal memory are closely intertwined, yet we still do not know how the overall understanding of complex events affects retrospective temporal judgments. The present study investigated the effect of a manipulation of the temporal linearity of a narrative on the subsequent memory-for-time performance. Participants indicated the time of occurrence of short video clips extracted from a previously encoded movie on a horizontal timeline representing the movie duration. Importantly, a group of participants (N = 20) watched the original movie, which depicts events occurring in chronological order, whereas another group (N = 30) watched a scrambled version of the same movie in which the temporal linearity was lost. This procedure allowed us to measure the quantity and direction of the temporal memory bias. The scrambled presentation produced a mild and general impairment of recognition memory compared to the linear presentation. More importantly, it biased temporal judgments as a function of the direction and amount of discrepancy between the story and the viewing time, in accordance with an automatic reshaping of temporal memory caused by a chronological representation of the storyline. This effect could be distinguished from a tendency to move judgments toward the center of the timeline, independently from the specific scrambling arrangement, consistent with the idea that the non-linearity of the story also generally increased the degree of temporal uncertainty. Taken together, our results provide further evidence that temporal memories are automatically reconstructed according to the general meaning of the events.

The motifs of radical embodied neuroscience

In this paper, I analyse how the emerging scientific framework of radical embodied neuroscience is different from contemporary mainstream cognitive neuroscience. To do so, I propose the notion of motif to enrich the philosophical toolkit of cognitive neuroscience. This notion can be used to characterize the guiding ideas of any given scientific framework in psychology and neuroscience. Motifs are highly unconstrained, open-ended concepts that support equally open-ended families of explanations. Different scientific frameworks-e.g., psychophysics or cognitive neuroscience-provide these motifs to answer the overarching themes of these disciplines, such as the relationship between stimuli and sensations or the proper methods of the sciences of the mind. Some motifs of mainstream cognitive neuroscience are the motif of encoding, the motif of input-output systems, and the motif of algorithms. The two first ones answer the question about the relationship between stimuli, sensations and experience (e.g., stimuli are input and are encoded by brain structures). The latter one answers the question regarding the mechanism of cognition and experience. The three of them are equally unconstrained and open-ended, and they serve as an umbrella for different kinds of explanation-i.e., different positions regarding what counts as a code or as an input. Along with the articulation of the notion of motif, the main aim of this article is to present three motifs for radical embodied neuroscience: the motif of complex stimulation, the motif of organic behaviour and the motif of resonance.

Inter- and intra-subject similarity in network functional connectivity across a full narrative movie

Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging, are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. Previous studies have primarily focused on using shorter movie clips, geared toward eliciting specific and often isolated emotions, while the potential behind using full narratives depicted in commercial movies as a proxy for real-life experiences has barely been explored. Here, we offer preliminary evidence that a full narrative movie (FNM), that is, a movie covering a complete narrative arc, can capture complex socio-affective dynamics and their links to individual differences. Using the studyforrest dataset, we investigated inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into eight consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results show that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. We conclude that FNMs offer complex content and dynamics that might be particularly valuable for studying individual differences. Further characterization of movie features, such as the overarching narratives, that enhance individual differences is needed for advancing the potential of NV research.

Unraveling dyadic psycho-physiology of social presence between strangers during an audio drama - a signal-analysis approach

A mere co-presence of an unfamiliar person may modulate an individual's attentive engagement with specific events or situations to a significant degree. To understand better how such social presence affects experiences, we recorded a set of parallel multimodal facial and psychophysiological data with subjects (N = 36) who listened to dramatic audio scenes alone or when facing an unfamiliar person. Both a selection of 6 s affective sound clips (IADS-2) followed by a 27 min soundtrack extracted from a Finnish episode film depicted familiar and often intense social situations familiar from the everyday world. Considering the systemic complexity of both the chosen naturalistic stimuli and expected variations in the experimental social situation, we applied a novel combination of signal analysis methods using inter-subject correlation (ISC) analysis, Representational Similarity Analysis (RSA) and Recurrence Quantification Analysis (RQA) followed by gradient boosting classification. We report our findings concerning three facial signals, gaze, eyebrow and smile that can be linked to socially motivated facial movements. We found that ISC values of pairs, whether calculated on true pairs or any two individuals who had a partner, were lower than the group with single individuals. Thus, audio stimuli induced more unique responses in those subjects who were listening to it in the presence of another person, while individual listeners tended to yield a more uniform response as it was driven by dramatized audio stimulus alone. Furthermore, our classifiers models trained using recurrence properties of gaze, eyebrows and smile signals demonstrated distinctive differences in the recurrence dynamics of signals from paired subjects and revealed the impact of individual differences on the latter. We showed that the presence of an unfamiliar co-listener that modifies social dynamics of dyadic listening tasks can be detected reliably from visible facial modalities. By applying our analysis framework to a broader range of psycho-physiological data, together with annotations of the content, and subjective reports of participants, we expected more detailed dyadic dependencies to be revealed. Our work contributes towards modeling and predicting human social behaviors to specific types of audio-visually mediated, virtual, and live social situations.

Narrative simulation of social experiences in naturalistic context - A neurocinematic approach

Narratives may be regarded as simulations of everyday social situations. They are key to studying the human mind in socio-culturally determined contexts as they allow anchoring to the common ground of embodied and environmentally-engaged cognition. Here we review recent findings from naturalistic neuroscience on neural functions in conditions that mimic lifelike situations. We will focus particularly on neurocinematics, a research field that applies mediated narratives as stimuli for neuroimaging experiments. During the last two decades, this paradigm has contributed to an accumulation of insights about the neural underpinnings of behavior and sense-making in various narratively contextualized situations particularly pertaining to socio-emotional encounters. One of the key questions in neurocinematics is, how do intersubjectively synchronized brain activations relate to subjective experiences? Another question we address is how to bring natural contexts into experimental studies. Seeking to respond to both questions, we suggest neurocinematic studies to examine three manifestations of the same phenomenon side-by-side: subjective experiences of narrative situations, unfolding of narrative stimulus structure, and neural processes that co-constitute the experience. This approach facilitates identifying experientially meaningful activity patterns in the brain and points out what they may mean in relation to shared and communicable contents. Via rich-featured and temporally contextualized narrative stimuli, neurocinematics attempts to contribute to emerging holistic theories of neural dynamics and connectomics explaining typical and atypical interindividual variability.

An embodiment of the cinematographer: emotional and perceptual responses to different camera movement techniques

We investigate the relationship between camera movement techniques and cognitive responses in audiences, reporting on an experiment exploring the effects of different camera movement methods on viewers' degree of immersion and emotional response. This follows directly from preceding experimental literature and is further motivated by accounts and experiences of practicing cinematographers (authors included), which indicates a correspondence between the two. We designed three different cinematic scenes with indifferent moods, and shot each one time with Steadicam, dolly, handheld, and static camera, resulting in 12 different clips. A total of 44 non-professional participants watched the clips and rated their reactions in terms of arousal and degree of involvement. Experimental results are mixed: movement affects the sense of involvement but not necessarily emotional response. We present and discuss some further explorative results and possible future directions to improve the design. We argue in this contribution that there is value in experimental approaches to cinematography, enabling the systematic study of creative intuitions and audience responses in controlled settings.

Integrating media content analysis, reception analysis, and media effects studies

Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.

Aberrant functional connectivity of the bed nucleus of the stria terminalis and its age dependence in children and adolescents with social anxiety disorder

BACKGROUND

Social anxiety disorder (SAD) is a prevalent and impairing mental disorder among children and adolescents. The bed nucleus of the stria terminalis (BNST) plays a critical role in anxiety disorders, including valence surveillance and hypervigilance for potential threats. However, the role of BNST and its related functional network in children and adolescents with SAD has not been fully investigated. This study examined the aberration of BNST's functional connectivity and its age dependence in adolescents with SAD.

METHODS

Using a sample of 75 SAD patients and 75 healthy controls (HCs) children aged 9-18 years old, we delineated the group-by-age interaction of BNST-seeded functional connectivity (FC) during resting state and movie-watching. The relationships between BNST-seeded FC and clinical scores were also examined.

RESULTS

During movie viewing, the FC between the right BNST and the left amygdala, bilateral posterior cingulate cortex (PCC), bilateral superior temporal cortex, and right pericalcarine cortex showed a diagnostic group-by-age interaction. Compared to HCs, SAD patients showed a significant enhancement of the above FC at younger ages. Meanwhile, they showed an age-dependent decrease in FC between the right BNST and left amygdala. Furthermore, for SAD patients, FC between the right BNST and left amygdala during movie viewing was positively correlated with separation anxiety scores.

CONCLUSIONS

The right BNST plays an essential role in the aberrant brain functioning in children and adolescents with SAD. The atypicality of BNST's FC has remarkable age dependence in SAD, suggesting an association of SAD with neurodevelopmental traits.

Neural unscrambling of temporal information during a nonlinear narrative

Although we must experience our lives chronologically, storytellers often manipulate the order in which they relay events. How the brain processes temporal information while encoding a nonlinear narrative remains unclear. Here, we use functional magnetic resonance imaging during movie watching to investigate which brain regions are sensitive to information about time in a narrative and test whether the representation of temporal context across a narrative is more influenced by the order in which events are presented or their underlying chronological sequence. Results indicate that medial parietal regions are sensitive to cued jumps through time over and above other changes in context (i.e., location). Moreover, when processing non-chronological narrative information, the precuneus and posterior cingulate engage in on-the-fly temporal unscrambling to represent information chronologically. Specifically, days that are closer together in chronological time are represented more similarly regardless of when they are presented in the movie, and this representation is consistent across participants. Additional analyses reveal a strong spatial signature associated with higher magnitude jumps through time. These findings are consistent with prior theorizing on medial parietal regions as central to maintaining and updating narrative situation models, and suggest the priority of chronological information when encoding narrative events.

Computational meaningfulness as the source of beneficial cognitive biases

The human brain has evolved to solve the problems it encounters in multiple environments. In solving these challenges, it forms mental simulations about multidimensional information about the world. These processes produce context-dependent behaviors. The brain as overparameterized modeling organ is an evolutionary solution for producing behavior in a complex world. One of the most essential characteristics of living creatures is that they compute the values of information they receive from external and internal contexts. As a result of this computation, the creature can behave in optimal ways in each environment. Whereas most other living creatures compute almost exclusively biological values (e.g., how to get food), the human as a cultural creature computes meaningfulness from the perspective of one's activity. The computational meaningfulness means the process of the human brain, with the help of which an individual tries to make the respective situation comprehensible to herself to know how to behave optimally. This paper challenges the bias-centric approach of behavioral economics by exploring different possibilities opened up by computational meaningfulness with insight into wider perspectives. We concentrate on confirmation bias and framing effect as behavioral economics examples of cognitive biases. We conclude that from the computational meaningfulness perspective of the brain, the use of these biases are indispensable property of an optimally designed computational system of what the human brain is like. From this perspective, cognitive biases can be rational under some conditions. Whereas the bias-centric approach relies on small-scale interpretable models which include only a few explanatory variables, the computational meaningfulness perspective emphasizes the behavioral models, which allow multiple variables in these models. People are used to working in multidimensional and varying environments. The human brain is at its best in such an environment and scientific study should increasingly take place in such situations simulating the real environment. By using naturalistic stimuli (e.g., videos and VR) we can create more realistic, life-like contexts for research purposes and analyze resulting data using machine learning algorithms. In this manner, we can better explain, understand and predict human behavior and choice in different contexts.

Do sparse brain activity patterns underlie human cognition?

Accumulating multivariate pattern analysis (MVPA) results from fMRI studies suggest that information is represented in fingerprint patterns of activations and deactivations during perception, emotions, and cognition. We postulate that these fingerprint patterns might reflect neuronal-population level sparse code documented in two-photon calcium imaging studies in animal models, i.e., information represented in specific and reproducible ensembles of a few percent of active neurons amidst widespread inhibition in neural populations. We suggest that such representations constitute a fundamental organizational principle via interacting across multiple levels of brain hierarchy, thus giving rise to perception, emotions, and cognition.

Predicting memory from the network structure of naturalistic events

When we remember events, we often do not only recall individual events, but also the connections between them. However, extant research has focused on how humans segment and remember discrete events from continuous input, with far less attention given to how the structure of connections between events impacts memory. Here we conduct a functional magnetic resonance imaging study in which participants watch and recall a series of realistic audiovisual narratives. By transforming narratives into networks of events, we demonstrate that more central events-those with stronger semantic or causal connections to other events-are better remembered. During encoding, central events evoke larger hippocampal event boundary responses associated with memory formation. During recall, high centrality is associated with stronger activation in cortical areas involved in episodic recollection, and more similar neural representations across individuals. Together, these results suggest that when humans encode and retrieve complex real-world experiences, the reliability and accessibility of memory representations is shaped by their location within a network of events.

Human's Intuitive Mental Models as a Source of Realistic Artificial Intelligence and Engineering

Despite the success of artificial intelligence (AI), we are still far away from AI that model the world as humans do. This study focuses for explaining human behavior from intuitive mental models' perspectives. We describe how behavior arises in biological systems and how the better understanding of this biological system can lead to advances in the development of human-like AI. Human can build intuitive models from physical, social, and cultural situations. In addition, we follow Bayesian inference to combine intuitive models and new information to make decisions. We should build similar intuitive models and Bayesian algorithms for the new AI. We suggest that the probability calculation in Bayesian sense is sensitive to semantic properties of the objects' combination formed by observation and prior experience. We call this brain process as computational meaningfulness and it is closer to the Bayesian ideal, when the occurrence of probabilities of these objects are believable. How does the human brain form models of the world and apply these models in its behavior? We outline the answers from three perspectives. First, intuitive models support an individual to use information meaningful ways in a current context. Second, neuroeconomics proposes that the valuation network in the brain has essential role in human decision making. It combines psychological, economical, and neuroscientific approaches to reveal the biological mechanisms by which decisions are made. Then, the brain is an over-parameterized modeling organ and produces optimal behavior in a complex word. Finally, a progress in data analysis techniques in AI has allowed us to decipher how the human brain valuates different options in complex situations. By combining big datasets with machine learning models, it is possible to gain insight from complex neural data beyond what was possible before. We describe these solutions by reviewing the current research from this perspective. In this study, we outline the basic aspects for human-like AI and we discuss on how science can benefit from AI. The better we understand human's brain mechanisms, the better we can apply this understanding for building new AI. Both development of AI and understanding of human behavior go hand in hand.

Test-retest reliability of dynamic functional connectivity in naturalistic paradigm functional magnetic resonance imaging

Dynamic functional connectivity (dFC) has been increasingly used to characterize the brain transient temporal functional patterns and their alterations in diseased brains. Meanwhile, naturalistic neuroimaging paradigms have been an emerging approach for cognitive neuroscience with high ecological validity. However, the test–retest reliability of dFC in naturalistic paradigm neuroimaging is largely unknown. To address this issue, we examined the test–retest reliability of dFC in functional magnetic resonance imaging (fMRI) under natural viewing condition. The intraclass correlation coefficients (ICC) of four dFC statistics including standard deviation (Std), coefficient of variation (COV), amplitude of low frequency fluctuation (ALFF), and excursion (Excursion) were used to measure the test–retest reliability. The test–retest reliability of dFC in naturalistic viewing condition was then compared with that under resting state. Our experimental results showed that: (a) Global test–retest reliability of dFC was much lower than that of static functional connectivity (sFC) in both resting‐state and naturalistic viewing conditions; (b) Both global and local (including visual, limbic and default mode networks) test–retest reliability of dFC could be significantly improved in naturalistic viewing condition compared to that in resting state; (c) There existed strong negative correlation between sFC and dFC, weak negative correlation between dFC and dFC‐ICC (i.e., ICC of dFC), as well as weak positive correlation between dFC‐ICC and sFC‐ICC (i.e., ICC of sFC). The present study provides novel evidence for the promotion of naturalistic paradigm fMRI in functional brain network studies.

Moment-by-moment tracking of naturalistic learning and its underlying hippocampo-cortical interactions

Humans form lasting memories of stimuli that were only encountered once. This naturally occurs when listening to a story, however it remains unclear how and when memories are stored and retrieved during story-listening. Here, we first confirm in behavioral experiments that participants can learn about the structure of a story after a single exposure and are able to recall upcoming words when the story is presented again. We then track mnemonic information in high frequency activity (70–200 Hz) as patients undergoing electrocorticographic recordings listen twice to the same story. We demonstrate predictive recall of upcoming information through neural responses in auditory processing regions. This neural measure correlates with behavioral measures of event segmentation and learning. Event boundaries are linked to information flow from cortex to hippocampus. When listening for a second time, information flow from hippocampus to cortex precedes moments of predictive recall. These results provide insight on a fine-grained temporal scale into how episodic memory encoding and retrieval work under naturalistic conditions.

When listening to a story, humans learn about its structure and content. Here the authors reveal the neural processes behind episodic memory and predictive recall at a fine temporal scale in this naturalistic setting

Relating the Past with the Present: Information Integration and Segregation during Ongoing Narrative Processing

This study examined how the brain dynamically updates event representations by integrating new information over multiple minutes while segregating irrelevant input. A professional writer custom-designed a narrative with two independent storylines, interleaving across minute-long segments (ABAB). In the last (C) part, characters from the two storylines meet and their shared history is revealed. Part C is designed to induce the spontaneous recall of past events, upon the recurrence of narrative motifs from A/B, and to shed new light on them. Our fMRI results showed storyline-specific neural patterns, which were reinstated (i.e., became more active) during storyline transitions. This effect increased along the processing timescale hierarchy, peaking in the default mode network. Similarly, the neural reinstatement of motifs was found during Part C. Furthermore, participants showing stronger motif reinstatement performed better in integrating A/B and C events, demonstrating the role of memory reactivation in information integration over intervening irrelevant events.

Enhanced reinstatement of naturalistic event memories due to hippocampal-network-targeted stimulation

Episodic memory involves the reinstatement of distributed patterns of brain activity present when events were initially experienced. The hippocampus is thought to coordinate reinstatement via its interactions with a network of brain regions, but this hypothesis has not been causally tested in humans. The current study directly tested the involvement of the hippocampal network in reinstatement using network-targeted noninvasive stimulation. We measured reinstatement of multi-voxel patterns of functional magnetic resonance imaging (fMRI) activity during encoding and retrieval of naturalistic video clips depicting everyday activities. Reinstatement of video-specific activity patterns was robust in posterior parietal and occipital areas previously implicated in event reinstatement. Theta-burst stimulation targeting the hippocampal network increased video-specific reinstatement of fMRI activity patterns in occipital cortex and improved memory accuracy relative to stimulation of a control out-of-network location. Furthermore, stimulation targeting the hippocampal network influenced the trial-by-trial relationship between hippocampal activity during encoding and later reinstatement in occipital cortex. These findings implicate the hippocampal network in the reinstatement of spatially distributed patterns of event-specific activity and identify a role for the hippocampus in encoding complex naturalistic events that later undergo cortical reinstatement.

Design of complex neuroscience experiments using mixed-integer linear programming

Over the past few decades, neuroscience experiments have become increasingly complex and naturalistic. Experimental design has in turn become more challenging, as experiments must conform to an ever-increasing diversity of design constraints. In this article, we demonstrate how this design process can be greatly assisted using an optimization tool known as mixed-integer linear programming (MILP). MILP provides a rich framework for incorporating many types of real-world design constraints into a neuroscience experiment. We introduce the mathematical foundations of MILP, compare MILP to other experimental design techniques, and provide four case studies of how MILP can be used to solve complex experimental design challenges.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Reminders activate the prefrontal-medial temporal cortex and attenuate forgetting of event memory

Replicas of an aspect of an experienced event can serve as effective reminders, yet little is known about the neural basis of such reminding effects. Here we examined the neural activity underlying the memory-enhancing effect of reminders 1 week after encoding of naturalistic film clip events. We used fMRI to determine differences in network activity associated with recently reactivated memories relative to comparably aged, non-reactivated memories. Reminders were effective in facilitating overall retrieval of memory for film clips, in an all-or-none fashion. Prefrontal cortex and hippocampus were activated during both reminders and retrieval. Peak activation in ventro-lateral prefrontal cortex (vPFC) preceded peak activation in the right hippocampus during the reminders. For film clips that were successfully retrieved after 7 days, pre-retrieval reminders did not enhance the quality of the retrieved memory or the number of details retrieved, nor did they more strongly engage regions of the recollection network than did successful retrieval of a non-reminded film clip. These results suggest that reminders prior to retrieval are an effective means of boosting retrieval of otherwise inaccessible episodic events, and that the inability to recall certain events after a delay of a week largely reflects a retrieval deficit, rather than a storage deficit for this information. The results extend other evidence that vPFC drives activation of the hippocampus to facilitate memory retrieval and scene construction, and show that this facilitation also occurs when reminder cues precede successful retrieval attempts. The time course of vPFC-hippocampal activity during the reminder suggests that reminders may first engage schematic information meditated by vPFC followed by a recollection process mediated by the hippocampus.

Context-Dependent Coding of Temporal Distance Between Cinematic Events in the Human Precuneus

How temporal and contextual information interactively impact on behavior and brain activity during the retrieval of temporal order about naturalistic episodes remains incompletely understood. Here, we used fMRI to examine the effects of contextual signals derived from the content of the movie on the neural correlates underlying memory retrieval of temporal-order in human subjects of both sexes. By contrasting SAME versus DIFF storyline conditions during the retrieval of the temporal order of cinematic events, we found that the activation in the precuneus, as well as behavior, are significantly modulated according to storyline condition, supporting our prediction of contextual information contributing to temporal retrieval. We suggest that the precuneus engages in memory retrieval via reconstructive mechanisms, entailing search within a movie-specific, situational knowledge-structure. Furthermore, information-based analyses of multivoxel activity revealed that the precuneus also contains a context-independent linear representation of temporal distances, consistent with a chronological organization of memory traces. We thus put forward that the retrieval of the temporal-order of naturalistic events encoded in rich and dynamic contexts relies on the joint contribution of chronological and reconstructive mechanisms, both of which rely on the medioposterior parietal cortex in humans.SIGNIFICANCE STATEMENT Successful retrieval of episodic memory is dependent on both temporal and contextual signals. However, when contextual signals derived from multiple storylines or narratives are complex and intertwined, the behavioral and neural correlates underpinning the interplay between time and context is not completely understood. Here we characterized the activation level and multivoxel pattern of BOLD signals underlying the modulation of such contextual information during temporal order judgment in the precuneus. Our findings provide us with an elucidation of subprocesses implicating the medial parietal cortex in realizing temporal organization of episodic details.

Functional Imaging of Visuospatial Attention in Complex and Naturalistic Conditions

One of the ultimate goals of cognitive neuroscience is to understand how the brain works in the real world. Functional imaging with naturalistic stimuli provides us with the opportunity to study the brain in situations similar to the everyday life. This includes the processing of complex stimuli that can trigger many types of signals related both to the physical characteristics of the external input and to the internal knowledge that we have about natural objects and environments. In this chapter, I will first outline different types of stimuli that have been used in naturalistic imaging studies. These include static pictures, short video clips, full-length movies, and virtual reality, each comprising specific advantages and disadvantages. Next, I will turn to the main issue of visual-spatial orienting in naturalistic conditions and its neural substrates. I will discuss different classes of internal signals, related to objects, scene structure, and long-term memory. All of these, together with external signals about stimulus salience, have been found to modulate the activity and the connectivity of the frontoparietal attention networks. I will conclude by pointing out some promising future directions for functional imaging with naturalistic stimuli. Despite this field of research is still in its early days, I consider that it will play a major role in bridging the gap between standard laboratory paradigms and mechanisms of brain functioning in the real world.

Naturalistic Stimuli in Neuroscience: Critically Acclaimed

Cognitive neuroscience has traditionally focused on simple tasks, presented sparsely and using abstract stimuli. While this approach has yielded fundamental insights into functional specialisation in the brain, its ecological validity remains uncertain. Do these tasks capture how brains function 'in the wild', where stimuli are dynamic, multimodal, and crowded? Ecologically valid paradigms that approximate real life scenarios, using stimuli such as films, spoken narratives, music, and multiperson games emerged in response to these concerns over a decade ago. We critically appraise whether this approach has delivered on its promise to deliver new insights into brain function. We highlight the challenges, technological innovations, and clinical opportunities that are required should this field meet its full potential.