Attention Strongly Modulates Reliability of Neural Responses to Naturalistic Narrative Stimuli

Bidirectional brain-body interactions during natural story listening

Narratives can synchronize neural and physiological signals between individuals, but the relationship between these signals, and the underlying mechanism, is unclear. We hypothesized a top-down effect of cognition on arousal and predicted that auditory narratives will drive not only brain signals but also peripheral physiological signals. We find that auditory narratives entrained gaze variation, saccade initiation, pupil size, and heart rate. This is consistent with a top-down effect of cognition on autonomic function. We also hypothesized a bottom-up effect, whereby autonomic physiology affects arousal. Controlled breathing affected pupil size, and heart rate was entrained by controlled saccades. Additionally, fluctuations in heart rate preceded fluctuations of pupil size and brain signals. Gaze variation, pupil size, and heart rate were all associated with anterior-central brain signals. Together, these results suggest bidirectional causal effects between peripheral autonomic function and central brain circuits involved in the control of arousal.

Wearable Biosensor Technology in Education: A Systematic Review

Wearable Biosensor Technology (WBT) has emerged as a transformative tool in the educational system over the past decade. This systematic review encompasses a comprehensive analysis of WBT utilization in educational settings over a 10-year span (2012-2022), highlighting the evolution of this field to address challenges in education by integrating technology to solve specific educational challenges, such as enhancing student engagement, monitoring stress and cognitive load, improving learning experiences, and providing real-time feedback for both students and educators. By exploring these aspects, this review sheds light on the potential implications of WBT on the future of learning. A rigorous and systematic search of major academic databases, including Google Scholar and Scopus, was conducted in accordance with the PRISMA guidelines. Relevant studies were selected based on predefined inclusion and exclusion criteria. The articles selected were assessed for methodological quality and bias using established tools. The process of data extraction and synthesis followed a structured framework. Key findings include the shift from theoretical exploration to practical implementation, with EEG being the predominant measurement, aiming to explore mental states, physiological constructs, and teaching effectiveness. Wearable biosensors are significantly impacting the educational field, serving as an important resource for educators and a tool for students. Their application has the potential to transform and optimize academic practices through sensors that capture biometric data, enabling the implementation of metrics and models to understand the development and performance of students and professors in an academic environment, as well as to gain insights into the learning process.

The impact of internal-generated contextual clues on EFL vocabulary learning: insights from EEG

With the popularity of learning vocabulary online among English as a Foreign Language (EFL) learners today, educators and researchers have been considering ways to enhance the effectiveness of this approach. Prior research has underscored the significance of contextual clues in vocabulary acquisition. However, few studies have compared the context provided by instructional materials and that generated by learners themselves. Hence, this present study sought to explore the impact of internal-generated contextual clues in comparison to those provided by instructional materials on EFL learners' online vocabulary acquisition. A total of 26 university students were enrolled and underwent electroencephalography (EEG). Based on a within-subjects design, all participants learned two groups of vocabulary words through a series of video clips under two conditions: one where the contexts were externally provided and the other where participants themselves generated the contexts. In this regard, participants were tasked with either viewing contextual clues presented on the screen or creating their own contextual clues for word comprehension. EEG signals were recorded during the learning process to explore neural activities, and post-tests were conducted to assess learning performance after each vocabulary learning session. Our behavioral results indicated that comprehending words with internal-generated contextual clues resulted in superior learning performance compared to using context provided by instructional materials. Furthermore, EEG data revealed that learners expended greater cognitive resources and mental effort in semantically integrating the meaning of words when they self-created contextual clues, as evidenced by stronger alpha and beta-band oscillations. Moreover, the stronger alpha-band oscillations and lower inter-subject correlation (ISC) among learners suggested that the generative task of creating context enhanced their top-down attentional control mechanisms and selective visual processing when learning vocabulary from videos. These findings underscored the positive effects of internal-generated contextual clues, indicating that instructors should encourage learners to construct their own contexts in online EFL vocabulary instruction rather than providing pre-defined contexts. Future research should aim to explore the limits and conditions of employing these two types of contextual clues in online EFL vocabulary learning. This could be achieved by manipulating the quality and understandability of contexts and considering learners' language proficiency levels.

A model of time-varying music engagement

The current paper offers a model of time-varying music engagement, defined as changes in curiosity, attention and positive valence, as music unfolds over time. First, we present research (including new data) showing that listeners tend to allocate attention to music in a manner that is guided by both features of the music and listeners' individual differences. Next, we review relevant predictive processing literature before using this body of work to inform our model. In brief, we propose that music engagement, over the course of an extended listening episode, may constitute several cycles of curiosity, attention and positive valence that are interspersed with moments of mind-wandering. Further, we suggest that refocusing on music after an episode of mind-wandering can be due to triggers in the music or, conversely, mental action that occurs when the listener realizes they are mind-wandering. Finally, we argue that factors that modulate both overall levels of music engagement and how it changes over time include music complexity, listener background and the listening context. Our paper highlights how music can be used to provide insights into the temporal dynamics of attention and into how curiosity might emerge in everyday contexts. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Examining the exposure-reception-retention link in realistic communication environments via VR and eye-tracking: The VR billboard paradigm

Exposure is key to message effects. No effects can ensue if a health, political, or commercial message is not noticed. Yet, existing research in communication, advertising, and related disciplines often measures 'opportunities for exposure' at an aggregate level, whereas knowing whether recipients were 'actually exposed' to a message requires a micro-level approach. Micro-level research, on the other hand, focuses on message processing and retention, takes place under highly controlled laboratory conditions with forced message exposure, and largely ignores how recipients attend selectively to messages under more natural conditions. Eye-tracking enables us to assess actual exposure, but its previous applications were restricted to screen-based reading paradigms lacking ecological validity or field studies that suffer from limited experimental control. Our solution is to measure eye-tracking within an immersive VR environment that creates the message delivery and reception context. Specifically, we simulate a car ride down a highway alongside which billboards are placed. The VR headset (HP Omnicept Pro) provides an interactive 3D view of the environment and holds a seamlessly integrated binocular eye tracker that records the drivers' gaze and detects all fixations on the billboards. This allows us to quantify the nexus between exposure and reception rigorously, and to link our measures to subsequent memory, i.e., whether messages were remembered, forgotten, or not even encoded. An empirical study shows that incidental memory for messages differs based on participants' gaze behavior while passing the billboards. The study further shows how an experimental manipulation of attentional demands directly impacts drivers' gaze behavior and memory. We discuss the large potential of this paradigm to quantify exposure and message reception in realistic communication environments and the equally promising applications in new media contexts (e.g., the Metaverse).

The effect of repetition on intersubject synchrony assessed with fMRI

We investigated how repeated exposure to a stimulus affects intersubject synchrony in the brains of young and older adults. We used functional magnetic resonance imaging (fMRI) to measure brain responses to familiar and novel stimuli. Young adults participated in a familiarization paradigm designed to mimic 'natural' exposure while older adults were presented with stimuli they had known for more than 50 years. Intersubject synchrony was calculated to detect common stimulus-driven brain activity across young and older adults as they listened to the novel and familiar stimuli. Contrary to our hypotheses, synchrony was not related to the amount of stimulus exposure; both young and older adults showed more synchrony to novel than to familiar stimuli regardless of whether the stimuli had been heard once, known for a few weeks, or known for more than 50 years. In young adults these synchrony differences were found across the brain in the bilateral temporal lobes, and in the frontal orbital cortex. In older adults the synchrony differences were found only in the bilateral temporal lobes. This reduction may be related to an increase in idiosyncratic responses after exposure to a stimulus but does not seem to be related to how well the stimuli are learned or to differences in attention. Until the effects of repeated exposure on synchrony are fully understood, future studies using intersubject synchrony, where the novelty of the stimuli cannot be guaranteed, may consider exposing all of their participants to the stimuli once before data are collected to mitigate the effects of any systematic differences in stimulus exposure.

The Synergy Zone: Connecting the Mind, Brain, and Heart for the Ideal Classroom Learning Environment

This paper proposes a new perspective on implementing neuroeducation in the classroom. The pandemic exacerbated the mental health issues of faculty and students, creating a mental health crisis that impairs learning. It is important to get our students back in "the zone", both cognitively and emotionally, by creating an ideal learning environment for capturing our students and keeping them-the Synergy Zone. Research that examines the classroom environment often focuses on the foreground-instructors' organizational and instructional aspects and content. However, the emotional climate of the classroom affects student well-being. This emotional climate would ideally exhibit the brain states of engagement, attention, connection, and enjoyment by addressing the mind, brain, and heart. This ideal learning environment would be achieved by combining proposed practices derived from three areas of research: flow theory, brain synchronization, and positive emotion with heart engagement. Each of these enhances the desired brain states in a way that the whole is greater than the sum of the individual parts. I call this the Synergy Zone. A limitation of this proposed model is that implementation of some aspects may be challenging, and professional development resources might be needed. This essay presenting this perspective provides the relevant scientific research and the educational implications of implementation.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

Inter-subject correlations of EEG reflect subjective arousal and acoustic features of music

Background

Research on music-induced emotion and brain activity is constantly expanding. Although studies using inter-subject correlation (ISC), a collectively shared brain activity analysis method, have been conducted, whether ISC during music listening represents the music preferences of a large population remains uncertain; additionally, it remains unclear which factors influence ISC during music listening. Therefore, here, we aimed to investigate whether the ISCs of electroencephalography (EEG) during music listening represent a preference for music reflecting engagement or interest of a large population in music.

Methods

First, we selected 21 pieces of music from the Billboard Japan Hot 100 chart of 2017, which served as an indicator of preference reflecting the engagement and interest of a large population. To ensure even representation, we chose one piece for every fifth song on the chart, spanning from highly popular music to less popular ones. Next, we recorded EEG signals while the subjects listened to the selected music, and they were asked to evaluate four aspects (preference, enjoyment, frequency of listening, and arousal) for each song. Subsequently, we conducted ISC analysis by utilizing the first three principal components of EEG, which were highly correlated across subjects and extracted through correlated component analysis (CorrCA). We then explored whether music with high preferences that reflected the engagement and interest of large population had high ISC values. Additionally, we employed cluster analysis on all 21 pieces of music, utilizing the first three principal components of EEG, to investigate the impact of emotions and musical characteristics on EEG ISC during music listening.

Results

A significant distinction was noted between the mean ISC values of the 10 higher-ranked pieces of music compared to the 10 lower-ranked pieces of music [t(542) = -1.97, p = 0.0025]. This finding suggests that ISC values may correspond preferences reflecting engagement or interest of a large population. Furthermore, we found that significant variations were observed in the first three principal component values among the three clusters identified through cluster analysis, along with significant differences in arousal levels. Moreover, the characteristics of the music (tonality and tempo) differed among the three clusters. This indicates that the principal components, which exhibit high correlation among subjects and were employed in calculating ISC values, represent both subjects' arousal levels and specific characteristics of the music.

Conclusion

Subjects' arousal values during music listening and music characteristics (tonality and tempo) affect ISC values, which represent the interest of a large population in music.

Neural synchronization as a function of engagement with the narrative

We can all agree that a good story engages us, however, agreeing which story is good is far more debatable. In this study, we explored whether engagement with a narrative synchronizes listeners' brain responses, by examining individual differences in engagement to the same story. To do so, we pre-registered and re-analyzed a previously collected dataset by Chang et al. (2021) of functional Magnetic Resonance Imaging (fMRI) scans of 25 participants who listened to a one-hour story and answered questionnaires. We assessed the degree of their overall engagement with the story and their engagement with the main characters. The questionnaires revealed individual differences in engagement with the story, as well as different valence towards specific characters. Neuroimaging data showed that the auditory cortex, the default mode network (DMN) and language regions were involved in processing the story. Increased engagement with the story was correlated with increased neural synchronization within regions in the DMN (especially the medial prefrontal cortex), as well as regions outside the DMN such as the dorso-lateral prefrontal cortex and the reward system. Interestingly, positively and negatively engaging characters elicited different patterns of neural synchronization. Finally, engagement increased functional connectivity within and between the DMN, the ventral attention network and the control network. Taken together, these findings suggest that engagement with a narrative synchronizes listeners' responses in regions involved in mentalizing, reward, working memory and attention. By examining individual differences in engagement, we revealed that these synchronization patterns are due to engagement, and not due to differences in the narrative's content.

Exploring the Neural Processes behind Narrative Engagement: An EEG Study

Past cognitive neuroscience studies using naturalistic stimuli have considered narratives holistically and focused on cognitive processes. In this study, we incorporated the narrative structure, the dramatic arc, as an object of investigation, to examine how engagement levels fluctuate across a narrative-aligned dramatic arc. We explored the possibility of predicting self-reported engagement ratings from neural activity and investigated the idiosyncratic effects of each phase of the dramatic arc on brain responses as well as the relationship between engagement and brain responses. We presented a movie excerpt following the six-phase narrative arc structure to female and male participants while collecting EEG signals. We then asked this group of participants to recall the excerpt, another group to segment the video based on the dramatic arc model, and a third to rate their engagement levels while watching the movie. The results showed that the self-reported engagement ratings followed the pattern of the narrative dramatic arc. Moreover, while EEG amplitude could not predict group-averaged engagement ratings, other features comprising dynamic intersubject correlation (dISC), including certain frequency bands, dynamic functional connectivity patterns and graph features were able to achieve this. Furthermore, neural activity in the last two phases of the dramatic arc significantly predicted engagement patterns. This study is the first to explore the cognitive processes behind the dramatic arc and its phases. By demonstrating how neural activity predicts self-reported engagement, which itself aligns with the narrative structure, this study provides insights on the interrelationships between narrative structure, neural responses, and viewer engagement.

Physiological synchrony in electrodermal activity predicts decreased vigilant attention induced by sleep deprivation

Introduction

When multiple individuals are presented with narrative movie or audio clips, their electrodermal activity (EDA) and heart rate show significant similarities. Higher levels of such inter-subject physiological synchrony are related with higher levels of attention toward the narrative, as for instance expressed by more correctly answered questions about the narrative. We here investigate whether physiological synchrony in EDA and heart rate during watching of movie clips predicts performance on a subsequent vigilant attention task among participants exposed to a night of total sleep deprivation.

Methods

We recorded EDA and heart rate of 54 participants during a night of total sleep deprivation. Every hour from 22:00 to 07:00 participants watched a 10-min movie clip during which we computed inter-subject physiological synchrony. Afterwards, they answered questions about the movie and performed the psychomotor vigilance task (PVT) to capture attentional performance.

Results

We replicated findings that inter-subject correlations in EDA and heart rate predicted the number of correct answers on questions about the movie clips. Furthermore, we found that inter-subject correlations in EDA, but not in heart rate, predicted PVT performance. Individuals' mean EDA and heart rate also predicted their PVT performance. For EDA, inter-subject correlations explained more variance of PVT performance than individuals' mean EDA.

Discussion

Together, these findings confirm the association between physiological synchrony and attention. Physiological synchrony in EDA does not only capture the attentional processing during the time that it is determined, but also proves valuable for capturing more general changes in the attentional state of monitored individuals.

Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals

Introduction

The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels.

Methods

Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants' engagement levels were calculated based on the composite of their self-reported attention and immersion scores.

Results

Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement.

Discussion

We believe that these findings are a step toward the elucidation of the viewers' way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.

Audience immersion: validating attentional and physiological measures against self-report

When an audience member becomes immersed, their attention shifts towards the media and story, and they allocate cognitive resources to represent events and characters. Here, we investigate whether it is possible to measure immersion using continuous behavioural and physiological measures. Using television and film clips, we validated dual-task reaction times, heart rate, and skin conductance against self-reported narrative engagement. We find that reaction times to a secondary task were strongly positively correlated with self-reported immersion: slower reaction times were indicative of greater immersion, particularly emotional engagement. Synchrony in heart rate across participants was associated with self-reported attentional and emotional engagement with the story, although we found no such relationship with skin conductance. These results establish both dual-task reaction times and heart rate as candidate measures for the real-time, continuous, assessment of audience immersion.

Neural mechanisms of expert persuasion on willingness to pay for sugar

Introduction: Sugar consumption is associated with many negative health consequences. It is, therefore, important to understand what can effectively influence individuals to consume less sugar. We recently showed that a healthy eating call by a health expert can significantly decrease the willingness to pay (WTP) for sugar-containing food. Here, we investigate which aspects of neural responses to the same healthy eating call can predict the efficacy of expert persuasion.

Methods: Forty-five healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free and non-edible products, while their electroencephalography (EEG) was recorded. In between the two blocks, they listened to a healthy eating call by a nutritionist emphasizing the risks of sugar consumption.

Results: We found that after listening to the healthy eating call, participants significantly decreased their WTP for sugar-containing products. Moreover, a higher intersubject correlation of EEG (a measure of engagement) during listening to the healthy eating call resulted in a larger decrease in WTP for sugar-containing food. Whether or not a participant’s valuation of a product was highly influenced by the healthy eating call could also be predicted by spatiotemporal patterns of EEG responses to the healthy eating call, using a machine learning classification model. Finally, the healthy eating call increased the amplitude of the P300 component of the visual event-related potential in response to sugar-containing food.

Disussion: Overall, our results shed light on the neural basis of expert persuasion and demonstrate that EEG is a powerful tool to design and assess health-related advertisements before they are released to the public.

Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips

Individuals that pay attention to narrative stimuli show synchronized heart rate (HR) and electrodermal activity (EDA) responses. The degree to which this physiological synchrony occurs is related to attentional engagement. Factors that can influence attention, such as instructions, salience of the narrative stimulus and characteristics of the individual, affect physiological synchrony. The demonstrability of synchrony depends on the amount of data used in the analysis. We investigated how demonstrability of physiological synchrony varies with varying group size and stimulus duration. Thirty participants watched six 10 min movie clips while their HR and EDA were monitored using wearable sensors (Movisens EdaMove 4 and Wahoo Tickr, respectively). We calculated inter-subject correlations as a measure of synchrony. Group size and stimulus duration were varied by using data from subsets of the participants and movie clips in the analysis. We found that for HR, higher synchrony correlated significantly with the number of answers correct for questions about the movie, confirming that physiological synchrony is associated with attention. For both HR and EDA, with increasing amounts of data used, the percentage of participants with significant synchrony increased. Importantly, we found that it did not matter how the amount of data was increased. Increasing the group size or increasing the stimulus duration led to the same results. Initial comparisons with results from other studies suggest that our results do not only apply to our specific set of stimuli and participants. All in all, the current work can act as a guideline for future research, indicating the amount of data minimally needed for robust analysis of synchrony based on inter-subject correlations.

Developmental differences in the perception of naturalistic human movements

Introduction

It is widely believed that we are more attentive towards moving versus static stimuli. However, the neural correlates underlying the perception of human movements have not been extensively investigated in ecologically valid settings, nor has the developmental aspect of this phenomenon. Here, we set forth to investigate how human limb movements displayed in naturalistic videos influence the attentional engagement of children and young adults.

Methods

Thirty-nine healthy participants (4-26 years old) were presented with naturalistic videos featuring human goal-directed movements, while neural activity was recorded using electroencephalography (EEG). Video scenes were automatically annotated as containing arm, leg or no movement, using a machine learning algorithm. The viewers' attentional engagement was quantified by the intersubject correlation of EEG responses evoked by the videos.

Results

Our results demonstrate that scenes featuring limb movements, especially simultaneous arm and leg movements, elicit higher attentional engagement than scenes with no limb movement. Interestingly, this effect was found to diminish with age.

Discussion

Overall, our findings extend previous work on the perception of human motion by implementing naturalistic stimuli in the experimental design and extend the list of factors influencing the viewer's engagement exerted by naturalistic videos.

Neural dynamics during emotional video engagement relate to anxiety

Inter-subject correlations (ISCs) of physiological data can reveal common stimulus-driven processing across subjects. ISC has been applied to passive video viewing in small samples to measure common engagement and emotional processing. Here, in a large sample study of healthy adults (N = 163) who watched an emotional film (The Lion Cage by Charlie Chaplin), we recorded electroencephalography (EEG) across participants and measured ISC in theta, alpha and beta frequency bands. Peak ISC on the emotionally engaging video was observed three-quarters into the film clip, during a time period which potentially elicited a positive emotion of relief. Peak ISC in all frequency bands was focused over centro-parietal electrodes localizing to superior parietal cortex. ISC in both alpha and beta frequencies had a significant inverse relationship with anxiety symptoms. Our study suggests that ISC measured during continuous non-event-locked passive viewing may serve as a useful marker for anxious mood.

Altered Cerebral Processing of Videos in Children with Motor Dysfunction Suggests Broad Embodiment of Perceptual Cognitive Functions

Embodied cognition theory suggests that motor dysfunctions affect cognition. We examined this hypothesis by inspecting whether cerebral processing of movies, featuring both goal-directed movements and content without humans, differ between children with congenital motor dysfunction and healthy controls. Electroencephalography was recorded from 23 healthy children and 23 children with limited or absent arm movement due to either arthrogryposis multiplex congenita or obstetric brachial plexus palsy. Each individual patient exhibited divergent neural responses, disclosed by significantly lower inter-subject correlation (ISC) of brain activity, during the videos compared to the healthy children. We failed to observe associations between this finding and the motor-related content of the various video scenes, suggesting that differences between the patients and controls reflect modulation of perceptual-cognitive processing of videos by upper-limb motor dysfunctions not limited to the watching-mirroring of motor actions. Thus, perceptual-cognitive processes in the brain seem to be more robustly embodied than has previously been thought.

Functional near-infrared spectroscopy imaging of the prefrontal cortex during a naturalistic comedy movie

Naturalistic stimulation (i.e., movies and auditory narratives of some minutes' length) has been a powerful approach to bringing more real-life experiences into laboratory experiments. Data-driven, intersubject correlation (ISC) analysis permits examining to what extent activity in a specific brain region correlates across participants during exposure to a naturalistic stimulus, as well as testing whether neural activity correlates with behavioral measures. Notably, most of the previous research with naturalistic stimuli was conducted using functional fMRI (fMRI). Here, we tested whether a naturalistic approach and the ISC are feasible using functional near-infrared spectroscopy (fNIRS) - the imaging method particularly suited for populations of patients and children. Fifty-three healthy adult participants watched twice a 3-min segment of a Charlie Chaplin movie while we recorded the brain activity on the surface of their prefrontal cortex using fNIRS. In addition, an independent group of 18 participants used a continuous scoring procedure to rate the extent to which they felt that different parts of the movie fragment were funny. Our two findings were as follows. First, we found higher-than-zero ISC in fNIRS signals in the prefrontal cortex lobes, a result that was particularly high in the oxygenated channels during the first repetition of the movie. Second, we found a significant negative correlation between oxygenated brain signals and ratings of the movie's humorousness. In a series of control analyses we demonstrated that this latter correlation could not be explained by various non-humor-related movie sensory properties (e.g., auditory volume and image brightness). The key overall outcome of the present study is that fNIRS in combination with the naturalistic paradigms and the ISC might be a sensitive and powerful research method to explore cognitive processing. Our results also suggest a potential role of the prefrontal cortex in humor appreciation.

Mapping and understanding of correlated electroencephalogram (EEG) responses to the newsvendor problem

Decision-making is one of the most critical activities of human beings. To better understand the underlying neurocognitive mechanism while making decisions under an economic context, we designed a decision-making paradigm based on the newsvendor problem (NP) with two scenarios: low-profit margins as the more challenging scenario and high-profit margins as the less difficult one. The EEG signals were acquired from healthy humans while subjects were performing the task. We adopted the Correlated Component Analysis (CorrCA) method to identify linear combinations of EEG channels that maximize the correlation across subjects ([Formula: see text]) or trials ([Formula: see text]). The inter-subject or inter-trial correlation values (ISC or ITC) of the first three components were estimated to investigate the modulation of the task difficulty on subjects' EEG signals and respective correlations. We also calculated the alpha- and beta-band power of the projection components obtained by the CorrCA to assess the brain responses across multiple task periods. Finally, the CorrCA forward models, which represent the scalp projections of the brain activities by the maximally correlated components, were further translated into source distributions of underlying cortical activity using the exact Low Resolution Electromagnetic Tomography Algorithm (eLORETA). Our results revealed strong and significant correlations in EEG signals among multiple subjects and trials during the more difficult decision-making task than the easier one. We also observed that the NP decision-making and feedback tasks desynchronized the normalized alpha and beta powers of the CorrCA components, reflecting the engagement state of subjects. Source localization results furthermore suggested several sources of neural activities during the NP decision-making process, including the dorsolateral prefrontal cortex, anterior PFC, orbitofrontal cortex, posterior cingulate cortex, and somatosensory association cortex.

Brain connectivity at rest predicts individual differences in normative activity during movie watching

When exposed to the same sensory event, some individuals are bound to have less typical experiences than others. Previous research has investigated this phenomenon by showing that the typicality of one's sensory experience is associated with the typicality of their stimulus-evoked brain activity (as measured by intersubject correlation, or ISC). Individual differences in ISC have recently been attributed to variability in focal neural processing. However, the extent to which these differences reflect purely intra-regional variability versus variation in the brain's baseline ability to transmit information between regions has yet to be established. Here, we show that an individual's degree and spatial distribution of ISC are closely related to their brain's functional organization at rest. Using resting state and movie watching fMRI data from the Human Connectome Project, we reveal that resting state functional connectivity (RSFC) profiles can be used to predict cortex-wide ISC. Similar region-level analyses demonstrate that the levels of ISC exhibited by brain regions during movie watching are associated with their connectivity to other regions at rest, and that the nature of these connectivity-activity relationships varies as a function of regional roles in sensory information processing. Finally, we show that an individual's unique spatial distribution of ISC, independent of its magnitude, is also related to their RSFC profile. These findings contextualize reports of localized individual differences in ISC as potentially reflecting larger, network-level alterations in resting brain function and detail how the brain's ability to process complex sensory information is linked to its baseline functional organization.

Ear-EEG Measures of Auditory Attention to Continuous Speech

Auditory attention is an important cognitive function used to separate relevant from irrelevant auditory information. However, most findings on attentional selection have been obtained in highly controlled laboratory settings using bulky recording setups and unnaturalistic stimuli. Recent advances in electroencephalography (EEG) facilitate the measurement of brain activity outside the laboratory, and around-the-ear sensors such as the cEEGrid promise unobtrusive acquisition. In parallel, methods such as speech envelope tracking, intersubject correlations and spectral entropy measures emerged which allow us to study attentional effects in the neural processing of natural, continuous auditory scenes. In the current study, we investigated whether these three attentional measures can be reliably obtained when using around-the-ear EEG. To this end, we analyzed the cEEGrid data of 36 participants who attended to one of two simultaneously presented speech streams. Speech envelope tracking results confirmed a reliable identification of the attended speaker from cEEGrid data. The accuracies in identifying the attended speaker increased when fitting the classification model to the individual. Artifact correction of the cEEGrid data with artifact subspace reconstruction did not increase the classification accuracy. Intersubject correlations were higher for those participants attending to the same speech stream than for those attending to different speech streams, replicating previously obtained results with high-density cap-EEG. We also found that spectral entropy decreased over time, possibly reflecting the decrease in the listener's level of attention. Overall, these results support the idea of using ear-EEG measurements to unobtrusively monitor auditory attention to continuous speech. This knowledge may help to develop assistive devices that support listeners separating relevant from irrelevant information in complex auditory environments.

Interbrain synchrony: on wavy ground

In recent years the study of dynamic, between-brain coupling mechanisms has taken social neuroscience by storm. In particular, interbrain synchrony (IBS) is a putative neural mechanism said to promote social interactions by enabling the functional integration of multiple brains. In this article, I argue that this research is beset with three pervasive and interrelated problems. First, the field lacks a widely accepted definition of IBS. Second, IBS wants for theories that can guide the design and interpretation of experiments. Third, a potpourri of tasks and empirical methods permits undue flexibility when testing the hypothesis. These factors synergistically undermine IBS as a theoretical construct. I finish by recommending measures that can address these issues.

Brain networks are decoupled from external stimuli during internal cognition

Our cognition can be directed to external stimuli or to internal information. While there are many different forms of internal cognition (mind-wandering, recall, imagery etc.), their essential feature is independence from the immediate sensory input, conceptually referred to as perceptual decoupling. Perceptual decoupling is thought to be reflected in brain activity transitioning from a stimulus-processing to internally-processing mode, but a direct investigation of this remains outstanding. Here we present a conceptual and analysis framework that quantifies the extent to which brain networks reflect stimulus processing. We tested this framework by presenting subjects with an audiovisual stimulus and instructing them to either attend to the stimulus (external task) or engage in mental imagery, recall or arithmetic (internal tasks) while measuring the evoked brain activity using functional MRI. We found that stimulus responses were generally attenuated for the internal tasks, though they increased in a subset of tasks and brain networks. However, using our new framework, we showed that brain networks became less reflective of stimulus processing, even in the subset of tasks and brain networks in which stimulus responses increased. These results quantitatively demonstrate that during internal cognition brain networks become decoupled from the external stimuli, opening the door for a fundamental and quantitative understanding of internal cognition.

Age-related deficits in dip-listening evident for isolated sentences but not for spoken stories

Fluctuating background sounds facilitate speech intelligibility by providing speech ‘glimpses’ (masking release). Older adults benefit less from glimpses, but masking release is typically investigated using isolated sentences. Recent work indicates that using engaging, continuous speech materials (e.g., spoken stories) may qualitatively alter speech-in-noise listening. Moreover, neural sensitivity to different amplitude envelope profiles (ramped, damped) changes with age, but whether this affects speech listening is unknown. In three online experiments, we investigate how masking release in younger and older adults differs for masked sentences and stories, and how speech intelligibility varies with masker amplitude profile. Intelligibility was generally greater for damped than ramped maskers. Masking release was reduced in older relative to younger adults for disconnected sentences, and stories with a randomized sentence order. Critically, when listening to stories with an engaging and coherent narrative, older adults demonstrated equal or greater masking release compared to younger adults. Older adults thus appear to benefit from ‘glimpses’ as much as, or more than, younger adults when the speech they are listening to follows a coherent topical thread. Our results highlight the importance of cognitive and motivational factors for speech understanding, and suggest that previous work may have underestimated speech-listening abilities in older adults.

Asymmetrical cross-modal influence on neural encoding of auditory and visual features in natural scenes

Natural scenes contain multi-modal information, which is integrated to form a coherent perception. Previous studies have demonstrated that cross-modal information can modulate neural encoding of low-level sensory features. These studies, however, mostly focus on the processing of single sensory events or rhythmic sensory sequences. Here, we investigate how the neural encoding of basic auditory and visual features is modulated by cross-modal information when the participants watch movie clips primarily composed of non-rhythmic events. We presented audiovisual congruent and audiovisual incongruent movie clips, and since attention can modulate cross-modal interactions, we separately analyzed high- and low-arousal movie clips. We recorded neural responses using electroencephalography (EEG), and employed the temporal response function (TRF) to quantify the neural encoding of auditory and visual features. The neural encoding of sound envelope is enhanced in the audiovisual congruent condition than the incongruent condition, but this effect is only significant for high-arousal movie clips. In contrast, audiovisual congruency does not significantly modulate the neural encoding of visual features, e.g., luminance or visual motion. In summary, our findings demonstrate asymmetrical cross-modal interactions during the processing of natural scenes that lack rhythmicity: Congruent visual information enhances low-level auditory processing, while congruent auditory information does not significantly modulate low-level visual processing.

Neural Activity during Story Listening Is Synchronized across Individuals Despite Acoustic Masking

Older people with hearing problems often experience difficulties understanding speech in the presence of background sound. As a result, they may disengage in social situations, which has been associated with negative psychosocial health outcomes. Measuring listening (dis)engagement during challenging listening situations has received little attention thus far. We recruit young, normal-hearing human adults (both sexes) and investigate how speech intelligibility and engagement during naturalistic story listening is affected by the level of acoustic masking (12-talker babble) at different signal-to-noise ratios (SNRs). In Experiment 1, we observed that word-report scores were above 80% for all but the lowest SNR (-3 dB SNR) we tested, at which performance dropped to 54%. In Experiment 2, we calculated intersubject correlation (ISC) using EEG data to identify dynamic spatial patterns of shared neural activity evoked by the stories. ISC has been used as a neural measure of participants' engagement with naturalistic materials. Our results show that ISC was stable across all but the lowest SNRs, despite reduced speech intelligibility. Comparing ISC and intelligibility demonstrated that word-report performance declined more strongly with decreasing SNR compared to ISC. Our measure of neural engagement suggests that individuals remain engaged in story listening despite missing words because of background noise. Our work provides a potentially fruitful approach to investigate listener engagement with naturalistic, spoken stories that may be used to investigate (dis)engagement in older adults with hearing impairment.

Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain-body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.

Unsupervised Clustering of Individuals Sharing Selective Attentional Focus Using Physiological Synchrony

Research on brain signals as indicators of a certain attentional state is moving from laboratory environments to everyday settings. Uncovering the attentional focus of individuals in such settings is challenging because there is usually limited information about real-world events, as well as a lack of data from the real-world context at hand that is correctly labeled with respect to individuals' attentional state. In most approaches, such data is needed to train attention monitoring models. We here investigate whether unsupervised clustering can be combined with physiological synchrony in the electroencephalogram (EEG), electrodermal activity (EDA), and heart rate to automatically identify groups of individuals sharing attentional focus without using knowledge of the sensory stimuli or attentional focus of any of the individuals. We used data from an experiment in which 26 participants listened to an audiobook interspersed with emotional sounds and beeps. Thirteen participants were instructed to focus on the narrative of the audiobook and 13 participants were instructed to focus on the interspersed emotional sounds and beeps. We used a broad range of commonly applied dimensionality reduction ordination techniques-further referred to as mappings-in combination with unsupervised clustering algorithms to identify the two groups of individuals sharing attentional focus based on physiological synchrony. Analyses were performed using the three modalities EEG, EDA, and heart rate separately, and using all possible combinations of these modalities. The best unimodal results were obtained when applying clustering algorithms on physiological synchrony data in EEG, yielding a maximum clustering accuracy of 85%. Even though the use of EDA or heart rate by itself did not lead to accuracies significantly higher than chance level, combining EEG with these measures in a multimodal approach generally resulted in higher classification accuracies than when using only EEG. Additionally, classification results of multimodal data were found to be more consistent across algorithms than unimodal data, making algorithm choice less important. Our finding that unsupervised classification into attentional groups is possible is important to support studies on attentional engagement in everyday settings.

Inter-subject Correlation While Listening to Minimalist Music: A Study of Electrophysiological and Behavioral Responses to Steve Reich's Piano Phase

Musical minimalism utilizes the temporal manipulation of restricted collections of rhythmic, melodic, and/or harmonic materials. One example, Steve Reich's Piano Phase, offers listeners readily audible formal structure with unpredictable events at the local level. For example, pattern recurrences may generate strong expectations which are violated by small temporal and pitch deviations. A hyper-detailed listening strategy prompted by these minute deviations stands in contrast to the type of listening engagement typically cultivated around functional tonal Western music. Recent research has suggested that the inter-subject correlation (ISC) of electroencephalographic (EEG) responses to natural audio-visual stimuli objectively indexes a state of “engagement,” demonstrating the potential of this approach for analyzing music listening. But can ISCs capture engagement with minimalist music, which features less obvious expectation formation and has historically received a wide range of reactions? To approach this question, we collected EEG and continuous behavioral (CB) data while 30 adults listened to an excerpt from Steve Reich's Piano Phase, as well as three controlled manipulations and a popular-music remix of the work. Our analyses reveal that EEG and CB ISC are highest for the remix stimulus and lowest for our most repetitive manipulation, no statistical differences in overall EEG ISC between our most musically meaningful manipulations and Reich's original piece, and evidence that compositional features drove engagement in time-resolved ISC analyses. We also found that aesthetic evaluations corresponded well with overall EEG ISC. Finally we highlight co-occurrences between stimulus events and time-resolved EEG and CB ISC. We offer the CB paradigm as a useful analysis measure and note the value of minimalist compositions as a limit case for the neuroscientific study of music listening. Overall, our participants' neural, continuous behavioral, and question responses showed strong similarities that may help refine our understanding of the type of engagement indexed by ISC for musical stimuli.

Integrating Cognitive Developmental Neuroscience in Society: Lessons Learned From a Multidisciplinary Research Project on Education and Social Safety of Youth

Integrating fundamental science in society, with the goal to translate research findings to daily practice, comes with certain challenges. Successfully integrating research projects into society requires (1) good collaboration between scientists and societal stakeholders, (2) collaboration partners with common expectations and goals, and (3) investment in clear communication. Here we describe an integrative research project conducted by a large Dutch consortium that consisted of neuroscientists, psychologists, sociologists, ethicists, teachers, health care professionals and policy makers, focusing on applying cognitive developmental neuroscience for the benefit of youth in education and social safety. We argue that to effectively integrate cognitive developmental neuroscience in society, (1) it is necessary to invest in a well-functioning, diverse and multidisciplinary team involving societal stakeholders and youth themselves from the start of the project. This aids to build a so-called productive interactive network that increases the chances to realize societal impact in the long-term. Additionally, we propose that to integrate knowledge, (2) a different than standard research approach should be taken. When focusing on integration, the ultimate goal of research is not solely to understand the world better, but also to intervene with real-life situations, such as education or (forensic) youth care. To accomplish this goal, we propose an approach in which integration is not only started after the research has been conducted, but taken into account throughout the entire project. This approach helps to create common expectations and goals between different stakeholders. Finally, we argue that (3) dedicating sufficient resources to effective communication, both within the consortium and between scientists and society, greatly benefits the integration of cognitive developmental neuroscience in society.

Cognitive and Neural State Dynamics of Narrative Comprehension

Narrative comprehension involves a constant interplay of the accumulation of incoming events and their integration into a coherent structure. This study characterizes cognitive states during narrative comprehension and the network-level reconfiguration occurring dynamically in the functional brain. We presented movie clips of temporally scrambled sequences to human participants (male and female), eliciting fluctuations in the subjective feeling of comprehension. Comprehension occurred when processing events that were highly causally related to the previous events, suggesting that comprehension entails the integration of narratives into a causally coherent structure. The functional neuroimaging results demonstrated that the integrated and efficient brain state emerged during the moments of narrative integration with the increased level of activation and across-modular connections in the default mode network. Underlying brain states were synchronized across individuals when comprehending novel narratives, with increased occurrences of the default mode network state, integrated with sensory processing network, during narrative integration. A model based on time-resolved functional brain connectivity predicted changing cognitive states related to comprehension that are general across narratives. Together, these results support adaptive reconfiguration and interaction of the functional brain networks on causal integration of the narratives.SIGNIFICANCE STATEMENT The human brain can integrate temporally disconnected pieces of information into coherent narratives. However, the underlying cognitive and neural mechanisms of how the brain builds a narrative representation remain largely unknown. We showed that comprehension occurs as the causally related events are integrated to form a coherent situational model. Using fMRI, we revealed that the large-scale brain states and interaction between brain regions dynamically reconfigure as comprehension evolves, with the default mode network playing a central role during moments of narrative integration. Overall, the study demonstrates that narrative comprehension occurs through a dynamic process of information accumulation and causal integration, supported by the time-varying reconfiguration and brain network interaction.

Neural responses to natural visual motion are spatially selective across the visual field, with selectivity differing across brain areas and task

It is well established that neural responses to visual stimuli are enhanced at select locations in the visual field. Although spatial selectivity and the effects of spatial attention are well understood for discrete tasks (e.g. visual cueing), little is known for naturalistic experience that involves continuous dynamic visual stimuli (e.g. driving). Here, we assess the strength of neural responses across the visual space during a kart‐race game. Given the varying relevance of visual location in this task, we hypothesized that the strength of neural responses to movement will vary across the visual field, and it would differ between active play and passive viewing. To test this, we measure the correlation strength of scalp‐evoked potentials with optical flow magnitude at individual locations on the screen. We find that neural responses are strongly correlated at task‐relevant locations in visual space, extending beyond the focus of overt attention. Although the driver's gaze is directed upon the heading direction at the centre of the screen, neural responses were robust at the peripheral areas (e.g. roads and surrounding buildings). Importantly, neural responses to visual movement are broadly distributed across the scalp, with visual spatial selectivity differing across electrode locations. Moreover, during active gameplay, neural responses are enhanced at select locations in the visual space. Conventionally, spatial selectivity of neural response has been interpreted as an attentional gain mechanism. In the present study, the data suggest that different brain areas focus attention on different portions of the visual field that are task‐relevant, beyond the focus of overt attention.

Conscious processing of narrative stimuli synchronizes heart rate between individuals

Heart rate has natural fluctuations that are typically ascribed to autonomic function. Recent evidence suggests that conscious processing can affect the timing of the heartbeat. We hypothesized that heart rate is modulated by conscious processing and therefore dependent on attentional focus. To test this, we leverage the observation that neural processes synchronize between subjects by presenting an identical narrative stimulus. As predicted, we find significant inter-subject correlation of heart rate (ISC-HR) when subjects are presented with an auditory or audiovisual narrative. Consistent with our hypothesis, we find that ISC-HR is reduced when subjects are distracted from the narrative, and higher ISC-HR predicts better recall of the narrative. Finally, patients with disorders of consciousness have lower ISC-HR, as compared to healthy individuals. We conclude that heart rate fluctuations are partially driven by conscious processing, depend on attentional state, and may represent a simple metric to assess conscious state in unresponsive patients.

Teacher-Student Neural Coupling During Teaching and Learning

Human communication is remarkably versatile, enabling teachers to share highly abstracted and novel information with their students. What neural processes enable such transfer of information across brains during naturalistic teaching and learning? Here, a teacher was scanned in functional magnetic resonance imaging while giving an oral lecture with slides on a scientific topic followed by a review lecture. Students were then scanned while watching either the intact Lecture and Review (N = 20) or a temporally scrambled version of the lecture (N = 20). Using intersubject correlation, we observed widespread Teacher–Student neural coupling spanning sensory cortex and language regions along the superior temporal sulcus as well as higher-level regions including posterior medial cortex (PMC), superior parietal lobule, and dorsolateral and dorsomedial prefrontal cortex. Teacher–student alignment in higher-level areas was not observed when learning was disrupted by temporally scrambling the lecture. Moreover, teacher–student coupling in PMC was significantly correlated with learning: the more closely the student’s brain mirrored the teacher’s brain, the more the student improved their learning score. Together, these results suggest that the alignment of neural responses between teacher and students may reflect effective communication of complex information across brains in classroom settings.

Neural signatures of attentional engagement during narratives and its consequences for event memory

As we comprehend narratives, our attentional engagement fluctuates over time. Despite theoretical conceptions of narrative engagement as emotion-laden attention, little empirical work has characterized the cognitive and neural processes that comprise subjective engagement in naturalistic contexts or its consequences for memory. Here, we relate fluctuations in narrative engagement to patterns of brain coactivation and test whether neural signatures of engagement predict subsequent memory. In behavioral studies, participants continuously rated how engaged they were as they watched a television episode or listened to a story. Self-reported engagement was synchronized across individuals and driven by the emotional content of the narratives. In functional MRI datasets collected as different individuals watched the same show or listened to the same story, engagement drove neural synchrony, such that default mode network activity was more synchronized across individuals during more engaging moments of the narratives. Furthermore, models based on time-varying functional brain connectivity predicted evolving states of engagement across participants and independent datasets. The functional connections that predicted engagement overlapped with a validated neuromarker of sustained attention and predicted recall of narrative events. Together, our findings characterize the neural signatures of attentional engagement in naturalistic contexts and elucidate relationships among narrative engagement, sustained attention, and event memory.

Decreased intersubject synchrony in dynamic valence ratings of sad movie contents in dysphoric individuals

Emotional reactions to movies are typically similar between people. However, depressive symptoms decrease synchrony in brain responses. Less is known about the effect of depressive symptoms on intersubject synchrony in conscious stimulus-related processing. In this study, we presented amusing, sad and fearful movie clips to dysphoric individuals (those with elevated depressive symptoms) and control participants to dynamically rate the clips' valences (positive vs. negative). We analysed both the valence ratings' mean values and intersubject correlation (ISC). We used electrodermal activity (EDA) to complement the measurement in a separate session. There were no group differences in either the EDA or mean valence rating values for each movie type. As expected, the valence ratings' ISC was lower in the dysphoric than the control group, specifically for the sad movie clips. In addition, there was a negative relationship between the valence ratings' ISC and depressive symptoms for sad movie clips in the full sample. The results are discussed in the context of the negative attentional bias in depression. The findings extend previous brain activity results of ISC by showing that depressive symptoms also increase variance in conscious ratings of valence of stimuli in a mood-congruent manner.

EEG-Based Intersubject Correlations Reflect Selective Attention in a Competing Speaker Scenario

Several solutions have been proposed to study the relationship between ongoing brain activity and natural sensory stimuli, such as running speech. Computing the intersubject correlation (ISC) has been proposed as one possible approach. Previous evidence suggests that ISCs between the participants' electroencephalogram (EEG) may be modulated by attention. The current study addressed this question in a competing-speaker paradigm, where participants (N = 41) had to attend to one of two concurrently presented speech streams. ISCs between participants' EEG were higher for participants attending to the same story compared to participants attending to different stories. Furthermore, we found that ISCs between individual and group data predicted whether an individual attended to the left or right speech stream. Interestingly, the magnitude of the shared neural response with others attending to the same story was related to the individual neural representation of the attended and ignored speech envelope. Overall, our findings indicate that ISC differences reflect the magnitude of selective attentional engagement to speech.

Cortical response to naturalistic stimuli is largely predictable with deep neural networks

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions.

MEG Intersubject Phase Locking of Stimulus-Driven Activity during Naturalistic Speech Listening Correlates with Musical Training

Musical training is associated with increased structural and functional connectivity between auditory sensory areas and higher-order brain networks involved in speech and motor processing. Whether such changed connectivity patterns facilitate the cortical propagation of speech information in musicians remains poorly understood. We here used magnetoencephalography (MEG) source imaging and a novel seed-based intersubject phase-locking approach to investigate the effects of musical training on the interregional synchronization of stimulus-driven neural responses during listening to naturalistic continuous speech presented in silence. MEG data were obtained from 20 young human subjects (both sexes) with different degrees of musical training. Our data show robust bilateral patterns of stimulus-driven interregional phase synchronization between auditory cortex and frontotemporal brain regions previously associated with speech processing. Stimulus-driven phase locking was maximal in the delta band, but was also observed in the theta and alpha bands. The individual duration of musical training was positively associated with the magnitude of stimulus-driven alpha-band phase locking between auditory cortex and parts of the dorsal and ventral auditory processing streams. These findings provide evidence for a positive relationship between musical training and the propagation of speech-related information between auditory sensory areas and higher-order processing networks, even when speech is presented in silence. We suggest that the increased synchronization of higher-order cortical regions to auditory cortex may contribute to the previously described musician advantage in processing speech in background noise.SIGNIFICANCE STATEMENT Musical training has been associated with widespread structural and functional brain plasticity. It has been suggested that these changes benefit the production and perception of music but can also translate to other domains of auditory processing, such as speech. We developed a new magnetoencephalography intersubject analysis approach to study the cortical synchronization of stimulus-driven neural responses during the perception of continuous natural speech and its relationship to individual musical training. Our results provide evidence that musical training is associated with higher synchronization of stimulus-driven activity between brain regions involved in early auditory sensory and higher-order processing. We suggest that the increased synchronized propagation of speech information may contribute to the previously described musician advantage in processing speech in background noise.

The default mode network: where the idiosyncratic self meets the shared social world

The default mode network (DMN) is classically considered an 'intrinsic' system, specializing in internally oriented cognitive processes such as daydreaming, reminiscing and future planning. In this Perspective, we suggest that the DMN is an active and dynamic 'sense-making' network that integrates incoming extrinsic information with prior intrinsic information to form rich, context-dependent models of situations as they unfold over time. We review studies that relied on naturalistic stimuli, such as stories and movies, to demonstrate how an individual's DMN neural responses are influenced both by external information accumulated as events unfold over time and by the individual's idiosyncratic past memories and knowledge. The integration of extrinsic and intrinsic information over long timescales provides a space for negotiating a shared neural code, which is necessary for establishing shared meaning, shared communication tools, shared narratives and, above all, shared communities and social networks.

Synchronized eye movements predict test scores in online video education

Experienced teachers pay close attention to their students, adjusting their teaching when students seem lost. This dynamic interaction is missing in online education. We hypothesized that attentive students follow videos similarly with their eyes. Thus, attention to instructional videos could be assessed remotely by tracking eye movements. Here we show that intersubject correlation of eye movements during video presentation is substantially higher for attentive students and that synchronized eye movements are predictive of individual test scores on the material presented in the video. These findings replicate for videos in a variety of production styles, for incidental and intentional learning and for recall and comprehension questions alike. We reproduce the result using standard web cameras to capture eye movements in a classroom setting and with over 1,000 participants at home without the need to transmit user data. Our results suggest that online education could be made adaptive to a student's level of attention in real time.

Wisdom of crowds benefits perceptual decision making across difficulty levels

Decades of research on collective decision making has claimed that aggregated judgment of multiple individuals is more accurate than expert individual judgement. A longstanding problem in this regard has been to determine how decisions of individuals can be combined to form intelligent group decisions. Our study consisted of a random target detection task in natural scenes, where human subjects (18 subjects, 7 female) detected the presence or absence of a random target as indicated by the cue word displayed prior to stimulus display. Concurrently the neural activities (EEG signals) were recorded. A separate behavioural experiment was performed by different subjects (20 subjects, 11 female) on the same set of images to categorize the tasks according to their difficulty levels. We demonstrate that the weighted average of individual decision confidence/neural decision variables produces significantly better performance than the frequently used majority pooling algorithm. Further, the classification error rates from individual judgement were found to increase with increasing task difficulty. This error could be significantly reduced upon combining the individual decisions using group aggregation rules. Using statistical tests, we show that combining all available participants is unnecessary to achieve minimum classification error rate. We also try to explore if group aggregation benefits depend on the correlation between the individual judgements of the group and our results seem to suggest that reduced inter-subject correlation can improve collective decision making for a fixed difficulty level.

Morning brain: real-world neural evidence that high school class times matter

Researchers, parents and educators consistently observe a stark mismatch between biologically preferred and socially imposed sleep-wake hours in adolescents, fueling debate about high school start times. We contribute neural evidence to this debate with electroencephalogram data collected from high school students during their regular morning, mid-morning and afternoon classes. Overall, student alpha power was lower when class content was taught via videos than through lectures. Students' resting state alpha brain activity decreased as the day progressed, consistent with adolescents being least attentive early in the morning. During the lessons, students showed consistently worse performance and higher alpha power for early morning classes than for mid-morning classes, while afternoon quiz scores and alpha levels varied. Together, our findings demonstrate that both class activity and class time are reflected in adolescents' brain states in a real-world setting, and corroborate educational research suggesting that mid-morning may be the best time to learn.

Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Detects Attentionally Relevant Events in Time

Interpersonal physiological synchrony (PS), or the similarity of physiological signals between individuals over time, may be used to detect attentionally engaging moments in time. We here investigated whether PS in the electroencephalogram (EEG), electrodermal activity (EDA), heart rate and a multimodal metric signals the occurrence of attentionally relevant events in time in two groups of participants. Both groups were presented with the same auditory stimulus, but were instructed to attend either to the narrative of an audiobook (audiobook-attending: AA group) or to interspersed emotional sounds and beeps (stimulus-attending: SA group). We hypothesized that emotional sounds could be detected in both groups as they are expected to draw attention involuntarily, in a bottom-up fashion. Indeed, we found this to be the case for PS in EDA or the multimodal metric. Beeps, that are expected to be only relevant due to specific "top-down" attentional instructions, could indeed only be detected using PS among SA participants, for EDA, EEG and the multimodal metric. We further hypothesized that moments in the audiobook accompanied by high PS in either EEG, EDA, heart rate or the multimodal metric for AA participants would be rated as more engaging by an independent group of participants compared to moments corresponding to low PS. This hypothesis was not supported. Our results show that PS can support the detection of attentionally engaging events over time. Currently, the relation between PS and engagement is only established for well-defined, interspersed stimuli, whereas the relation between PS and a more abstract self-reported metric of engagement over time has not been established. As the relation between PS and engagement is dependent on event type and physiological measure, we suggest to choose a measure matching with the stimulus of interest. When the stimulus type is unknown, a multimodal metric is most robust.

Protocol for the Prognostication of Consciousness Recovery Following a Brain Injury

Individuals who have suffered a severe brain injury typically require extensive hospitalization in intensive care units (ICUs), where critical treatment decisions are made to maximize their likelihood of recovering consciousness and cognitive function. These treatment decisions can be difficult when the neurological assessment of the patient is limited by unreliable behavioral responses. Reliable objective and quantifiable markers are lacking and there is both (1) a poor understanding of the mechanisms underlying the brain's ability to reconstitute consciousness and cognition after an injury and (2) the absence of a reliable and clinically feasible method of tracking cognitive recovery in ICU survivors. Our goal is to develop and validate a clinically relevant EEG paradigm that can inform the prognosis of unresponsive, brain-injured patients in the ICU. This protocol describes a study to develop a point-of-care system intended to accurately predict outcomes of unresponsive, brain-injured patients in the ICU. We will recruit 200 continuously-sedated brain-injured patients across five ICUs. Between 24 h and 7 days post-ICU admission, high-density EEG will be recorded from behaviorally unresponsive patients before, during and after a brief cessation of pharmacological sedation. Once patients have reached the waking stage, they will be asked to complete an abridged Cambridge Brain Sciences battery, a web-based series of neurocognitive tests. The test series will be repeated every day during acute admission (ICU, ward), or as often as possible given the constraints of ICU and ward care. Following discharge, patients will continue to complete the same test series on weekly, and then monthly basis, for up to 12 months following injury. Functional outcomes will also be assessed up to 12 months post-injury. We anticipate our findings will lead to an increased ability to identify patients, as soon as possible after their brain injury, who are most likely to survive, and to make accurate predictions about their long-term cognitive and functional outcome. In addition to providing critically needed support for clinical decision-making, this study has the potential to transform our understanding of key functional EEG networks associated with consciousness and cognition.

Absorption and Enjoyment During Listening to Acoustically Masked Stories

Comprehension of speech masked by background sound requires increased cognitive processing, which makes listening effortful. Research in hearing has focused on such challenging listening experiences, in part because they are thought to contribute to social withdrawal in people with hearing impairment. Research has focused less on positive listening experiences, such as enjoyment, despite their potential importance in motivating effortful listening. Moreover, the artificial speech materials-such as disconnected, brief sentences-commonly used to investigate speech intelligibility and listening effort may be ill-suited to capture positive experiences when listening is challenging. Here, we investigate how listening to naturalistic spoken stories under acoustic challenges influences the quality of listening experiences. We assess absorption (the feeling of being immersed/engaged in a story), enjoyment, and listening effort and show that (a) story absorption and enjoyment are only minimally affected by moderate speech masking although listening effort increases, (b) thematic knowledge increases absorption and enjoyment and reduces listening effort when listening to a story presented in multitalker babble, and (c) absorption and enjoyment increase and effort decreases over time as individuals listen to several stories successively in multitalker babble. Our research indicates that naturalistic, spoken stories can reveal several concurrent listening experiences and that expertise in a topic can increase engagement and reduce effort. Our work also demonstrates that, although listening effort may increase with speech masking, listeners may still find the experience both absorbing and enjoyable.