Phase reset affects auditory-visual simultaneity judgment

Beyond alpha band: prestimulus local oscillation and interregional synchrony of the beta band shape the temporal perception of the audiovisual beep-flash stimulus

Objective.Multisensory integration is more likely to occur if the multimodal inputs are within a narrow temporal window called temporal binding window (TBW). Prestimulus local neural oscillations and interregional synchrony within sensory areas can modulate cross-modal integration. Previous work has examined the role of ongoing neural oscillations in audiovisual temporal integration, but there is no unified conclusion. This study aimed to explore whether local ongoing neural oscillations and interregional audiovisual synchrony modulate audiovisual temporal integration.Approach.The human participants performed a simultaneity judgment (SJ) task with the beep-flash stimuli while recording electroencephalography. We focused on two stimulus onset asynchrony (SOA) conditions where subjects report ∼50% proportion of synchronous responses in auditory- and visual-leading SOA (A50V and V50A).Main results.We found that the alpha band power is larger in synchronous response in the central-right posterior and posterior sensors in A50V and V50A conditions, respectively. The results suggested that the alpha band power reflects neuronal excitability in the auditory or visual cortex, which can modulate audiovisual temporal perception depending on the leading sense. Additionally, the SJs were modulated by the opposite phases of alpha (5-10 Hz) and low beta (14-20 Hz) bands in the A50V condition while the low beta band (14-18 Hz) in the V50A condition. One cycle of alpha or two cycles of beta oscillations matched an auditory-leading TBW of ∼86 ms, while two cycles of beta oscillations matched a visual-leading TBW of ∼105 ms. This result indicated the opposite phases in the alpha and beta bands reflect opposite cortical excitability, which modulated the audiovisual SJs. Finally, we found stronger high beta (21-28 Hz) audiovisual phase synchronization for synchronous response in the A50V condition. The phase synchrony of the beta band might be related to maintaining information flow between visual and auditory regions in a top-down manner.Significance.These results clarified whether and how the prestimulus brain state, including local neural oscillations and functional connectivity between brain regions, affects audiovisual temporal integration.

Neural Oscillations and Multisensory Processing

Neural oscillations play a role in sensory processing by coordinating synchronized neuronal activity. Synchronization of gamma oscillations is engaged in local computation of feedforward signals and synchronization of alpha-beta oscillations is engaged in feedback processing over long-range areas. These spatially and spectrally segregated bi-directional signals may be integrated by a mechanism of cross-frequency coupling. Synchronization of neural oscillations has also been proposed as a mechanism for information integration across multiple sensory modalities. A transient stimulus or rhythmic stimulus from one modality may lead to phase alignment of ongoing neural oscillations in multiple sensory cortices, through a mechanism of cross-modal phase reset or cross-modal neural entrainment. Synchronized activities in multiple sensory cortices are more likely to boost stronger activities in downstream areas. Compared to synchronized oscillations, asynchronized oscillations may impede signal processing, and may contribute to sensory selection by setting the oscillations in the target-related cortex and the oscillations in the distractor-related cortex to opposite phases.

The impact of cardiac phases on multisensory integration

The brain continuously processes information coming from both the external environment and visceral signals generated by the body. This constant information exchange between the body and the brain allows signals originating from the oscillatory activity of the heart, among others, to influence perception. Here, we investigated how the cardiac phase modulates multisensory integration, which is the process that allows information from multiple senses to combine non-linearly to reduce environmental uncertainty. Forty healthy participants completed a Simple Detection Task with unimodal (Auditory, Visual, Tactile) and bimodal (Audio-Tactile, Audio-Visual, Visuo-Tactile) stimuli presented 250 ms and 500 ms after the R-peak of the electrocardiogram, that is, systole and diastole, respectively. First, we found a nonspecific effect of the cardiac cycle phases on detection of both unimodal and bimodal stimuli. Reaction times were faster for stimuli presented during diastole, compared to systole. Then, applying the Race Model Inequality approach to quantify multisensory integration, Audio-Tactile and Visuo-Tactile, but not Audio-Visual stimuli, showed higher integration when presented during diastole than during systole. These findings indicate that the impact of the cardiac phase on multisensory integration may be specific for stimuli including somatosensory (i.e., tactile) inputs. This suggests that the heartbeat-related noise, which according to the interoceptive predictive coding theory suppresses somatosensory inputs, also affects multisensory integration during systole. In conclusion, our data extend the interoceptive predictive coding theory to the multisensory domain. From a more mechanistic view, they may reflect a reduced optimization of neural oscillations orchestrating multisensory integration during systole.

Empirical mode decomposition of local field potential data from optogenetic experiments

Introduction

This study investigated the effects of cocaine administration and parvalbumin-type interneuron stimulation on local field potentials (LFPs) recorded in vivo from the medial prefrontal cortex (mPFC) of six mice using optogenetic tools.

Methods

The local network was subject to a brief 10 ms laser pulse, and the response was recorded for 2 s over 100 trials for each of the six subjects who showed stable coupling between the mPFC and the optrode. Due to the strong non-stationary and nonlinearity of the LFP, we used the adaptive, data-driven, Empirical Mode Decomposition (EMD) method to decompose the signal into orthogonal Intrinsic Mode Functions (IMFs).

Results

Through trial and error, we found that seven is the optimum number of orthogonal IMFs that overlaps with known frequency bands of brain activity. We found that the Index of Orthogonality (IO) of IMF amplitudes was close to zero. The Index of Energy Conservation (IEC) for each decomposition was close to unity, as expected for orthogonal decompositions. We found that the power density distribution vs. frequency follows a power law with an average scaling exponent of ~1.4 over the entire range of IMF frequencies 2-2,000 Hz.

Discussion

The scaling exponent is slightly smaller for cocaine than the control, suggesting that neural activity avalanches under cocaine have longer life spans and sizes.

Neural dynamics driving audio-visual integration in autism

Audio-visual (AV) integration plays a crucial role in supporting social functions and communication in autism spectrum disorder (ASD). However, behavioral findings remain mixed and, importantly, little is known about the underlying neurophysiological bases. Studies in neurotypical adults indicate that oscillatory brain activity in different frequencies subserves AV integration, pointing to a central role of (i) individual alpha frequency (IAF), which would determine the width of the cross-modal binding window; (ii) pre-/peri-stimulus theta oscillations, which would reflect the expectation of AV co-occurrence; (iii) post-stimulus oscillatory phase reset, which would temporally align the different unisensory signals. Here, we investigate the neural correlates of AV integration in children with ASD and typically developing (TD) peers, measuring electroencephalography during resting state and in an AV integration paradigm. As for neurotypical adults, AV integration dynamics in TD children could be predicted by the IAF measured at rest and by a modulation of anticipatory theta oscillations at single-trial level. Conversely, in ASD participants, AV integration/segregation was driven exclusively by the neural processing of the auditory stimulus and the consequent auditory-induced phase reset in visual regions, suggesting that a disproportionate elaboration of the auditory input could be the main factor characterizing atypical AV integration in autism.

Individual differences in beta frequency correlate with the audio-visual fusion illusion

Presenting one flash with two beeps induces a perception of two flashes (audio-visual [AV] fission illusion), while presenting two flashes with one beep induces a perception of one flash (fusion illusion). Although previous studies showed a relationship between the frequency of the alpha rhythm (alpha cycle) and one's susceptibility to the fission illusion, the relationship between neural oscillations and the fusion illusion is unknown. Using electroencephalography, here I investigated the frequency of oscillatory signals in the pre-stimulus period and found a significant correlation between the beta rhythm and the fusion illusion; specifically, participants with a lower beta frequency showed a larger fusion illusion. These data indicate two separate time windows of AV integration in the human brain, one defined by the alpha cycle (fission) and another defined by the beta cycle (fusion).

Individual Differences in Multisensory Processing Are Related to Broad Differences in the Balance of Local versus Distributed Information

The brain's ability to extract information from multiple sensory channels is crucial to perception and effective engagement with the environment, but the individual differences observed in multisensory processing lack mechanistic explanation. We hypothesized that, from the perspective of information theory, individuals with more effective multisensory processing will exhibit a higher degree of shared information among distributed neural populations while engaged in a multisensory task, representing more effective coordination of information among regions. To investigate this, healthy young adults completed an audiovisual simultaneity judgment task to measure their temporal binding window (TBW), which quantifies the ability to distinguish fine discrepancies in timing between auditory and visual stimuli. EEG was then recorded during a second run of the simultaneity judgment task, and partial least squares was used to relate individual differences in the TBW width to source-localized EEG measures of local entropy and mutual information, indexing local and distributed processing of information, respectively. The narrowness of the TBW, reflecting more effective multisensory processing, was related to a broad pattern of higher mutual information and lower local entropy at multiple timescales. Furthermore, a small group of temporal and frontal cortical regions, including those previously implicated in multisensory integration and response selection, respectively, played a prominent role in this pattern. Overall, these findings suggest that individual differences in multisensory processing are related to widespread individual differences in the balance of distributed versus local information processing among a large subset of brain regions, with more distributed information being associated with more effective multisensory processing. The balance of distributed versus local information processing may therefore be a useful measure for exploring individual differences in multisensory processing, its relationship to higher cognitive traits, and its disruption in neurodevelopmental disorders and clinical conditions.

Beta and Theta Oscillations Correlate With Subjective Time During Musical Improvisation in Ecological and Controlled Settings: A Single Subject Study

In this paper, we describe the results of a single subject study attempting at a better understanding of the subjective mental state during musical improvisation. In a first experiment, we setup an ecological paradigm measuring EEG on a musician in free improvised concerts with an audience, followed by retrospective rating of the mental state of the improviser. We introduce Subjective Temporal Resolution (STR), a retrospective rating assessing the instantaneous quantization of subjective timing of the improviser. We identified high and low STR states using Hidden Markov Models in two performances, and were able to decode those states using supervised learning on instantaneous EEG power spectrum, showing increases in theta and alpha power with high STR values. In a second experiment, we found an increase of theta and beta power when experimentally manipulating STR in a musical improvisation imagery experiment. These results are interpreted with respect to previous research on flow state in creativity, as well as with the temporal processing literature. We suggest that a component of the subjective state of musical improvisation may be reflected in an underlying mechanism related to the subjective quantization of time. We also demonstrate the feasibility of single case studies of musical improvisation using brain activity measurements and retrospective reports, by obtaining consistent results across multiple sessions.

Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions

To integrate auditory and visual signals into a unified percept, the paired stimuli must co-occur within a limited time window known as the Temporal Binding Window (TBW). The width of the TBW, a proxy of audiovisual temporal integration ability, has been found to be correlated with higher-order cognitive and social functions. A comprehensive review of studies investigating audiovisual TBW reveals several findings: (1) a wide range of top-down processes and bottom-up features can modulate the width of the TBW, facilitating adaptation to the changing and multisensory external environment; (2) a large-scale brain network works in coordination to ensure successful detection of audiovisual (a)synchrony; (3) developmentally, audiovisual TBW follows a U-shaped pattern across the lifespan, with a protracted developmental course into late adolescence and rebounding in size again in late life; (4) an enlarged TBW is characteristic of a number of neurodevelopmental disorders; and (5) the TBW is highly flexible via perceptual and musical training. Interventions targeting the TBW may be able to improve multisensory function and ameliorate social communicative symptoms in clinical populations.

Neural Oscillations Orchestrate Multisensory Processing

At any given moment, we receive input through our different sensory systems, and this information needs to be processed and integrated. Multisensory processing requires the coordinated activity of distinct cortical areas. Key mechanisms implicated in these processes include local neural oscillations and functional connectivity between distant cortical areas. Evidence is now emerging that neural oscillations in distinct frequency bands reflect different mechanisms of multisensory processing. Moreover, studies suggest that aberrant neural oscillations contribute to multisensory processing deficits in clinical populations, such as schizophrenia. In this article, we review recent literature on the neural mechanisms underlying multisensory processing, focusing on neural oscillations. We derive a framework that summarizes findings on (1) stimulus-driven multisensory processing, (2) the influence of top-down information on multisensory processing, and (3) the role of predictions for the formation of multisensory perception. We propose that different frequency band oscillations subserve complementary mechanisms of multisensory processing. These processes can act in parallel and are essential for multisensory processing.

Cocaine-Induced Changes in Low-Dimensional Attractors of Local Field Potentials in Optogenetic Mice

Optogenetically evoked local field potential (LFP) recorded from the medial prefrontal cortex (mPFC) of mice during basal conditions and following a systemic cocaine administration were analyzed. Blue light stimuli were delivered to mPFC through a fiber optic every 2 s and each trial was repeated 100 times. As in the previous study, we used a surrogate data method to check that nonlinearity was present in the experimental LFPs and only used the last 1.5 s of steady activity to measure the LFPs phase resetting induced by the brief 10 ms light stimulus. We found that the steady dynamics of the mPFC in response to light stimuli could be reconstructed in a three-dimensional phase space with topologically similar "8"-shaped attractors across different animals. Therefore, cocaine did not change the complexity of the recorded nonlinear data compared to the control case. The phase space of the reconstructed attractor is determined by the LFP time series and its temporally shifted versions by a multiple of some lag time. We also compared the change in the attractor shape between cocaine-injected and control using (1) dendrogram clustering and (2) Frechet distance. We found about 20% overlap between control and cocaine trials when classified using dendrogram method, which suggest that it may be possible to describe mathematically both data sets with the same model and slightly different model parameters. We also found that the lag times are about three times shorter for cocaine trials compared to control. As a result, although the phase space trajectories for control and cocaine may look similar, their dynamics is significantly different.

Spontaneous analogising caused by text stimuli in design thinking: differences between higher- and lower-creativity groups

Understanding the cognitive processes used in creative practices is essential to design research. In this study, electroencephalography was applied to investigate the brain activations of visual designers when they responded to various types of word stimuli during design thinking. Thirty visual designers were recruited, with the top third and bottom third of the participants divided into high-creativity (HC) and low-creativity (LC) groups. The word stimuli used in this study were two short poems, adjectives with similar meanings, and adjectives with opposing meanings. The derived results are outlined as follows: (1) the brain activations of the designers increased in the frontal and right temporal regions and decreased in the right prefrontal region; (2) the negative association between the right temporal and middle frontal regions was notable; (3) the differences in activations caused by distinct word stimuli varied between HC and LC designers; (4) the spectral power in the middle frontal region of HC designers was lower than that of LC designers during the short love poem task; (5) the spectral power in the bilateral temporal regions of HC designers was higher than that of LC designers during the short autumn poem task; (6) the spectral power in the frontoparietal region of HC designers was lower than that of LC designers during the similar concept task; and (7) the spectral power in the frontoparietal and left frontotemporal regions of HC designers was higher than that of LC designers during the opposing concept task.