Evidence for enhanced multisensory facilitation with stimulus relevance: an electrophysiological investigation

Event-related modulation of alpha rhythm explains the auditory P300-evoked response in EEG

Evoked responses and oscillations represent two major electrophysiological phenomena in the human brain yet the link between them remains rather obscure. Here we show how most frequently studied EEG signals: the P300-evoked response and alpha oscillations (8-12 Hz) can be linked with the baseline-shift mechanism. This mechanism states that oscillations generate evoked responses if oscillations have a non-zero mean and their amplitude is modulated by the stimulus. Therefore, the following predictions should hold: (1) the temporal evolution of P300 and alpha amplitude is similar, (2) spatial localisations of the P300 and alpha amplitude modulation overlap, (3) oscillations are non-zero mean, (4) P300 and alpha amplitude correlate with cognitive scores in a similar fashion. To validate these predictions, we analysed the data set of elderly participants (N=2230, 60-82 years old), using (a) resting-state EEG recordings to quantify the mean of oscillations, (b) the event-related data, to extract parameters of P300 and alpha rhythm amplitude envelope. We showed that P300 is indeed linked to alpha rhythm, according to all four predictions. Our results provide an unifying view on the interdependency of evoked responses and neuronal oscillations and suggest that P300, at least partly, is generated by the modulation of alpha oscillations.

Interaction within and between cortical networks subserving multisensory learning and its reorganization due to musical expertise

Recent advancements in the field of network science allow us to quantify inter-network information exchange and model the interaction within and between task-defined states of large-scale networks. Here, we modeled the inter- and intra- network interactions related to multisensory statistical learning. To this aim, we implemented a multifeatured statistical learning paradigm and measured evoked magnetoencephalographic responses to estimate task-defined state of functional connectivity based on cortical phase interaction. Each network state represented the whole-brain network processing modality-specific (auditory, visual and audiovisual) statistical learning irregularities embedded within a multisensory stimulation stream. The way by which domain-specific expertise re-organizes the interaction between the networks was investigated by a comparison of musicians and non-musicians. Between the modality-specific network states, the estimated connectivity quantified the characteristics of a supramodal mechanism supporting the identification of statistical irregularities that are compartmentalized and applied in the identification of uni-modal irregularities embedded within multisensory stimuli. Expertise-related re-organization was expressed by an increase of intra- and a decrease of inter-network connectivity, showing increased compartmentalization.

Maladaptive reorganization following SCI: The role of body representation and multisensory integration

In this review we focus on maladaptive brain reorganization after spinal cord injury (SCI), including the development of neuropathic pain, and its relationship with impairments in body representation and multisensory integration. We will discuss the implications of altered sensorimotor interactions after SCI with and without neuropathic pain and possible deficits in multisensory integration and body representation. Within this framework we will examine published research findings focused on the use of bodily illusions to manipulate multisensory body representation to induce analgesic effects in heterogeneous chronic pain populations and in SCI-related neuropathic pain. We propose that the development and intensification of neuropathic pain after SCI is partly dependent on brain reorganization associated with dysfunctional multisensory integration processes and distorted body representation. We conclude this review by suggesting future research avenues that may lead to a better understanding of the complex mechanisms underlying the sense of the body after SCI, with a focus on cortical changes.

Top-down task-specific determinants of multisensory motor reaction time enhancements and sensory switch costs

This study was designed to investigate the complex interplay between multisensory processing, top–down processes related to the task relevance of sensory signals, and sensory switching. Thirty-five adults completed either a speeded detection or a discrimination task using the same auditory and visual stimuli and experimental setup. The stimuli consisted of unisensory and multisensory presentations of the letters ‘b’ and ‘d’. The multisensory stimuli were either congruent (e.g., the grapheme ‘b’ with the phoneme /b/) or incongruent (e.g., the grapheme ‘b’ with the phoneme /d/). In the detection task, the participants had to respond to all of the stimuli as rapidly as possible while, in the discrimination task, they only responded on those trials where one prespecified letter (either ‘b’ or ‘d’) was present. Incongruent multisensory stimuli resulted in faster responses as compared to unisensory stimuli in the detection task. In the discrimination task, only the dual-target congruent stimuli resulted in faster RTs, while the incongruent multisensory stimuli led to slower RTs than to unisensory stimuli; RTs were the slowest when the visual (rather than the auditory) signal was irrelevant, thus suggesting visual dominance. Switch costs were also observed when switching between unisensory target stimuli, while dual-target multisensory stimuli were less likely to be affected by sensory switching. Taken together, these findings suggest that multisensory motor enhancements and sensory switch costs are influenced by top–down modulations determined by task instructions, which can override the influence of prior learnt associations.

An Experimenter's Influence on Motor Enhancements: The Effects of Letter Congruency and Sensory Switch-Costs on Multisensory Integration

Multisensory integration can alter information processing, and previous research has shown that such processes are modulated by sensory switch costs and prior experience (e.g., semantic or letter congruence). Here we report an incidental finding demonstrating, for the first time, the interplay between these processes and experimental factors, specifically the presence (vs. absence) of the experimenter in the testing room. Experiment 1 demonstrates that multisensory motor facilitation in response to audiovisual stimuli (circle and tone with no prior learnt associations) is higher in those trials in which the sensory modality switches than when it repeats. Those participants who completed the study while alone exhibited increased RT variability. Experiment 2 replicated these findings using the letters “b” and “d” presented as unisensory stimuli or congruent and incongruent multisensory stimuli (i.e., grapheme-phoneme pairs). Multisensory enhancements were inflated following a sensory switch; that is, congruent and incongruent multisensory stimuli resulted in significant gains following a sensory switch in the monitored condition. However, when the participants were left alone, multisensory enhancements were only observed for repeating incongruent multisensory stimuli. These incidental findings therefore suggest that the effects of letter congruence and sensory switching on multisensory integration are partly modulated by the presence of an experimenter.

Multisensory processing in event-based prospective memory

Failures in prospective memory (PM) - that is, the failure to remember intended future actions - can have adverse consequences. It is therefore important to study those processes that may help to minimize such cognitive failures. Although multisensory integration has been shown to enhance a wide variety of behaviors, including perception, learning, and memory, its effect on prospective memory, in particular, is largely unknown. In the present study, we investigated the effects of multisensory processing on two simultaneously-performed memory tasks: An ongoing 2- or 3-back working memory (WM) task (20% target ratio), and a PM task in which the participants had to respond to a rare predefined letter (8% target ratio). For PM trials, multisensory enhancement was observed for congruent multisensory signals; however, this effect did not generalize to the ongoing WM task. Participants were less likely to make errors for PM than for WM trials, thus suggesting that they may have biased their attention toward the PM task. Multisensory advantages on memory tasks, such as PM and WM, may be dependent on how attention resources are allocated across dual tasks.

Neural Oscillations Orchestrate Multisensory Processing

At any given moment, we receive input through our different sensory systems, and this information needs to be processed and integrated. Multisensory processing requires the coordinated activity of distinct cortical areas. Key mechanisms implicated in these processes include local neural oscillations and functional connectivity between distant cortical areas. Evidence is now emerging that neural oscillations in distinct frequency bands reflect different mechanisms of multisensory processing. Moreover, studies suggest that aberrant neural oscillations contribute to multisensory processing deficits in clinical populations, such as schizophrenia. In this article, we review recent literature on the neural mechanisms underlying multisensory processing, focusing on neural oscillations. We derive a framework that summarizes findings on (1) stimulus-driven multisensory processing, (2) the influence of top-down information on multisensory processing, and (3) the role of predictions for the formation of multisensory perception. We propose that different frequency band oscillations subserve complementary mechanisms of multisensory processing. These processes can act in parallel and are essential for multisensory processing.

Multisensory enhancement elicited by unconscious visual stimuli

The merging of information from different senses (i.e., multisensory integration) can facilitate information processing. Processing enhancements have been observed with signals that are irrelevant to the task at hand, and with cues that are non-predictive. Such findings are consistent with the notion that multiple sensory signals are sometimes integrated automatically. Multisensory enhancement has even been reported with stimuli that have been presented subliminally, though only with meaningful multisensory relations that have already been learned. The question of whether there exist cases where multisensory effects occur without either learning or awareness has, though, not been clearly established in the literature to date. Here, we present a case study of a patient with Posterior Cortical Atrophy, who was unable to consciously perceive visual stimuli with our task parameters, yet who nevertheless still exhibited signs of multisensory enhancement even with unlearned relations between audiovisual stimuli. In a simple speeded detection task, both response speed, and the variability of reaction times, decreased in a similar manner to controls for multisensory stimuli. These results are consistent with the view that the conscious perception of stimuli and prior learning are not always a prerequisite for multisensory integration to enhance human performance.

The interactions of multisensory integration with endogenous and exogenous attention

Stimuli from multiple sensory organs can be integrated into a coherent representation through multiple phases of multisensory processing; this phenomenon is called multisensory integration. Multisensory integration can interact with attention. Here, we propose a framework in which attention modulates multisensory processing in both endogenous (goal-driven) and exogenous (stimulus-driven) ways. Moreover, multisensory integration exerts not only bottom-up but also top-down control over attention. Specifically, we propose the following: (1) endogenous attentional selectivity acts on multiple levels of multisensory processing to determine the extent to which simultaneous stimuli from different modalities can be integrated; (2) integrated multisensory events exert top-down control on attentional capture via multisensory search templates that are stored in the brain; (3) integrated multisensory events can capture attention efficiently, even in quite complex circumstances, due to their increased salience compared to unimodal events and can thus improve search accuracy; and (4) within a multisensory object, endogenous attention can spread from one modality to another in an exogenous manner.

Oscillatory signatures of crossmodal congruence effects: An EEG investigation employing a visuotactile pattern matching paradigm

Coherent percepts emerge from the accurate combination of inputs from the different sensory systems. There is an ongoing debate about the neurophysiological mechanisms of crossmodal interactions in the brain, and it has been proposed that transient synchronization of neurons might be of central importance. Oscillatory activity in lower frequency ranges (<30Hz) has been implicated in mediating long-range communication as typically studied in multisensory research. In the current study, we recorded high-density electroencephalograms while human participants were engaged in a visuotactile pattern matching paradigm and analyzed oscillatory power in the theta- (4-7Hz), alpha- (8-13Hz) and beta-bands (13-30Hz). Employing the same physical stimuli, separate tasks of the experiment either required the detection of predefined targets in visual and tactile modalities or the explicit evaluation of crossmodal stimulus congruence. Analysis of the behavioral data showed benefits for congruent visuotactile stimulus combinations. Differences in oscillatory dynamics related to crossmodal congruence within the two tasks were observed in the beta-band for crossmodal target detection, as well as in the theta-band for congruence evaluation. Contrasting ongoing activity preceding visuotactile stimulation between the two tasks revealed differences in the alpha- and beta-bands. Source reconstruction of between-task differences showed prominent involvement of premotor cortex, supplementary motor area, somatosensory association cortex and the supramarginal gyrus. These areas not only exhibited more involvement in the pre-stimulus interval for target detection compared to congruence evaluation, but were also crucially involved in post-stimulus differences related to crossmodal stimulus congruence within the detection task. These results add to the increasing evidence that low frequency oscillations are functionally relevant for integration in distributed brain networks, as demonstrated for crossmodal interactions in visuotactile pattern matching in the current study.

Event-related oscillations (ERO) during an active discrimination task: Effects of lesions of the nucleus basalis magnocellularis

The cholinergic system in the brain is involved in attentional processes that are engaged for the identification and selection of relevant information in the environment and the formation of new stimulus associations. In the present study we determined the effects of cholinergic lesions of nucleus basalis magnocellularis (NBM) on amplitude and phase characteristics of event related oscillations (EROs) generated in an auditory active discrimination task in rats. Rats were trained to press a lever to begin a series of 1kHz tones and to release the lever upon hearing a 2kHz tone. A time-frequency based representation was used to determine ERO energy and phase synchronization (phase lock index, PLI) across trials, recorded within frontal cortical structures. Lesions in NBM produced by an infusion of a-amino-3-hydroxy-5-methyl-4-isoxazole propionic acid (AMPA) resulted in (1) a reduction of the number of correct behavioral responses in the active discrimination task, (2) an increase in ERO energy in the delta frequency bands, (3) an increase in theta, alpha and beta ERO energy in the N1, P3a and P3b regions of interest (ROI), and (4) an increase in PLI in the theta frequency band in the N1 ROIs. These studies suggest that the NBM cholinergic system is involved in maintaining the synchronization/phase resetting of oscillations in different frequencies in response to the presentation of the target stimuli in an active discrimination task.

Developmental trends in the facilitation of multisensory objects with distractors

Sensory integration and the ability to discriminate target objects from distractors are critical to survival, yet the developmental trajectories of these abilities are unknown. This study investigated developmental changes in 9- (n = 18) and 11-year-old (n = 20) children, adolescents (n = 19) and adults (n = 22) using an audiovisual object discrimination task with uni- and multisensory distractors. Reaction times (RTs) were slower with visual/audiovisual distractors, and although all groups demonstrated facilitation of multisensory RTs in these conditions, children's and adolescents' responses corresponded to fewer race model violations than adults', suggesting protracted maturation of multisensory processes. Multisensory facilitation could not be explained by changes in RT variability, suggesting that tests of race model violations may still have theoretical value at least for familiar multisensory stimuli.

Cholinergic modulation of event-related oscillations (ERO)

The cholinergic system in the brain modulates patterns of activity involved in general arousal, attention processing, memory and consciousness. In the present study we determined the effects of selective cholinergic lesions of the medial septum area (MS) or nucleus basalis magnocellularis (NBM) on amplitude and phase characteristics of event related oscillations (EROs). A time-frequency based representation was used to determine ERO energy, phase synchronization across trials, recorded within a structure (phase lock index, PLI), and phase synchronization across trials, recorded between brain structures (phase difference lock index, PDLI), in the frontal cortex (Fctx), dorsal hippocampus (DHPC) and central amygdala (Amyg). Lesions in MS produced: (1) decreases in ERO energy in delta, theta, alpha, beta and gamma frequencies in Amyg, (2) reductions in gamma ERO energy and PLI in Fctx, (3) decreases in PDLI between the Fctx-Amyg in the theta, alpha, beta and gamma frequencies, and (4) decreases in PDLI between the DHPC-Amyg and Fctx-DHPC in the theta frequency bands. Lesions in NBM resulted in: (1) increased ERO energy in delta and theta frequency bands in Fctx, (2) reduced gamma ERO energy in Fctx and Amyg, (3) reductions in PLI in the theta, beta and gamma frequency ranges in Fctx, (4) reductions in gamma PLI in DHPC and (5) reduced beta PLI in Amyg. These studies suggest that the MS cholinergic system can alter phase synchronization between brain areas whereas the NBM cholinergic system modifies phase synchronization/phase resetting within a brain area.