Source Localisation of Visual Evoked Potentials in Congenitally Deaf Individuals

Reduced Visual-Cortex Reorganization Before and After Cochlear Implantation Relates to Better Speech Recognition Ability

Although a cochlear implant (CI) can partially restore auditory function, CI recipients show alterations not only in auditory but also in visual cortical processing. Yet, it is not well understood how these visual changes relate to the CI outcome and to what extent these changes are induced by auditory deprivation and the limited CI input, respectively. Here, we present a prospective longitudinal electroencephalography study which examined the deprivation- and CI-induced alterations on cortical face processing by comparing visual evoked potentials (VEP) in CI users before and 6 months after implantation. A group of normal-hearing (NH) listeners served as a control. The participants performed a word-identification task and a face-categorization task to study the cortical processing of static and articulating faces in attended and unattended conditions. The CI candidates and CI users showed a reduced visual-cortex activation, a stronger functional connectivity between the visual and auditory cortex, and a reduced attention effect in the (extended) alpha frequency range (8-18 Hz) when compared to NH listeners. There was a positive correlation between the P1 VEP amplitude recorded before implantation and the speech recognition ability after implantation. Our results suggest that the CI users' alterations in cortical face processing are mainly induced by auditory deprivation and not by CI experience. Importantly, these deprivation-induced changes seem to be related to the CI outcome. Our results suggest that the visual P1 amplitude as recorded before implantation provides an objective index of cortical visual reorganization that may help predict the CI outcome.

Cross-Modal Plasticity in Postlingual Hearing Loss Predicts Speech Perception Outcomes After Cochlear Implantation

Background: Sensory loss may lead to intra- and cross-modal cortical reorganization. Previous research showed a significant correlation between the cross-modal contribution of the right auditory cortex to visual evoked potentials (VEP) and speech perception in cochlear implant (CI) users with prelingual hearing loss (HL), but not in those with postlingual HL. The present study aimed to explore the cortical reorganization induced by postlingual HL, particularly in the right temporal region, and how it correlates with speech perception outcome with a CI. Material and Methods: A total of 53 adult participants were divided into two groups according to hearing ability: 35 had normal hearing (NH) (mean age = 62.10 years (±7.48)) and 18 had profound postlingual HL (mean age = 63.78 years (±8.44)). VEPs, using a 29-channel electroencephalogram (EEG) system, were recorded preoperatively in the 18 patients scheduled for cochlear implantation and in 35 NH adults who served as the control group. Amplitudes and latencies of the P100, N100, and P200 components were analyzed across frontal, temporal, and occipital areas and compared between NH and HL subjects using repeated measures ANOVA. For the HL group, speech perception in quiet was assessed at 6 and 12 months of CI use. Results: No difference was found in amplitudes or latencies of the P100, N100, and P200 VEP components between the NH and HL groups. Further analysis using Spearman correlations between preoperative amplitudes and latencies of the P100, N100, and P200 VEP components at the right temporal electrode position T8 and postoperative speech perception showed that the HL group had either significantly higher or significantly lower amplitudes of the P200 component at the right temporal electrode position T8 compared to the NH controls. The HL subgroup with higher amplitudes had better speech perception than the subgroup with lower amplitudes at 6 months and 12 months of CI use. Conclusions: Preoperative evaluation of cortical plasticity can reveal plasticity profiles, which might help to better predict postoperative speech outcomes and adapt the rehabilitation regimen after CI activation. Further research is needed to understand the susceptibility of each component to cross-modal reorganization and their specific contribution to outcome prediction.

Changes in visually and auditory attended audiovisual speech processing in cochlear implant users: A longitudinal ERP study

Limited auditory input, whether caused by hearing loss or by electrical stimulation through a cochlear implant (CI), can be compensated by the remaining senses. Specifically for CI users, previous studies reported not only improved visual skills, but also altered cortical processing of unisensory visual and auditory stimuli. However, in multisensory scenarios, it is still unclear how auditory deprivation (before implantation) and electrical hearing experience (after implantation) affect cortical audiovisual speech processing. Here, we present a prospective longitudinal electroencephalography (EEG) study which systematically examined the deprivation- and CI-induced alterations of cortical processing of audiovisual words by comparing event-related potentials (ERPs) in postlingually deafened CI users before and after implantation (five weeks and six months of CI use). A group of matched normal-hearing (NH) listeners served as controls. The participants performed a word-identification task with congruent and incongruent audiovisual words, focusing their attention on either the visual (lip movement) or the auditory speech signal. This allowed us to study the (top-down) attention effect on the (bottom-up) sensory cortical processing of audiovisual speech. When compared to the NH listeners, the CI candidates (before implantation) and the CI users (after implantation) exhibited enhanced lipreading abilities and an altered cortical response at the N1 latency range (90-150 ms) that was characterized by a decreased theta oscillation power (4-8 Hz) and a smaller amplitude in the auditory cortex. After implantation, however, the auditory-cortex response gradually increased and developed a stronger intra-modal connectivity. Nevertheless, task efficiency and activation in the visual cortex was significantly modulated in both groups by focusing attention on the visual as compared to the auditory speech signal, with the NH listeners additionally showing an attention-dependent decrease in beta oscillation power (13-30 Hz). In sum, these results suggest remarkable deprivation effects on audiovisual speech processing in the auditory cortex, which partially reverse after implantation. Although even experienced CI users still show distinct audiovisual speech processing compared to NH listeners, pronounced effects of (top-down) direction of attention on (bottom-up) audiovisual processing can be observed in both groups. However, NH listeners but not CI users appear to show enhanced allocation of cognitive resources in visually as compared to auditory attended audiovisual speech conditions, which supports our behavioural observations of poorer lipreading abilities and reduced visual influence on audition in NH listeners as compared to CI users.

Cross-modal plasticity in children with cochlear implant: converging evidence from EEG and functional near-infrared spectroscopy

Over the first years of life, the brain undergoes substantial organization in response to environmental stimulation. In a silent world, it may promote vision by (i) recruiting resources from the auditory cortex and (ii) making the visual cortex more efficient. It is unclear when such changes occur and how adaptive they are, questions that children with cochlear implants can help address. Here, we examined 7-18 years old children: 50 had cochlear implants, with delayed or age-appropriate language abilities, and 25 had typical hearing and language. High-density electroencephalography and functional near-infrared spectroscopy were used to evaluate cortical responses to a low-level visual task. Evidence for a 'weaker visual cortex response' and 'less synchronized or less inhibitory activity of auditory association areas' in the implanted children with language delays suggests that cross-modal reorganization can be maladaptive and does not necessarily strengthen the dominant visual sense.

Theta and alpha oscillatory signatures of auditory sensory and cognitive loads during complex listening

The neuronal signatures of sensory and cognitive load provide access to brain activities related to complex listening situations. Sensory and cognitive loads are typically reflected in measures like response time (RT) and event-related potentials (ERPs) components. It's, however, strenuous to distinguish the underlying brain processes solely from these measures. In this study, along with RT- and ERP-analysis, we performed time-frequency analysis and source localization of oscillatory activity in participants performing two different auditory tasks with varying degrees of complexity and related them to sensory and cognitive load. We studied neuronal oscillatory activity in both periods before the behavioral response (pre-response) and after it (post-response). Robust oscillatory activities were found in both periods and were differentially affected by sensory and cognitive load. Oscillatory activity under sensory load was characterized by decrease in pre-response (early) theta activity and increased alpha activity. Oscillatory activity under cognitive load was characterized by increased theta activity, mainly in post-response (late) time. Furthermore, source localization revealed specific brain regions responsible for processing these loads, such as temporal and frontal lobe, cingulate cortex and precuneus. The results provide evidence that in complex listening situations, the brain processes sensory and cognitive loads differently. These neural processes have specific oscillatory signatures and are long lasting, extending beyond the behavioral response.

Electrophysiological study of visual processing in children with cochlear implants

Electrophysiological studies of congenitally deaf children and adults have reported atypical visual evoked potentials (VEPs) which have been associated with both behavioral enhancements of visual attention as well as poorer performance and outcomes in tests of spoken language speech processing. This pattern has often been interpreted as a maladaptive consequence of early auditory deprivation, whereby a remapping of auditory cortex by the visual system ultimately reduces resources necessary for optimal rehabilitative outcomes of spoken language acquisition and use. Making use of a novel electrophysiological paradigm, we compare VEPs in children with severe to profound congenital deafness who received a cochlear implant(s) prior to 31 months (n = 28) and typically developing age matched controls (n = 28). We observe amplitude enhancements and in some cases latency differences in occipitally expressed P1 and N1 VEP components in CI-using children as well as an early frontal negativity, N1a. We relate these findings to developmental factors such as chronological age and spoken language understanding. We further evaluate whether VEPs are additionally modulated by auditory stimulation. Collectively, these data provide a means to examine the extent to which atypical VEPs are consistent with prior accounts of maladaptive cross-modal plasticity. Our results support a view that VEP changes reflect alterations to visual-sensory attention and saliency mechanisms rather than a re-mapping of auditory cortex. The present data suggests that early auditory deprivation may have temporally prolonged effects on visual system processing even after activation and use of cochlear implant.

Crossmodal plasticity in hearing loss

Crossmodal plasticity is a textbook example of the ability of the brain to reorganize based on use. We review evidence from the auditory system showing that such reorganization has significant limits, is dependent on pre-existing circuitry and top-down interactions, and that extensive reorganization is often absent. We argue that the evidence does not support the hypothesis that crossmodal reorganization is responsible for closing critical periods in deafness, and crossmodal plasticity instead represents a neuronal process that is dynamically adaptable. We evaluate the evidence for crossmodal changes in both developmental and adult-onset deafness, which start as early as mild-moderate hearing loss and show reversibility when hearing is restored. Finally, crossmodal plasticity does not appear to affect the neuronal preconditions for successful hearing restoration. Given its dynamic and versatile nature, we describe how this plasticity can be exploited for improving clinical outcomes after neurosensory restoration.

Visually evoked potentials (VEPs) across the visual field in hearing and deaf cats

Introduction

Congenitally deaf cats perform better on visual localization tasks than hearing cats, and this advantage has been attributed to the posterior auditory field. Successful visual localization requires both visual processing of the target and timely generation of an action to approach the target. Activation of auditory cortex in deaf subjects during visual localization in the peripheral visual field can occur either via bottom-up stimulus-driven and/or top-down goal-directed pathways.

Methods

In this study, we recorded visually evoked potentials (VEPs) in response to a reversing checkerboard stimulus presented in the hemifield contralateral to the recorded hemisphere in both hearing and deaf cats under light anesthesia.

Results

Although VEP amplitudes and latencies were systematically modulated by stimulus eccentricity, we found little evidence of changes in VEP in deaf cats that can explain their behavioral advantage. A statistical trend was observed, showing larger peak amplitudes and shorter peak latencies in deaf subjects for stimuli in the near- and mid-peripheral field. Additionally, latency of the P1 wave component had a larger inter-sweep variation in deaf subjects.

Discussion

Our results suggested that cross-modal plasticity following deafness does not play a major part in cortical processing of the peripheral visual field when the "vision for action" system is not recruited.

Temporal visual representation elicits early auditory-like responses in hearing but not in deaf individuals

It is evident that the brain is capable of large-scale reorganization following sensory deprivation, but the extent of such reorganization is to date, not clear. The auditory modality is the most accurate to represent temporal information, and deafness is an ideal clinical condition to study the reorganization of temporal representation when the audio signal is not available. Here we show that hearing, but not deaf individuals, show a strong ERP response to visual stimuli in temporal areas during a time-bisection task. This ERP response appears 50-90 ms after the flash and recalls some aspects of the N1 ERP component usually elicited by auditory stimuli. The same ERP is not evident for a visual space-bisection task, suggesting that the early recruitment of temporal cortex is specific for building a highly resolved temporal representation within the visual modality. These findings provide evidence that the lack of auditory input can interfere with typical development of complex visual temporal representations.

The timecourse of multisensory speech processing in unilaterally stimulated cochlear implant users revealed by ERPs

•

Both normal-hearing (NH) and cochlear implant (CI) users show a clear benefit in multisensory speech processing.

•

Group differences in ERP topographies and cortical source activation suggest distinct audiovisual speech processing in CI users when compared to NH listeners.

•

Electrical neuroimaging, including topographic and ERP source analysis, provides a suitable tool to study the timecourse of multisensory speech processing in CI users.

A cochlear implant (CI) is an auditory prosthesis which can partially restore the auditory function in patients with severe to profound hearing loss. However, this bionic device provides only limited auditory information, and CI patients may compensate for this limitation by means of a stronger interaction between the auditory and visual system. To better understand the electrophysiological correlates of audiovisual speech perception, the present study used electroencephalography (EEG) and a redundant target paradigm. Postlingually deafened CI users and normal-hearing (NH) listeners were compared in auditory, visual and audiovisual speech conditions. The behavioural results revealed multisensory integration for both groups, as indicated by shortened response times for the audiovisual as compared to the two unisensory conditions. The analysis of the N1 and P2 event-related potentials (ERPs), including topographic and source analyses, confirmed a multisensory effect for both groups and showed a cortical auditory response which was modulated by the simultaneous processing of the visual stimulus. Nevertheless, the CI users in particular revealed a distinct pattern of N1 topography, pointing to a strong visual impact on auditory speech processing. Apart from these condition effects, the results revealed ERP differences between CI users and NH listeners, not only in N1/P2 ERP topographies, but also in the cortical source configuration. When compared to the NH listeners, the CI users showed an additional activation in the visual cortex at N1 latency, which was positively correlated with CI experience, and a delayed auditory-cortex activation with a reversed, rightward functional lateralisation. In sum, our behavioural and ERP findings demonstrate a clear audiovisual benefit for both groups, and a CI-specific alteration in cortical activation at N1 latency when auditory and visual input is combined. These cortical alterations may reflect a compensatory strategy to overcome the limited CI input, which allows the CI users to improve the lip-reading skills and to approximate the behavioural performance of NH listeners in audiovisual speech conditions. Our results are clinically relevant, as they highlight the importance of assessing the CI outcome not only in auditory-only, but also in audiovisual speech conditions.

Use of Functional Near-Infrared Spectroscopy to Predict and Measure Cochlear Implant Outcomes: A Scoping Review

Outcomes following cochlear implantation vary widely for both adults and children, and behavioral tests are currently relied upon to assess this. However, these behavioral tests rely on subjective judgements that can be unreliable, particularly for infants and young children. The addition of an objective test of outcome following cochlear implantation is therefore desirable. The aim of this scoping review was to comprehensively catalogue the evidence for the potential of functional near infrared spectroscopy (fNIRS) to be used as a tool to objectively predict and measure cochlear implant outcomes. A scoping review of the literature was conducted following the PRISMA extension for scoping review framework. Searches were conducted in the MEDLINE, EMBASE, PubMed, CINAHL, SCOPUS, and Web of Science electronic databases, with a hand search conducted in Google Scholar. Key terms relating to near infrared spectroscopy and cochlear implants were used to identify relevant publications. Eight records met the criteria for inclusion. Seven records reported on adult populations, with five records only including post-lingually deaf individuals and two including both pre- and post-lingually deaf individuals. Studies were either longitudinal or cross-sectional, and all studies compared fNIRS measurements with receptive speech outcomes. This review identified and collated key work in this field. The homogeneity of the populations studied so far identifies key gaps for future research, including the use of fNIRS in infants. By mapping the literature on this important topic, this review contributes knowledge towards the improvement of outcomes following cochlear implantation.

Oscillatory signatures of Repetition Suppression and Novelty Detection reveal altered induced visual responses in early deafness

The ability to differentiate between repeated and novel events represents a fundamental property of the visual system. Neural responses are typically reduced upon stimulus repetition, a phenomenon called Repetition Suppression (RS). On the contrary, following a novel visual stimulus, the neural response is generally enhanced, a phenomenon referred to as Novelty Detection (ND). Here, we aimed to investigate the impact of early deafness on the oscillatory signatures of RS and ND brain responses. To this aim, electrophysiological data were acquired in early deaf and hearing control individuals during processing of repeated and novel visual events unattended by participants. By studying evoked and induced oscillatory brain activities, as well as inter-trial phase coherence, we linked response modulations to feedback and/or feedforward processes. Results revealed selective experience-dependent changes on both RS and ND mechanisms. Compared to hearing controls, early deaf individuals displayed: (i) greater attenuation of the response following stimulus repetition, selectively in the induced theta-band (4-7 Hz); (ii) reduced desynchronization following the onset of novel visual stimuli, in the induced alpha and beta bands (8-12 and 13-25 Hz); (iii) comparable modulation of evoked responses and inter-trial phase coherence. The selectivity of the effects in the induced responses parallels findings observed in the auditory cortex of deaf animal models following intracochlear electric stimulation. The present results support the idea that early deafness alters induced oscillatory activity and the functional tuning of basic visual processing.

A noise-robust sparse approach to the time-frequency representation of visual evoked potentials

BACKGROUND

Visual evoked potential (VEP) offers a promising research strategy in the effort to characterise brain disorders. Pertinent signal processing techniques enable the development of potential applications of VEP. A joint time-frequency (TF) representation provides more comprehensive information about the underlying complex structures of these signals than individual time or frequency analysis. However, this representation comes at the expense of low TF resolution, increased data volume, poor energy concentration and increased computational time. Owing to the high non-stationarity and low signal-to-noise ratio of VEP, a TF representation that retains only the pertinent components is indispensable.

METHOD

The objective of this study is to investigate and demonstrate the ability of various TF approaches to provide an energy-concentrated and sparse TF representation of VEP. The performance of each method has been assessed for its energy concentration and reconstruction ability on both simulated and real VEPs. Renyi entropy, computation time and correlation coefficient are chosen as the performance measures for the assessment.

RESULTS

In comparison with the other state-of-the-art approaches, Synchroextracting transform (SET) exhibits the lowest Renyi entropy and the highest correlation coefficient, thereby ensuring a compact TF representation for the better characterisation of VEP signals. These results are also statistically verified through the Friedman test (p<0.001).

CONCLUSION

SET assures a powerful TF framework with improved energy concentration at a faster pace while remaining invertible and preserving vital information.

Electrophysiological Dynamics of Visual-Tactile Temporal Order Perception in Early Deaf Adults

Studies of compensatory plasticity in early deaf (ED) individuals have mainly focused on unisensory processing, and on spatial rather than temporal coding. However, precise discrimination of the temporal relationship between stimuli is imperative for successful perception of and interaction with the complex, multimodal environment. Although the properties of cross-modal temporal processing have been extensively studied in neurotypical populations, remarkably little is known about how the loss of one sense impacts the integrity of temporal interactions among the remaining senses. To understand how auditory deprivation affects multisensory temporal interactions, ED and age-matched normal hearing (NH) controls performed a visual-tactile temporal order judgment task in which visual and tactile stimuli were separated by varying stimulus onset asynchronies (SOAs) and subjects had to discern the leading stimulus. Participants performed the task while EEG data were recorded. Group averaged event-related potential waveforms were compared between groups in occipital and fronto-central electrodes. Despite similar temporal order sensitivities and performance accuracy, ED had larger visual P100 amplitudes for all SOA levels and larger tactile N140 amplitudes for the shortest asynchronous (± 30 ms) and synchronous SOA levels. The enhanced signal strength reflected in these components from ED adults are discussed in terms of compensatory recruitment of cortical areas for visual-tactile processing. In addition, ED adults had similar tactile P200 amplitudes as NH but longer P200 latencies suggesting reduced efficiency in later processing of tactile information. Overall, these results suggest that greater responses by ED for early processing of visual and tactile signals are likely critical for maintained performance in visual-tactile temporal order discrimination.

What and How the Deaf Brain Sees

Over the past decade, there has been an unprecedented level of interest and progress into understanding visual processing in the brain of the deaf. Specifically, when the brain is deprived of input from one sensory modality (such as hearing), it often compensates with supranormal performance in one or more of the intact sensory systems (such as vision). Recent psychophysical, functional imaging, and reversible deactivation studies have converged to define the specific visual abilities that are enhanced in the deaf, as well as the cortical loci that undergo crossmodal plasticity in the deaf and are responsible for mediating these superior visual functions. Examination of these investigations reveals that central visual functions, such as object and facial discrimination, and peripheral visual functions, such as motion detection, visual localization, visuomotor synchronization, and Vernier acuity (measured in the periphery), are specifically enhanced in the deaf, compared with hearing participants. Furthermore, the cortical loci identified to mediate these functions reside in deaf auditory cortex: BA 41, BA 42, and BA 22, in addition to the rostral area, planum temporale, Te3, and temporal voice area in humans; primary auditory cortex, anterior auditory field, dorsal zone of auditory cortex, auditory field of the anterior ectosylvian sulcus, and posterior auditory field in cats; and primary auditory cortex and anterior auditory field in both ferrets and mice. Overall, the findings from these studies show that crossmodal reorganization in auditory cortex of the deaf is responsible for the superior visual abilities of the deaf.

Increased cross-modal functional connectivity in cochlear implant users

Previous studies have reported increased cross-modal auditory and visual cortical activation in cochlear implant (CI) users, suggesting cross-modal reorganization of both visual and auditory cortices in CI users as a consequence of sensory deprivation and restoration. How these processes affect the functional connectivity of the auditory and visual system in CI users is however unknown. We here investigated task-induced intra-modal functional connectivity between hemispheres for both visual and auditory cortices and cross-modal functional connectivity between visual and auditory cortices using functional near infrared spectroscopy in post-lingually deaf CI users and age-matched normal hearing controls. Compared to controls, CI users exhibited decreased intra-modal functional connectivity between hemispheres and increased cross-modal functional connectivity between visual and left auditory cortices for both visual and auditory stimulus processing. Importantly, the difference between cross-modal functional connectivity for visual and for auditory stimuli correlated with speech recognition outcome in CI users. Higher cross-modal connectivity for auditory than for visual stimuli was associated with better speech recognition abilities, pointing to a new pattern of functional reorganization that is related to successful hearing restoration with a CI.

Auditory and Visual Electrophysiology of Deaf Children with Cochlear Implants: Implications for Cross-modal Plasticity

Deaf children who receive a cochlear implant early in life and engage in intensive oral/aural therapy often make great strides in spoken language acquisition. However, despite clinicians' best efforts, there is a great deal of variability in language outcomes. One concern is that cortical regions which normally support auditory processing may become reorganized for visual function, leaving fewer available resources for auditory language acquisition. The conditions under which these changes occur are not well understood, but we may begin investigating this phenomenon by looking for interactions between auditory and visual evoked cortical potentials in deaf children. If children with abnormal auditory responses show increased sensitivity to visual stimuli, this may indicate the presence of maladaptive cortical plasticity. We recorded evoked potentials, using both auditory and visual paradigms, from 25 typical hearing children and 26 deaf children (ages 2-8 years) with cochlear implants. An auditory oddball paradigm was used (85% /ba/ syllables vs. 15% frequency modulated tone sweeps) to elicit an auditory P1 component. Visual evoked potentials (VEPs) were recorded during presentation of an intermittent peripheral radial checkerboard while children watched a silent cartoon, eliciting a P1-N1 response. We observed reduced auditory P1 amplitudes and a lack of latency shift associated with normative aging in our deaf sample. We also observed shorter latencies in N1 VEPs to visual stimulus offset in deaf participants. While these data demonstrate cortical changes associated with auditory deprivation, we did not find evidence for a relationship between cortical auditory evoked potentials and the VEPs. This is consistent with descriptions of intra-modal plasticity within visual systems of deaf children, but do not provide evidence for cross-modal plasticity. In addition, we note that sign language experience had no effect on deaf children's early auditory and visual ERP responses.

The Right Hemisphere Planum Temporale Supports Enhanced Visual Motion Detection Ability in Deaf People: Evidence from Cortical Thickness

After sensory loss, the deprived cortex can reorganize to process information from the remaining modalities, a phenomenon known as cross-modal reorganization. In blind people this cross-modal processing supports compensatory behavioural enhancements in the nondeprived modalities. Deaf people also show some compensatory visual enhancements, but a direct relationship between these abilities and cross-modally reorganized auditory cortex has only been established in an animal model, the congenitally deaf cat, and not in humans. Using T1-weighted magnetic resonance imaging, we measured cortical thickness in the planum temporale, Heschl's gyrus and sulcus, the middle temporal area MT+, and the calcarine sulcus, in early-deaf persons. We tested for a correlation between this measure and visual motion detection thresholds, a visual function where deaf people show enhancements as compared to hearing. We found that the cortical thickness of a region in the right hemisphere planum temporale, typically an auditory region, was greater in deaf individuals with better visual motion detection thresholds. This same region has previously been implicated in functional imaging studies as important for functional reorganization. The structure-behaviour correlation observed here demonstrates this area's involvement in compensatory vision and indicates an anatomical correlate, increased cortical thickness, of cross-modal plasticity.

The role of auditory cortices in the retrieval of single-trial auditory-visual object memories

Single-trial encounters with multisensory stimuli affect both memory performance and early-latency brain responses to visual stimuli. Whether and how auditory cortices support memory processes based on single-trial multisensory learning is unknown and may differ qualitatively and quantitatively from comparable processes within visual cortices due to purported differences in memory capacities across the senses. We recorded event-related potentials (ERPs) as healthy adults (n = 18) performed a continuous recognition task in the auditory modality, discriminating initial (new) from repeated (old) sounds of environmental objects. Initial presentations were either unisensory or multisensory; the latter entailed synchronous presentation of a semantically congruent or a meaningless image. Repeated presentations were exclusively auditory, thus differing only according to the context in which the sound was initially encountered. Discrimination abilities (indexed by d') were increased for repeated sounds that were initially encountered with a semantically congruent image versus sounds initially encountered with either a meaningless or no image. Analyses of ERPs within an electrical neuroimaging framework revealed that early stages of auditory processing of repeated sounds were affected by prior single-trial multisensory contexts. These effects followed from significantly reduced activity within a distributed network, including the right superior temporal cortex, suggesting an inverse relationship between brain activity and behavioural outcome on this task. The present findings demonstrate how auditory cortices contribute to long-term effects of multisensory experiences on auditory object discrimination. We propose a new framework for the efficacy of multisensory processes to impact both current multisensory stimulus processing and unisensory discrimination abilities later in time.

Temporal dynamics of contingency extraction from tonal and verbal auditory sequences

Consecutive sound events are often to some degree predictive of each other. Here we investigated the brain's capacity to detect contingencies between consecutive sounds by means of electroencephalography (EEG) during passive listening. Contingencies were embedded either within tonal or verbal stimuli. Contingency extraction was measured indirectly via the elicitation of the mismatch negativity (MMN) component of the event-related potential (ERP) by contingency violations. MMN results indicate that structurally identical forms of predictability can be extracted from both tonal and verbal stimuli. We also found similar generators to underlie the processing of contingency violations across stimulus types, as well as similar performance in an active-listening follow-up test. However, the process of passive contingency extraction was considerably slower (twice as many rule exemplars were needed) for verbal than for tonal stimuli These results suggest caution in transferring findings on complex predictive regularity processing obtained with tonal stimuli directly to the speech domain.

Visuo-tactile interactions in the congenitally deaf: a behavioral and event-related potential study

Auditory deprivation is known to be accompanied by alterations in visual processing. Yet not much is known about tactile processing and the interplay of the intact sensory modalities in the deaf. We presented visual, tactile, and visuo-tactile stimuli to congenitally deaf and hearing individuals in a speeded detection task. Analyses of multisensory responses showed a redundant signals effect that was attributable to a coactivation mechanism in both groups, although the redundancy gain was less in the deaf. In line with these behavioral results, on a neural level, there were multisensory interactions in both groups that were again weaker in the deaf. In hearing but not deaf participants, somatosensory event-related potential N200 latencies were modulated by simultaneous visual stimulation. A comparison of unisensory responses between groups revealed larger N200 amplitudes for visual and shorter N200 latencies for tactile stimuli in the deaf. Furthermore, P300 amplitudes were also larger in the deaf. This group difference was significant for tactile and approached significance for visual targets. The differences in visual and tactile processing between deaf and hearing participants, however, were not reflected in behavior. Both the behavioral and electroencephalography (EEG) results suggest more pronounced multisensory interaction in hearing than in deaf individuals. Visuo-tactile enhancements could not be explained by perceptual deficiency, but could be partly attributable to inverse effectiveness.