Signal clustering modulates auditory cortical activity in humans

Hearing in categories aids speech streaming at the "cocktail party"

Our perceptual system bins elements of the speech signal into categories to make speech perception manageable. Here, we aimed to test whether hearing speech in categories (as opposed to a continuous/gradient fashion) affords yet another benefit to speech recognition: parsing noisy speech at the "cocktail party." We measured speech recognition in a simulated 3D cocktail party environment. We manipulated task difficulty by varying the number of additional maskers presented at other spatial locations in the horizontal soundfield (1-4 talkers) and via forward vs. time-reversed maskers, promoting more and less informational masking (IM), respectively. In separate tasks, we measured isolated phoneme categorization using two-alternative forced choice (2AFC) and visual analog scaling (VAS) tasks designed to promote more/less categorical hearing and thus test putative links between categorization and real-world speech-in-noise skills. We first show that listeners can only monitor up to ~3 talkers despite up to 5 in the soundscape and streaming is not related to extended high-frequency hearing thresholds (though QuickSIN scores are). We then confirm speech streaming accuracy and speed decline with additional competing talkers and amidst forward compared to reverse maskers with added IM. Dividing listeners into "discrete" vs. "continuous" categorizers based on their VAS labeling (i.e., whether responses were binary or continuous judgments), we then show the degree of IM experienced at the cocktail party is predicted by their degree of categoricity in phoneme labeling; more discrete listeners are less susceptible to IM than their gradient responding peers. Our results establish a link between speech categorization skills and cocktail party processing, with a categorical (rather than gradient) listening strategy benefiting degraded speech perception. These findings imply figure-ground deficits common in many disorders might arise through a surprisingly simple mechanism: a failure to properly bin sounds into categories.

Cortical effects of wrist tendon vibration during an arm tracking task in chronic stroke survivors: An EEG study

The purpose of this study was to characterize changes in cortical activity and connectivity in stroke survivors when vibration is applied to the wrist flexor tendons during a visuomotor tracking task. Data were collected from 10 chronic stroke participants and 10 neurologically-intact controls while tracking a target through a figure-8 pattern in the horizontal plane. Electroencephalography (EEG) was used to measure cortical activity (beta band desynchronization) and connectivity (beta band task-based coherence) with movement kinematics and performance error also being recorded during the task. All participants came into our lab on two separate days and performed three blocks (16 trials each, 48 total trials) of tracking, with the middle block including vibration or sham applied at the wrist flexor tendons. The order of the sessions (Vibe vs. Sham) was counterbalanced across participants to prevent ordering effects. During the Sham session, cortical activity increased as the tracking task progressed (over blocks). This effect was reduced when vibration was applied to controls. In contrast, vibration increased cortical activity during the vibration period in participants with stroke. Cortical connectivity increased during vibration, with larger effect sizes in participants with stroke. Changes in tracking performance, standard deviation of hand speed, were observed in both control and stroke groups. Overall, EEG measures of brain activity and connectivity provided insight into effects of vibration on brain control of a visuomotor task. The increases in cortical activity and connectivity with vibration improved patterns of activity in people with stroke. These findings suggest that reactivation of normal cortical networks via tendon vibration may be useful during physical rehabilitation of stroke patients.

Auditory cortex activity related to perceptual awareness versus masking of tone sequences

Sequences of repeating tones can be masked by other tones of different frequency. When these tone sequences are perceived, nevertheless, a prominent neural response in the auditory cortex is evoked by each tone of the sequence. When the targets are detected based on their isochrony, participants know that they are listening to the target once they detected it. To explore if the neural activity is more closely related to this detection task or to perceptual awareness, this magnetoencephalography (MEG) study used targets that could only be identified with cues provided after or before the masked target. In experiment 1, multiple mono-tone streams with jittered inter-stimulus interval were used, and the tone frequency of the target was indicated by a cue. Results showed no differential auditory cortex activity between hit and miss trials with post-stimulus cues. A late negative response for hit trials was only observed for pre-stimulus cues, suggesting a task-related component. Since experiment 1 provided no evidence for a link of a difference response with tone awareness, experiment 2 was planned to probe if detection of tone streams was linked to a difference response in auditory cortex. Random-tone sequences were presented in the presence of a multi-tone masker, and the sequence was repeated without masker thereafter. Results showed a prominent difference wave for hit compared to miss trials in experiment 2 evoked by targets in the presence of the masker. These results suggest that perceptual awareness of tone streams is linked to neural activity in auditory cortex.

Subjective perceptual organization of a complex auditory scene

Empirical research on the sequential decomposition of an auditory scene primarily relies on interleaved sound mixtures of only two tone sequences (e.g., ABAB…). This oversimplifies the sound decomposition problem by limiting the number of putative perceptual organizations. The current study used a sound mixture composed of three different tones (ABCABC…) that could be perceptually organized in many different ways. Participants listened to these sequences and reported their subjective perception by continuously choosing one out of 12 visually presented perceptual organization alternatives. Different levels of frequency and spatial separation were implemented to check whether participants' perceptual reports would be systematic and plausible. As hypothesized, while perception switched back and forth in each condition between various perceptual alternatives (multistability), spatial as well as frequency separation generally raised the proportion of segregated and reduced the proportion of integrated alternatives. During segregated percepts, in contrast to the hypothesis, many participants had a tendency to perceive two streams in the foreground, rather than reporting alternatives with a clear foreground-background differentiation. Finally, participants perceived the organization with intermediate feature values (e.g., middle tones of the pattern) segregated in the foreground slightly less often than similar alternatives with outer feature values (e.g., higher tones).

Sound segregation via embedded repetition is robust to inattention

The segregation of sound sources from the mixture of sounds that enters the ear is a core capacity of human hearing, but the extent to which this process is dependent on attention remains unclear. This study investigated the effect of attention on the ability to segregate sounds via repetition. We utilized a dual task design in which stimuli to be segregated were presented along with stimuli for a "decoy" task that required continuous monitoring. The task to assess segregation presented a target sound 10 times in a row, each time concurrent with a different distractor sound. McDermott, Wrobleski, and Oxenham (2011) demonstrated that repetition causes the target sound to be segregated from the distractors. Segregation was queried by asking listeners whether a subsequent probe sound was identical to the target. A control task presented similar stimuli but probed discrimination without engaging segregation processes. We present results from 3 different decoy tasks: a visual multiple object tracking task, a rapid serial visual presentation (RSVP) digit encoding task, and a demanding auditory monitoring task. Load was manipulated by using high- and low-demand versions of each decoy task. The data provide converging evidence of a small effect of attention that is nonspecific, in that it affected the segregation and control tasks to a similar extent. In all cases, segregation performance remained high despite the presence of a concurrent, objectively demanding decoy task. The results suggest that repetition-based segregation is robust to inattention.

Stream segregation in the anesthetized auditory cortex

Auditory stream segregation describes the way that sounds are perceptually segregated into groups or streams on the basis of perceptual attributes such as pitch or spectral content. For sequences of pure tones, segregation depends on the tones' proximity in frequency and time. In the auditory cortex (and elsewhere) responses to sequences of tones are dependent on stimulus conditions in a similar way to the perception of these stimuli. However, although highly dependent on stimulus conditions, perception is also clearly influenced by factors unrelated to the stimulus, such as attention. Exactly how ‘bottom-up’ sensory processes and non-sensory ‘top-down’ influences interact is still not clear.

Here, we recorded responses to alternating tones (ABAB …) of varying frequency difference (FD) and rate of presentation (PR) in the auditory cortex of anesthetized guinea-pigs. These data complement previous studies, in that top-down processing resulting from conscious perception should be absent or at least considerably attenuated.

Under anesthesia, the responses of cortical neurons to the tone sequences adapted rapidly, in a manner sensitive to both the FD and PR of the sequences. While the responses to tones at frequencies more distant from neuron best frequencies (BFs) decreased as the FD increased, the responses to tones near to BF increased, consistent with a release from adaptation, or forward suppression. Increases in PR resulted in reductions in responses to all tones, but the reduction was greater for tones further from BF. Although asymptotically adapted responses to tones showed behavior that was qualitatively consistent with perceptual stream segregation, responses reached asymptote within 2 s, and responses to all tones were very weak at high PRs (>12 tones per second).

A signal-detection model, driven by the cortical population response, made decisions that were dependent on both FD and PR in ways consistent with perceptual stream segregation. This included showing a range of conditions over which decisions could be made either in favor of perceptual integration or segregation, depending on the model ‘decision criterion’. However, the rate of ‘build-up’ was more rapid than seen perceptually, and at high PR responses to tones were sometimes so weak as to be undetectable by the model.

Under anesthesia, adaptation occurs rapidly, and at high PRs tones are generally poorly represented, which compromises the interpretation of the experiment. However, within these limitations, these results complement experiments in awake animals and humans. They generally support the hypothesis that ‘bottom-up’ sensory processing plays a major role in perceptual organization, and that processes underlying stream segregation are active in the absence of attention.

•

We recorded responses of cortical neurons to sequences of tones under anesthesia.

•

Fully adapted responses correlated reasonably with perceptual stream segregation.

•

Responses to tone sequences were weak during rapid tone presentation (>12 Hz).

•

Adaptation under anesthesia is too rapid to account for perceptual ‘build-up’.

•

Neural correlates of stream segregation are not reliant on top-down influences.

Paying attention to MMN in schizophrenia

The aim of this review is to explore the phenomenon of reduced mismatch negativity (MMN) in persons with schizophrenia and the possible relationship it has with attention impairments. In doing so we discuss (i) the prediction error account of MMN, (ii) reduced MMN as a faulty predictive processing system in persons with schizophrenia, (iii) the role of these systems in relevance filtering and attentional resource protection, (iv) attentional impairments in persons with schizophrenia, and (v) research that has explored MMN and attention in schizophrenia groups. Our review of the literature suggests that no study has appropriately examined the functional impact of smaller MMN in schizophrenia on the performance of a concurrent attention task. We conclude that future research should explore this notion further in the hope that it might embed MMN findings within outcomes of functional significance to individuals with the illness and those providing treatment. This article is part of a Special Issue entitled SI: Prediction and Attention.

Neural correlates of auditory stream segregation: an analysis of onset- and change-related responses

The temporal order discrimination of target tone pairs is hindered by the presence of flanker tones but is improved when the flanker tones are captured by a separate stream of tones that match the flankers in frequency [Bregman and Rudnicky (1975). J. Exp. Psychol. 1, 263-267]. In an event-related potential (ERP) study with these stimuli, listeners' mismatch negativity (MMN) responses were temporally linked to the position of the changing target tones, irrespective of streaming. In contrast, N1 response latency varied as a function of the perceived grouping of flanker tones established by previous behavioral studies, providing a neurophysiological index of auditory stream segregation.

Rejection positivity predicts trial-to-trial reaction times in an auditory selective attention task: a computational analysis of inhibitory control

A series of computer simulations using variants of a formal model of attention (Melara and Algom, 2003) probed the role of rejection positivity (RP), a slow-wave electroencephalographic (EEG) component, in the inhibitory control of distraction. Behavioral and EEG data were recorded as participants performed auditory selective attention tasks. Simulations that modulated processes of distractor inhibition accounted well for reaction-time (RT) performance, whereas those that modulated target excitation did not. A model that incorporated RP from actual EEG recordings in estimating distractor inhibition was superior in predicting changes in RT as a function of distractor salience across conditions. A model that additionally incorporated momentary fluctuations in EEG as the source of trial-to-trial variation in performance precisely predicted individual RTs within each condition. The results lend support to the linking proposition that RP controls the speed of responding to targets through the inhibitory control of distractors.

Differential modulation of auditory responses to attended and unattended speech in different listening conditions

This study investigates how top-down attention modulates neural tracking of the speech envelope in different listening conditions. In the quiet conditions, a single speech stream was presented and the subjects paid attention to the speech stream (active listening) or watched a silent movie instead (passive listening). In the competing speaker (CS) conditions, two speakers of opposite genders were presented diotically. Ongoing electroencephalographic (EEG) responses were measured in each condition and cross-correlated with the speech envelope of each speaker at different time lags. In quiet, active and passive listening resulted in similar neural responses to the speech envelope. In the CS conditions, however, the shape of the cross-correlation function was remarkably different between the attended and unattended speech. The cross-correlation with the attended speech showed stronger N1 and P2 responses but a weaker P1 response compared to the cross-correlation with the unattended speech. Furthermore, the N1 response to the attended speech in the CS condition was enhanced and delayed compared with the active listening condition in quiet, while the P2 response to the unattended speaker in the CS condition was attenuated compared with the passive listening in quiet. Taken together, these results demonstrate that top-down attention differentially modulates envelope-tracking neural activity at different time lags and suggest that top-down attention can both enhance the neural responses to the attended sound stream and suppress the responses to the unattended sound stream.

Impaired Facilitatory Mechanisms of Auditory Attention After Damage of the Lateral Prefrontal Cortex

There is growing evidence that auditory selective attention operates via distinct facilitatory and inhibitory mechanisms enabling selective enhancement and suppression of sound processing, respectively. The lateral prefrontal cortex (LPFC) plays a crucial role in the top-down control of selective attention. However, whether the LPFC controls facilitatory, inhibitory, or both attentional mechanisms is unclear. Facilitatory and inhibitory mechanisms were assessed, in patients with LPFC damage, by comparing event-related potentials (ERPs) to attended and ignored sounds with ERPs to these same sounds when attention was equally distributed to all sounds. In control subjects, we observed 2 late frontally distributed ERP components: a transient facilitatory component occurring from 150 to 250 ms after sound onset; and an inhibitory component onsetting at 250 ms. Only the facilitatory component was affected in patients with LPFC damage: this component was absent when attending to sounds delivered in the ear contralateral to the lesion, with the most prominent decreases observed over the damaged brain regions. These findings have 2 important implications: (i) they provide evidence for functionally distinct facilitatory and inhibitory mechanisms supporting late auditory selective attention; (ii) they show that the LPFC is involved in the control of the facilitatory mechanisms of auditory attention.

Attention to memory: orienting attention to sound object representations

Despite a growing acceptance that attention and memory interact, and that attention can be focused on an active internal mental representation (i.e., reflective attention), there has been a paucity of work focusing on reflective attention to 'sound objects' (i.e., mental representations of actual sound sources in the environment). Further research on the dynamic interactions between auditory attention and memory, as well as its degree of neuroplasticity, is important for understanding how sound objects are represented, maintained, and accessed in the brain. This knowledge can then guide the development of training programs to help individuals with attention and memory problems. This review article focuses on attention to memory with an emphasis on behavioral and neuroimaging studies that have begun to explore the mechanisms that mediate reflective attentional orienting in vision and more recently, in audition. Reflective attention refers to situations in which attention is oriented toward internal representations rather than focused on external stimuli. We propose four general principles underlying attention to short-term memory. Furthermore, we suggest that mechanisms involved in orienting attention to visual object representations may also apply for orienting attention to sound object representations.

EEG investigations of duration discrimination: the intermodal effect is induced by an attentional bias

Previous studies indicated that empty time intervals are better discriminated in the auditory than in the visual modality, and when delimited by signals delivered from the same (intramodal intervals) rather than from different sensory modalities (intermodal intervals). The present electrophysiological study was conducted to determine the mechanisms which modulated the performances in inter- and intramodal conditions. Participants were asked to categorise as short or long empty intervals marked by auditory (A) and/or visual (V) signals (intramodal intervals: AA, VV; intermodal intervals: AV, VA). Behavioural data revealed that the performances were higher for the AA intervals than for the three other intervals and lower for inter- compared to intramodal intervals. Electrophysiological results indicated that the CNV amplitude recorded at fronto-central electrodes increased significantly until the end of the presentation of the long intervals in the AA conditions, while no significant change in the time course of this component was observed for the other three modalities of presentation. They also indicated that the N1 and P2 amplitudes recorded after the presentation of the signals which delimited the beginning of the intervals were higher for the inter- (AV/VA) compared to the intramodal intervals (AA/VV). The time course of the CNV revealed that the high performances observed with AA intervals would be related to the effectiveness of the neural mechanisms underlying the processing of the ongoing interval. The greater amplitude of the N1 and P2 components during the intermodal intervals suggests that the weak performances observed in these conditions would be caused by an attentional bias induced by the cognitive load and the necessity to switch between modalities.

Demand and modality of directed attention modulate "pre-attentive" sensory processes in schizophrenia patients and nonpsychiatric controls

BACKGROUND

Mismatch negativity (MNN) and P3a are event related potential (ERP) measures of early sensory information processing. These components are usually conceptualized as being "pre-attentive" and therefore immune to changes with variations in attentional functioning. This study aimed to determine whether manipulations of attention influence the amplitudes and latencies of MMN and P3a and, if so, the extent to which these early sensory processes govern concurrent behavioral vigilance performance in schizophrenia patients and normal subjects.

METHODS

Schizophrenia patients (SZ; n = 20) and Nonpsychiatric Control Subjects (NCS; n = 20) underwent auditory ERP testing to assess MMN and P3a across 4 EEG recording sessions in which attentional demand (low vs. high) and sensory modality of directed attention (visual vs. auditory) were experimentally varied.

RESULTS

Across conditions, SZ patients exhibited deficits in MMN and P3a amplitudes. Significant amplitude and latency modulation were observed in both SZ and NCS but there were no group-by-condition interactions. The amount of MMN amplitude attenuation from low- to high-demand tasks was significantly associated with increased vigilance performance in both SZ and NCS groups (r = -0.67 and r = -0.60). Several other robust associations were also observed among neurophysiologic, clinical and cognitive variables.

CONCLUSIONS

Attentional demand and modality of directed attention significantly influence the amplitude and latencies of "pre-attentive" ERP components in both SZ and NCS. Deficits in MMN and P3a were not "normalized" when attention was directed to the auditory stimuli in schizophrenia patients. The adaptive modulation of early sensory information processing appears to govern concurrent attentional task performance. The temporal window reflecting automatic sensory discrimination as indexed as MMN and P3a may serve as a gateway to some higher order cognitive operations necessary for psychosocial functioning.

Implicit temporal expectation attenuates auditory attentional blink

Attentional blink (AB) describes a phenomenon whereby correct identification of a first target impairs the processing of a second target (i.e., probe) nearby in time. Evidence suggests that explicit attention orienting in the time domain can attenuate the AB. Here, we used scalp-recorded, event-related potentials to examine whether auditory AB is also sensitive to implicit temporal attention orienting. Expectations were set up implicitly by varying the probability (i.e., 80% or 20%) that the probe would occur at the +2 or +8 position following target presentation. Participants showed a significant AB, which was reduced with the increased probe probability at the +2 position. The probe probability effect was paralleled by an increase in P3b amplitude elicited by the probe. The results suggest that implicit temporal attention orienting can facilitate short-term consolidation of the probe and attenuate auditory AB.

Attention, awareness, and the perception of auditory scenes

Auditory perception and cognition entails both low-level and high-level processes, which are likely to interact with each other to create our rich conscious experience of soundscapes. Recent research that we review has revealed numerous influences of high-level factors, such as attention, intention, and prior experience, on conscious auditory perception. And recently, studies have shown that auditory scene analysis tasks can exhibit multistability in a manner very similar to ambiguous visual stimuli, presenting a unique opportunity to study neural correlates of auditory awareness and the extent to which mechanisms of perception are shared across sensory modalities. Research has also led to a growing number of techniques through which auditory perception can be manipulated and even completely suppressed. Such findings have important consequences for our understanding of the mechanisms of perception and also should allow scientists to precisely distinguish the influences of different higher-level influences.

Regular patterns stabilize auditory streams

The auditory system continuously parses the acoustic environment into auditory objects, usually representing separate sound sources. Sound sources typically show characteristic emission patterns. These regular temporal sound patterns are possible cues for distinguishing sound sources. The present study was designed to test whether regular patterns are used as cues for source distinction and to specify the role that detecting these regularities may play in the process of auditory stream segregation. Participants were presented with tone sequences, and they were asked to continuously indicate whether they perceived the tones in terms of a single coherent sequence of sounds (integrated) or as two concurrent sound streams (segregated). Unknown to the participant, in some stimulus conditions, regular patterns were present in one or both putative streams. In all stimulus conditions, participants' perception switched back and forth between the two sound organizations. Importantly, regular patterns occurring in either one or both streams prolonged the mean duration of two-stream percepts, whereas the duration of one-stream percepts was unaffected. These results suggest that temporal regularities are utilized in auditory scene analysis. It appears that the role of this cue lies in stabilizing streams once they have been formed on the basis of simpler acoustic cues.

ERP evidence of early cross-modal links between auditory selective attention and visuo-spatial memory

Previous dual-task research pairing complex visual tasks involving non-spatial cognitive processes during dichotic listening have shown effects on the late component (Ndl) of the negative difference selective attention waveform but no effects on the early (Nde) response suggesting that the Ndl, but not the Nde, is affected by non-spatial processing in a dual-task. Thus to further explore the nature of this dissociation and whether the Nd waveform is affected by spatial processing; fourteen adult participants performed auditory dichotic listening in conjunction with visuo-spatial memory in a cross-modal dual-task paradigm. The results showed that the visuo-spatial memory task decreased both the Nde and Ndl waveforms, and also attenuated P300 and increased its latency. This pattern of results suggests that: (1) the Nde reflects a memory trace that is shared with vision when the information is spatial in nature, and (2) P300 latency appears to be influenced by the discriminability of stimuli underlying the Nde and Ndl memory trace.

Selective attention to pitch amid conflicting auditory information: context-coding and filtering strategies

An auditory Eriksen-flanker task was used to study how conflicting information interferes with selective attention to task-relevant differences in pure-tone frequency. Across the observation intervals of the discrimination task, the relevant frequency differences between target tones were positive, but within an observation interval, they could appear to be small or negative relative to conflicting differences in flanker tones leading or trailing the target. Being correct required attending to the between-target and ignoring the target-flanker pitch relation (across and within observation-interval, respectively). The interference index was an elevation of conflict-laden frequency discrimination thresholds (FDTs), relative to no-conflict FDTs. When conflicting differences in frequency or level (but not in duration) trailed the relevant differences, interference (i.e., FDT elevation) was large and persistent, increased with the target-flanker time proximity, but decreased with extensive training. Interference occurs when the target-flanker pitch relation is more prominent than the one between targets, and the physical and/or perceptual effects of relevant and conflicting differences tend to cancel one another, as with the above conflicting differences. With untrained participants, the target-flanker pitch relation is most prominent in conditions fostering both the perceptual grouping of the target and flanker (e.g., close time proximity), and the recency and salience of the conflicting differences (e.g., trailing conflicting difference); conversely, by lessening such grouping and salience, prolonged training decreases or nullifies the interference. The interference observed herein does not arise because the relevant and the conflicting differences each prompt separate decisions or responses that are in mutual conflict; instead, it arises from the early-stage interaction between their perceptual effects.

Neural dynamics of attending and ignoring in human auditory cortex

Studies in all sensory modalities have demonstrated amplification of early brain responses to attended signals, but less is known about the processes by which listeners selectively ignore stimuli. Here we use MEG and a new paradigm to dissociate the effects of selectively attending, and ignoring in time. Two different tasks were performed successively on the same acoustic stimuli: triplets of tones (A, B, C) with noise-bursts interspersed between the triplets. In the COMPARE task subjects were instructed to respond when tones A and C were of same frequency. In the PASSIVE task they were instructed to respond as fast as possible to noise-bursts. COMPARE requires attending to A and C and actively ignoring tone B, but PASSIVE involves neither attending to nor ignoring the tones. The data were analyzed separately for frontal and auditory-cortical channels to independently address attentional effects on low-level sensory versus putative control processing. We observe the earliest attend/ignore effects as early as 100 ms post-stimulus onset in auditory cortex. These appear to be generated by modulation of exogenous (stimulus-driven) sensory evoked activity. Specifically related to ignoring, we demonstrate that active-ignoring-induced input inhibition involves early selection. We identified a sequence of early (<200 ms post-onset) auditory cortical effects, comprised of onset response attenuation and the emergence of an inhibitory response, and provide new, direct evidence that listeners actively ignoring a sound can reduce their stimulus related activity in auditory cortex by 100 ms after onset when this is required to execute specific behavioral objectives.

Interaction between attention and bottom-up saliency mediates the representation of foreground and background in an auditory scene

The mechanism by which a complex auditory scene is parsed into coherent objects depends on poorly understood interactions between task-driven and stimulus-driven attentional processes. We illuminate these interactions in a simultaneous behavioral-neurophysiological study in which we manipulate participants' attention to different features of an auditory scene (with a regular target embedded in an irregular background). Our experimental results reveal that attention to the target, rather than to the background, correlates with a sustained (steady-state) increase in the measured neural target representation over the entire stimulus sequence, beyond auditory attention's well-known transient effects on onset responses. This enhancement, in both power and phase coherence, occurs exclusively at the frequency of the target rhythm, and is only revealed when contrasting two attentional states that direct participants' focus to different features of the acoustic stimulus. The enhancement originates in auditory cortex and covaries with both behavioral task and the bottom-up saliency of the target. Furthermore, the target's perceptual detectability improves over time, correlating strongly, within participants, with the target representation's neural buildup. These results have substantial implications for models of foreground/background organization, supporting a role of neuronal temporal synchrony in mediating auditory object formation.

Neural correlates of auditory perceptual awareness under informational masking

Our ability to detect target sounds in complex acoustic backgrounds is often limited not by the ear's resolution, but by the brain's information-processing capacity. The neural mechanisms and loci of this "informational masking" are unknown. We combined magnetoencephalography with simultaneous behavioral measures in humans to investigate neural correlates of informational masking and auditory perceptual awareness in the auditory cortex. Cortical responses were sorted according to whether or not target sounds were detected by the listener in a complex, randomly varying multi-tone background known to produce informational masking. Detected target sounds elicited a prominent, long-latency response (50-250 ms), whereas undetected targets did not. In contrast, both detected and undetected targets produced equally robust auditory middle-latency, steady-state responses, presumably from the primary auditory cortex. These findings indicate that neural correlates of auditory awareness in informational masking emerge between early and late stages of processing within the auditory cortex.

The mismatch negativity (MMN) in basic research of central auditory processing: a review

In the present article, the basic research using the mismatch negativity (MMN) and analogous results obtained by using the magnetoencephalography (MEG) and other brain-imaging technologies is reviewed. This response is elicited by any discriminable change in auditory stimulation but recent studies extended the notion of the MMN even to higher-order cognitive processes such as those involving grammar and semantic meaning. Moreover, MMN data also show the presence of automatic intelligent processes such as stimulus anticipation at the level of auditory cortex. In addition, the MMN enables one to establish the brain processes underlying the initiation of attention switch to, conscious perception of, sound change in an unattended stimulus stream.

Effects of selective attention on the electrophysiological representation of concurrent sounds in the human auditory cortex

In noisy environments, we use auditory selective attention to actively ignore distracting sounds and select relevant information, as during a cocktail party to follow one particular conversation. The present electrophysiological study aims at deciphering the spatiotemporal organization of the effect of selective attention on the representation of concurrent sounds in the human auditory cortex. Sound onset asynchrony was manipulated to induce the segregation of two concurrent auditory streams. Each stream consisted of amplitude modulated tones at different carrier and modulation frequencies. Electrophysiological recordings were performed in epileptic patients with pharmacologically resistant partial epilepsy, implanted with depth electrodes in the temporal cortex. Patients were presented with the stimuli while they either performed an auditory distracting task or actively selected one of the two concurrent streams. Selective attention was found to affect steady-state responses in the primary auditory cortex, and transient and sustained evoked responses in secondary auditory areas. The results provide new insights on the neural mechanisms of auditory selective attention: stream selection during sound rivalry would be facilitated not only by enhancing the neural representation of relevant sounds, but also by reducing the representation of irrelevant information in the auditory cortex. Finally, they suggest a specialization of the left hemisphere in the attentional selection of fine-grained acoustic information.

Human electrophysiological examination of buildup of the precedence effect

Event-related potential correlates of the buildup of precedence effect were examined. Buildup is a type of precedence effect illusion in which perception changes (from hearing two clicks to hearing one click) during a click train. Buildup occurs faster for right-leading than left-leading clicks. Continuous click trains that changed leading sides every 15 clicks were presented. Event-related potential N1 amplitudes became smaller with click train for right-leading only. N1 latency decreased with click trains. Mismatch negativity was seen after lead-lag sides were changed. When the perceived change differed in location (left-to-right), mismatch negativity peaked earlier than when the perceived change differed in location and number of clicks (right-to-left). Results suggest that buildup relates to: N1 refractoriness, event-related potential 'lead domination' and mismatch negativity differences.

Effects of attention on neuroelectric correlates of auditory stream segregation

A general assumption underlying auditory scene analysis is that the initial grouping of acoustic elements is independent of attention. The effects of attention on auditory stream segregation were investigated by recording event-related potentials (ERPs) while participants either attended to sound stimuli and indicated whether they heard one or two streams or watched a muted movie. The stimuli were pure-tone ABA--patterns that repeated for 10.8 sec with a stimulus onset asynchrony between A and B tones of 100 msec in which the A tone was fixed at 500 Hz, the B tone could be 500, 625, 750, or 1000 Hz, and--was a silence. In both listening conditions, an enhancement of the auditory-evoked response (P1-N1-P2 and N1c) to the B tone varied with Deltaf and correlated with perception of streaming. The ERP from 150 to 250 msec after the beginning of the repeating ABA- patterns became more positive during the course of the trial and was diminished when participants ignored the tones, consistent with behavioral studies indicating that streaming takes several seconds to build up. The N1c enhancement and the buildup over time were larger at right than left temporal electrodes, suggesting a right-hemisphere dominance for stream segregation. Sources in Heschl's gyrus accounted for the ERP modulations related to Deltaf-based segregation and buildup. These findings provide evidence for two cortical mechanisms of streaming: automatic segregation of sounds and attention-dependent buildup process that integrates successive tones within streams over several seconds.

Event-related potentials, cognition, and behavior: A biological approach

The prevailing cognitive-psychological accounts of event-related brain potentials (ERPs) assume that ERP components manifest information processing operations leading from stimulus to response. Since this view encounters numerous difficulties already analyzed in previous studies, an alternative view is presented here that regards cortical control of behavior as a repetitive sensorimotor cycle consisting of two phases: (i) feedforward anticipation and (ii) feedback cortical performance. This view allows us to interpret in an integrative manner numerous data obtained from very different domains of ERP studies: from biophysics of ERP waves to their relationship to the processing of language, in which verbal behavior is viewed as likewise controlled by the same two basic control processes: feedforward (hypothesis building) and feedback (hypothesis checking). The proposed approach is intentionally simplified, explaining numerous effects on the basis of few assumptions and relating several levels of analysis: neurophysiology, macroelectrical processes (i.e. ERPs), cognition and behavior. It can, therefore, be regarded as a first approximation to a general theory of ERPs.

“…and were instructed to read a self-selected book while ignoring the auditory stimuli”: The effects of task demands on the mismatch negativity

OBJECTIVE

The Mismatch Negativity (MMN) is commonly recorded while the subject is reading, and instructed to ignore the auditory stimuli. It is generally assumed that the demands of the diversion task will have no effect on the MMN. Several studies, however, have reported that a diversion task presumably requiring strong attentional focus is associated with a smaller MMN than that elicited during a less demanding task. This study examines the effect of variations in the classical reading paradigm on the MMN.

METHODS

In Experiment 1, event-related potentials (ERP) were recorded while subjects were presented with standard (80 dB SPL 1000 Hz) and frequency deviant (1050 Hz) stimuli. Subjects were instructed to ignore the tone pips and, in separate conditions, engage in different tasks. They were asked to read a text or to sit passively. Subjects were informed that they would subsequently be queried or not about the content of the reading. In Experiment 2, the auditory sequence included the same standard (80 dB SPL 1000 Hz) but the deviant was changed to an intensity decrement (70 dB SPL). A different sample of subjects was again asked to ignore the auditory stimuli and engage in different reading tasks that would or not be followed by query.

RESULTS

In all task conditions, MMN was elicited by the frequency and intensity change. The intensity MMN did not significantly vary with task. A significant effect of task was, however, found for the frequency MMN. Its amplitude was largest when subjects were later queried about their reading.

CONCLUSIONS

This finding is counter-intuitive in light of previous research on the attentional modulation of the MMN. The pattern of frequency MMN results may relate to the differences in cortical excitability across tasks.

SIGNIFICANCE

The present results indicate that the nature of the diversion task may affect the MMN. The choice of diversion task during MMN recording should thus be carefully considered.

The late Nd reflects a memory trace containing amodal spatial information

The early Nd reflects the analysis of simple features of selectively attended auditory stimuli, but the precise nature of the more complex processing reflected by the late Nd is unclear. The late but not the early Nd is sensitive to interference from a concurrently presented visual spatial attention switching task. This experiment investigated whether the late Nd is also sensitive to deeper visual attention switching. Twenty-one subjects performed a dichotic listening task concurrently with either visual spatial or visual letter matching attention switching tasks. Late Nd amplitude was reduced by the spatial but not the letter matching task, indicating insensitivity to deeper attention switching. P300 amplitude was reduced by both tasks. Reductions in N100 and P200 were uncorrelated. We propose that, in part, the late Nd reflects an amodal memory trace containing spatial information, possibly involving a "where" rather than a "what" auditory pathway.

The effects of memory scanning on the late Nd and P300: An interference study

Singhal and Fowler (2004) showed that the late negative difference (Nd) waveform elicited during dichotic listening was attenuated by concurrent visual short term memory (STM) scanning, but not long term memory (LTM) scanning. P300 was reduced by both tasks. The present study compared the effects of STM load on the late Nd and P300 by combining dichotic listening and visual memory scanning with varying set sizes. The results showed that the late Nd was sensitive to the introduction of the scanning task, but not to an increase in load. Furthermore, both the auditory and visual P300s were reduced when a second task was introduced, but only the visual P300 decreased as a function of memory-set size. These data suggest that (a) the auditory late Nd reflects working memory, but not memory scanning, (b) late Nd and P300 reflect parallel but distinct working memory processes, and (c) stimulus modality is an important determinant of P300 amplitude.

The ‘F-complex’ and MMN tap different aspects of deviance

OBJECTIVE

To compare the 'F(fusion)-complex' with the Mismatch negativity (MMN), both components associated with automatic detection of changes in the acoustic stimulus flow.

METHODS

Ten right-handed adult native Hebrew speakers discriminated vowel-consonant-vowel (V-C-V) sequences /ada/ (deviant) and /aga/ (standard) in an active auditory 'Oddball' task, and the brain potentials associated with performance of the task were recorded from 21 electrodes. Stimuli were generated by fusing the acoustic elements of the V-C-V sequences as follows: base was always presented in front of the subject, and formant transitions were presented to the front, left or right in a virtual reality room. An illusion of a lateralized echo (duplex sensation) accompanied base fusion with the lateralized formant locations. Source current density estimates were derived for the net response to the fusion of the speech elements (F-complex) and for the MMN, using low-resolution electromagnetic tomography (LORETA). Statistical non-parametric mapping was used to estimate the current density differences between the brain sources of the F-complex and the MMN.

RESULTS

Occipito-parietal regions and prefrontal regions were associated with the F-complex in all formant locations, whereas the vicinity of the supratemporal plane was bilaterally associated with the MMN, but only in case of front-fusion (no duplex effect).

CONCLUSIONS

MMN is sensitive to the novelty of the auditory object in relation to other stimuli in a sequence, whereas the F-complex is sensitive to the acoustic features of the auditory object and reflects a process of matching them with target categories.

SIGNIFICANCE

The F-complex and MMN reflect different aspects of auditory processing in a stimulus-rich and changing environment: content analysis of the stimulus and novelty detection, respectively.

The differential effects of Sternberg short- and long-term memory scanning on the late Nd and P300 in a dual-task paradigm

We previously reported that the late negative difference (Nd) waveform elicited during dichotic listening was reduced in amplitude by a concurrent simulated flying task having a wide variety of cognitive demands (Psychophysiology, 39, 2002, 236). The main purpose of the present study was to determine how tasks involving the specific demands of short-term memory (STM) or long-term memory (LTM) would influence the early and late Nd waveforms. To this end, 16 participants performed dichotic listening alone and in conjunction with the varied-set (STM) and fixed-set (LTM) versions of Sternberg's memory scanning paradigm. Event-related brain potential (ERP) data was collected concurrently from both the auditory and visual tasks. The results showed that the STM task, but not the LTM task, reduced the amplitude of the auditory late Nd. The auditory early Nd component was unaffected by either task. Furthermore, both the auditory and visual P300s were decreased in amplitude by all the dual-task conditions. These data suggest that the auditory late Nd is linked to more specific working memory processes than is P300.

Auditory stream segregation in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration

Auditory stream segregation refers to the organization of sequential sounds into "perceptual streams" reflecting individual environmental sound sources. In the present study, sequences of alternating high and low tones, "...ABAB...," similar to those used in psychoacoustic experiments on stream segregation, were presented to awake monkeys while neural activity was recorded in primary auditory cortex (A1). Tone frequency separation (AF), tone presentation rate (PR), and tone duration (TD) were systematically varied to examine whether neural responses correlate with effects of these variables on perceptual stream segregation. "A" tones were fixed at the best frequency of the recording site, while "B" tones were displaced in frequency from "A" tones by an amount = delta F. As PR increased, "B" tone responses decreased in amplitude to a greater extent than "A" tone responses, yielding neural response patterns dominated by "A" tone responses occurring at half the alternation rate. Increasing TD facilitated the differential attenuation of "B" tone responses. These findings parallel psychoacoustic data and suggest a physiological model of stream segregation whereby increasing delta F, PR, or TD enhances spatial differentiation of "A" tone and "B" tone responses along the tonotopic map in A1.

Auditory middle-latency components to fusion of speech elements forming an auditory object

OBJECTIVE

The purpose of this study was to define early brain activity associated with fusion of speech elements to form an auditory object in the middle-latency range preceding the F-Complex.

METHODS

Stimuli were binaural formant transition and base, that were presented separately or fused to form the vowel-consonant-vowel sequence /ada/. Eleven right-handed, adult, native Hebrew speakers listened to 2/s presentations, and the brain potentials from C(z) during the 250 msec following transition onset (in the responses to transition and to the fused word) or following the time it would have been presented (in the response to base alone) were recorded. The net-fusion response was extracted by subtracting the sum of potentials to the base and the formant transition from the potentials to the fused sound.

RESULTS

Auditory middle-latency components, comprising of 9 peaks and troughs were recorded in response to the base, to the formant transition and to the fused /ada/. In general, the responses to the fused object were significantly smaller in peak amplitude and in total activity (area under the curve) resulting in the difference waveform of the net-fusion response that also included 9 peaks, but with opposite polarities.

CONCLUSIONS

The early middle-latency components to fusion indicate that the fusion of speech elements to a word involves inhibition, occlusion or both. The results are in line with the uniqueness of speech perception and the early role of the auditory cortex in speech analysis.

Effects of Attentional Load on Auditory Scene Analysis

The effects of attention on the neural processes underlying auditory scene analysis were investigated through the manipulation of auditory task load. Participants were asked to focus their attention on tuned and mistuned stimuli presented to one ear and to ignore similar stimuli presented to the other ear. For both tuned and mistuned sounds, long (standard) and shorter (deviant) duration stimuli were presented in both ears. Auditory task load was manipulated by varying task instructions. In the easier condition, participants were asked to press a button for deviant sounds (target) at the attended location, irrespective of tuning. In the harder condition, participants were further asked to identify whether the targets were tuned or mistuned. Participants were faster in detecting targets defined by duration only than by both duration and tuning. At the unattended location, deviant stimuli generated a mismatch negativity wave at frontocentral sites whose amplitude decreased with increasing task demand. In comparison, standard mistuned stimuli generated an object-related negativity at central sites whose amplitude was not affected by task difficulty. These results show that the processing of sound sequences is differentially affected by attentional load than is the processing of sounds that occur simultaneously (i.e., sequential vs. simultaneous grouping processes), and that they each recruit distinct neural networks.

The electrophysiological net response ('F-complex') to spatial fusion of speech elements forming an auditory object

OBJECTIVE

The purpose of this study was to define and analyze the brain activity associated with fusion of speech elements to form an auditory object and to study the effects of presenting the elements at different spatial locations (duplex stimulus).

METHODS

Stimuli were formant transitions (presented to the front, left or right of the subject) and base (presented to the front), that fused to result in V-C-V sequences /aga/ and /ada/. Ten right-handed, adult, native Hebrew speakers discriminated each fused stimulus, and the brain potentials associated with performance of the task were recorded from 21 electrodes. The net-fusion response, the 'F(fusion)-complex', was extracted by subtracting the sum of potentials to the base and formant transitions from the potentials to the fused sound. Low resolution electromagnetic tomography analysis (LORETA) was performed to assess the timing and brain location of the fusion process.

RESULTS

The 'F-complex', comprising of the difference N(1), P(2), N(2b) (FN(1), FP(2), FN(2b)) components could be identified for each of the stimuli and reflected a process indicating inhibition, occlusion or both, with right ear advantage in fusion. LORETA analyses indicate sequential processing of speech fusion in the temporal lobes, beginning with right prominence in FN(1) and FP(2) shifting to a more symmetrical pattern in FN(2).

CONCLUSIONS

The electrophysiological correlates of speech fusion highlight the uniqueness of speech perception and the brain areas involved in its analysis.

Postrecognition of interleaved melodies as an indirect measure of auditory stream formation

Primitive processes involved in auditory stream formation are measured with indirect, objective method. A target melody interleaved with a distractor sequence is followed by a probe melody that was identical to the target or differed by 2 notes. Listeners decided whether the probe melody was present or not in the composite sequence. Interleaved melody recognition is not possible when distractor sequences have the same mean frequency and maximum contour crossover with target melodies. Performance increases with mean frequency separation and timbral dissimilarity and is unaffected by the duration of the silent interval between composite sequence and probe melody. The relation between this indirect task measuring the interleaved melody recognition boundary and direct judgments measuring the fission boundary is discussed.

Preattentive auditory context effects

The effects of auditory context on the preattentive and perceptual organization of tone sequences were investigated. Two sets of experiments were conducted in which the pitch of contextual tones was varied, bringing about two different contextual manipulations. Preattentive auditory organization was indexed by the mismatch negativity event-related potential, which is elicited by violations of auditory regularities even when participants ignore the sounds (e.g., by reading a book). The perceptual effects of the contextual manipulations on auditory grouping were assessed using target-detection and order-judgment tasks. The close correspondence found between the effects of auditory context on the perceptual and preattentive measures of auditory grouping suggests that a large part of contextual processing is preattentive.

Stepping out of the spotlight: MMN attenuation as a function of distance from the attended location

In this report we present neurophysiological evidence that spatial separation between attended and unattended sound sources influences a listener's ability to register changes in sounds presented outside the focus of attention. Standard and deviant stimuli were presented at three azimuth locations. Participants were asked to press a key whenever they heard a deviant at a designated location. Mismatch negativity waves were generated for deviants at the attended location and were attenuated for deviants occurring 30 degrees away from the attended location. Mismatch negativities were not observed at distances of 60 degrees or more. The results are consistent with a spotlight model of auditory attention in which the processing of stimuli outside the attentional focus is attenuated as a function of increasing distance from the focus.

Schema-based processing in auditory scene analysis

What is the involvement of what we know in what we perceive? In this article, the contribution of melodic schema-based processes to the perceptual organization of tone sequences is examined. Two unfamiliar six-tone melodies, one of which was interleaved with distractor tones, were presented successively to listeners who were required to decide whether the melodies were identical or different. In one condition, the comparison melody was presented after the mixed sequence: a target melody interleaved with distractor tones. In another condition, it was presented beforehand, so that the listeners had precise knowledge about the melody to be extracted from the mixture. In the latter condition, recognition performance was better and a bias toward same responses was reduced, as compared with the former condition. A third condition, in which the comparison melody presented beforehand was transposed up in frequency, revealed that whereas the performance improvement was explained in part by absolute pitch or frequency priming, relative pitch representation (interval and/or contour structure) may also have played a role. Differences in performance as a function of mean frequency separation between target and distractor sequences, when listeners did or did not have prior knowledge about the target melody, argue for a functional distinction between primitive and schema-based processes in auditory scene analysis.

Listening to polyphonic music recruits domain-general attention and working memory circuits

Polyphonic music combines multiple auditory streams to create complex auditory scenes, thus providing a tool for investigating the neural mechanisms that orient attention in natural auditory contexts. Across two fMRI experiments, we varied stimuli and task demands in order to identify the cortical areas that are activated during attentive listening to real music. In individual experiments and in a conjunction analysis of the two experiments, we found bilateral blood oxygen level dependent (BOLD) signal increases in temporal (the superior temporal gyrus), parietal (the intraparietal sulcus), and frontal (the precentral sulcus, the inferior frontal sulcus and gyrus, and the frontal operculum) areas during selective and global listening, as compared with passive rest without musical stimulation. Direct comparisons of the listening conditions showed significant differences between attending to single timbres (instruments) and attending across multiple instruments, although the patterns that were observed depended on the relative demands of the tasks being compared. The overall pattern of BOLD signal increases indicated that attentive listening to music recruits neural circuits underlying multiple forms of working memory, attention, semantic processing, target detection, and motor imagery. Thus, attentive listening to music appears to be enabled by areas that serve general functions, rather than by music-specific cortical modules.

Effects of water on cortical excitability in humans

The effects of water on cortical excitability, measured using magnetoencephalographic recordings, were investigated in a sample of 19 healthy volunteers in a double-blind, placebo experiment comparing water with saline solution. Spontaneous magnetoencephalogram as well as auditory-evoked magnetic fields were recorded before and after the drinking of 750 mL water (9 subjects) or saline solution (10 subjects) and during and after hyperventilation following the drinking conditions. Hyperventilation was used to enhance the hypothesized synchronizing effect of water on spontaneous magnetoencephalographic activity. In addition, the magnetic fields were measured during a dichotic listening task under attended and unattended conditions. The prediction, that intake of water, because of induced cell swelling, will increase neuronal excitability and lead to an increased synchronization of the spontaneous magnetoencephalogram during hyperventilation was confirmed. Hyperventilation induced an increase of spectral power in all frequency bands particularly theta and delta power after water drinking. Furthermore, there was an increase of magnetic mismatch negativity (MMNm) amplitude in attended conditions and a simultaneous decrease in unattended conditions after water drinking. N1m (magnetic N1 wave) revealed significant changes during experimental conditions: increase after drinking and decrease after hyperventilation in both groups. MMNm for attended conditions showed a high positive correlation with osmolality changes (difference in the mol solute per kg water before and after drinking); N1m and PNm (magnetic processing negativity) as well as MMNm for unattended conditions showed significant correlations with subjective ratings of thirst and mood state.

Time course of auditory cortex activation during speech processing

The purpose of the studies summarized in this report was to determine the time course of auditory cortex involvement in speech and language processing in the context of auditory object formation. Forty-one subjects took part in the three studies summarized in this report. In all three studies, subjects performed a choice-reaction task that required their pressing an appropriate button in response to auditory stimuli (speech/non-speech, good/worse fused phonemes, first/second language words) presented through earphones. Event-related potentials (ERPs) were recorded during performance of the task from 21 scalp electrodes, in addition to peri-ocular electrodes for monitoring eye movements. Current densities within the gray matter of the brain were estimated using the LORETA (low resolution electromagnetic tomography) method. In general, except for some periods, processing phonetic and linguistic information was associated with elevated activity in the left auditory cortex. Peaks in auditory cortex activation corresponded in time to scalp recorded peaks in the latencies of P1 and up to as late as P3. The adjacent posterior temporal areas showed a similar temporal pattern of activation, but tended to be less lateralized to the left, or even biased toward right hemisphere predominance, depending on the stimulus, particularly in the later time frames. The results indicate that the auditory cortex is engaged in auditory processing from its early stages and as long as a few hundreds of msec, even after cessation of the stimulus, defining sounds as distinct auditory objects and differentiating speech from non-speech material, relying on acoustic cues. Hemispheric dominance fluctuates to include activity in the 'non-dominant' hemisphere depending on stimulus type and stage of processing.