ERPs for infrequent omissions and inclusions of stimulus elements

Involuntary motor responses are elicited both by rare sounds and rare pitch changes

Unpredictable deviations from an otherwise regular auditory sequence, as well as rare sounds following a period of silence, are detected automatically. Recent evidence suggests that the latter also elicit quick involuntary modulations of ongoing motor activity emerging as early as 100 ms following sound onset, which was attributed to supramodal processing. We explored such force modulations for both rare and deviant sounds. Participants (N = 29) pinched a force sensitive device and maintained a force of 1-2 N for periods of 1 min. Task-irrelevant tones were presented under two conditions. In the Rare condition, 4000 Hz tones were presented every 8-to-16 s. In the Roving condition, 4000 Hz and 2996 Hz tones were presented at rate of 1 s, with infrequent (p = 1/12) frequency changes. In the Rare condition, transient force modulations were observed with a significant increase at ~ 234 ms, and a decrease at ~ 350 ms. In the Roving condition with low frequency deviant tones, an increase in force was observed at ~ 277 ms followed by a decrease at ~ 413 ms. No significant modulations were observed during perception of high frequency deviants. These results suggest that both rare silence-breaking sounds and low-pitched deviants evoke automatic fluctuations of motor responses, which opens up the possibility that these force modulations are triggered by stimulus-specific change-detection processes.

Stimulus-Specific Adaptation (SSA) in the Auditory System: Functional Relevance and Underlying Mechanisms

Rapid detection of novel stimuli that appear suddenly in the surrounding environment is crucial for an animal's survival. Stimulus-specific adaptation (SSA) may be an important mechanism underlying novelty detection. In this review, we discuss the latest advances in SSA research by addressing four main aspects: 1) the frequency dependence of SSA and the origin of SSA in the auditory cortex: 2) spatial SSA and its comparison with frequency SSA: 3) feature integration in SSA and its implications in novelty detection: 4) functional significance and the physiological mechanism of SSA. Although SSA has been extensively investigated, the cognitive insights from SSA studies are extremely limited. Future work should aim to bridge these gaps.

Evidence for predictions established by phantom sound

Predictions, the bridge between the internal and external worlds, are established by prior experience and updated by sensory stimuli. Responses to omitted but unexpected stimuli, known as omission responses, can break the one-to-one mapping of stimulus-response and can expose predictions established by the preceding stimulus built up. While research into exogenous predictions (driven by external stimuli) is often reported, that into endogenous predictions (driven by internal percepts) is rarely available in the literature. Here, we report evidence for endogenous predictions established by the Zwicker tone illusion, a phantom pure-tone-like auditory percept following notch noises. We found that MMN, P300, and theta oscillations could be recorded using an omission paradigm in subjects who can perceive Zwicker tone illusions, but could not in those who cannot. The MMN and P300 responses relied on attention, but theta oscillations did not. In-depth analysis shows that an increase in single-trial theta power, including total and induced theta, with the endogenous prediction, is lateralized to the left frontal brain areas. Our study depicts that the brain automatically analyzes internal perception, progressively establishes predictions and yields prediction errors in the left frontal region when a violation occurs.

Distraction by violation of sensory predictions: Functional distinction between deviant sounds and unexpected silences

It has been established that participants performing a continuous categorization task respond significantly slower following the presentation of unexpected, task-irrelevant, auditory stimuli, compared to a repetitive (standard) sound. Evidence indicates that such distraction emerges because of the violation of sensory predictions. This has typically been studied by measuring the impact of replacing the repeated sound by a different sound on rare and unpredictable trials. Here, we examine the impact of a different type of violation: the mere omission of the standard sound. Capitalizing upon the recent finding that deviant sounds exert distinct effects on response times as a function of whether participants produced or withheld a response on the previous trial, we present the results of an experiment seeking to disentangle two potential effects of sound omission: deviance distraction and the removal of an unspecific warning signal. The results indicate that deviant sound and the unexpected omission of the standard sound impact response times through, at least partially, distinct mechanisms. Deviant sounds affect performance by triggering the orienting of attention towards a new sensory input. Sound omissions, in contrast, appear to affect performance in part because responses no longer benefit from an unspecific warning signal to prepare for action.

Prediction-Related Frontal-Temporal Network for Omission Mismatch Activity in the Macaque Monkey

Sensory prediction is considered an important element of mismatch negativity (MMN) whose reduction is well known in patients with schizophrenia. Omission MMN is a variant of the MMN which is elicited by the absence of a tone previously sequentially presented. Omission MMN can eliminate the effects of sound differences in typical oddball paradigms and affords the opportunity to identify prediction-related signals in the brain. Auditory predictions are thought to reflect bottom-up and top-down processing within hierarchically organized auditory areas. However, the communications between the various subregions of the auditory cortex and the prefrontal cortex that generate and communicate sensory prediction-related signals remain poorly understood. To explore how the frontal and temporal cortices communicate for the generation and propagation of such signals, we investigated the response in the omission paradigm using electrocorticogram (ECoG) electrodes implanted in the temporal, lateral prefrontal, and orbitofrontal cortices of macaque monkeys. We recorded ECoG data from three monkeys during the omission paradigm and examined the functional connectivity between the temporal and frontal cortices by calculating phase-locking values (PLVs). This revealed that theta- (4-8 Hz), alpha- (8-12 Hz), and low-beta- (12-25 Hz) band synchronization increased at tone onset between the higher auditory cortex and the frontal pole where an early omission response was observed in the event-related potential (ERP). These synchronizations were absent when the tone was omitted. Conversely, low-beta-band (12-25 Hz) oscillation then became stronger for tone omission than for tone presentation approximately 200 ms after tone onset. The results suggest that auditory input is propagated to the frontal pole via the higher auditory cortex and that a reciprocal network may be involved in the generation of auditory prediction and prediction error. As impairments of prediction may underlie MMN reduction in patients with schizophrenia, an aberrant hierarchical temporal-frontal network might be related to this pathological condition.

Word class and word frequency in the MMN looking glass

The effects of lexical meaning and lexical familiarity on auditory deviance detection were investigated by presenting oddball sequences of words, while participants ignored the stimuli. Stimulus sequences were composed of words that were varied in word class (nouns vs. functions words) and frequency of language use (high vs. low frequency) in a factorial design with the roles of frequently presented stimuli (Standards) and infrequently presented ones (Deviants) were fully crossed. Deviants elicited the Mismatch Negativity component of the event-related brain potential. Modulating effects of lexical meaning were obtained, revealing processing advantages for denotationally meaningful items. However, no effect of word frequency was observed. These results demonstrate that an apparently low-level function, such as auditory deviance detection utilizes information from the mental lexicon even for task-irrelevant stimuli.

Perceptual Asymmetries and Auditory Processing of Estonian Quantities

Similar to visual perception, auditory perception also has a clearly described “pop-out” effect, where an element with some extra feature is easier to detect among elements without an extra feature. This phenomenon is better known as auditory perceptual asymmetry. We investigated such asymmetry between shorter or longer duration, and level or falling of pitch of linguistic stimuli that carry a meaning in one language (Estonian), but not in another (Russian). For the mismatch negativity (MMN) experiment, we created four different types of stimuli by modifying the duration of the first vowel [ɑ] (170, 290 ms) and pitch contour (level vs. falling pitch) of the stimuli words (‘SATA,’ ‘SAKI’). The stimuli were synthesized from Estonian words (‘SATA,’ ‘SAKI’) and follow the Estonian language three-way quantity system, which incorporates tonal features (falling pitch contour) together with temporal patterns. This made the meaning of the word dependent on the combination of both features and allows us to compare the relative contribution of duration and pitch contour in discrimination of language stimuli in the brain via MMN generation. The participants of the experiment were 12 Russian native speakers with little or no experience in Estonian and living in Estonia short-term, and 12 Estonian native speakers (age 18–27 years). We found that participants’ perception of the linguistic stimuli differed not only according to the physical features but also according to their native language, confirming that the meaning of the word interferes with the early automatic processing of phonological features. The GAMM and ANOVA analysis of the reversed design results showed that the deviant with longer duration among shorter standards elicited a MMN response with greater amplitude than the short deviant among long standards, while changes in pitch contour (falling vs. level pitch) produced neither strong MMN nor asymmetry. Thus, we demonstrate the effect of language background on asymmetric perception of linguistic stimuli that aligns with those of previous studies (Jaramillo et al., 2000), and contributes to the growing body of knowledge supporting auditory perceptual asymmetry.

Limited Evidence for Sensory Prediction Error Responses in Visual Cortex of Macaques and Humans

A recent formulation of predictive coding theory proposes that a subset of neurons in each cortical area encodes sensory prediction errors, the difference between predictions relayed from higher cortex and the sensory input. Here, we test for evidence of prediction error responses in spiking responses and local field potentials (LFP) recorded in primary visual cortex and area V4 of macaque monkeys, and in complementary electroencephalographic (EEG) scalp recordings in human participants. We presented a fixed sequence of visual stimuli on most trials, and violated the expected ordering on a small subset of trials. Under predictive coding theory, pattern-violating stimuli should trigger robust prediction errors, but we found that spiking, LFP and EEG responses to expected and pattern-violating stimuli were nearly identical. Our results challenge the assertion that a fundamental computational motif in sensory cortex is to signal prediction errors, at least those based on predictions derived from temporal patterns of visual stimulation.

Do rat auditory event related potentials exhibit human mismatch negativity attributes related to predictive coding?

Rodent models play a significant role in understanding disease mechanisms and the screening of new treatments. With regard to psychiatric disorders such as schizophrenia, however, it is difficult to replicate the human symptoms in rodents because these symptoms are often either 'uniquely human' or are only conveyed via self-report. There is a growing interest in rodent mismatch responses (MMRs) as a translatable 'biomarker' for disorders such as schizophrenia. In this review, we will summarize the attributes of human MMN, and discuss the scope of exploring the attributes of human MMN in rodents. Here, we examine how reliably MMRs that are measured in rats mimic human attributes, and present original data examining whether manipulations of stimulus conditions known to modulate human MMN, do the same for rat MMRs. Using surgically-implanted epidural electroencephalographic electrodes and wireless telemetry in freely-moving rats, we observed human-like modulations of MMRs, namely that larger MMRs were elicited to unexpected (deviant) stimuli that a) had a larger change in pitch compared to the expected (standard) stimulus, b) were less frequently presented (lower probability), and c) had no jitter (stable stimulus onset asynchrony) compared to high jitter. Overall, these findings contribute to the mounting evidence for rat MMRs as a good analogue of human MMN, bolstering the development of a novel approach in future to validate the preclinical models based on a translatable biomarker, MMN.

Principles of Temporal Processing Across the Cortical Hierarchy

The world is richly structured on multiple spatiotemporal scales. In order to represent spatial structure, many machine-learning models repeat a set of basic operations at each layer of a hierarchical architecture. These iterated spatial operations - including pooling, normalization and pattern completion - enable these systems to recognize and predict spatial structure, while robust to changes in the spatial scale, contrast and noisiness of the input signal. Because our brains also process temporal information that is rich and occurs across multiple time scales, might the brain employ an analogous set of operations for temporal information processing? Here we define a candidate set of temporal operations, and we review evidence that they are implemented in the mammalian cerebral cortex in a hierarchical manner. We conclude that multiple consecutive stages of cortical processing can be understood to perform temporal pooling, temporal normalization and temporal pattern completion.

Cardio-audio synchronization drives neural surprise response

Successful prediction of future events depends on the brain’s capacity to extract temporal regularities from sensory inputs. Neuroimaging studies mainly investigated regularity processing for exteroceptive sensory inputs (i.e. from outside the body). Here we investigated whether interoceptive signals (i.e. from inside the body) can mediate auditory regularity processing. Human participants passively listened to sound sequences presented in synchrony or asynchrony to their heartbeat while concomitant electroencephalography was recorded. We hypothesized that the cardio-audio synchronicity would induce a brain expectation of future sounds. Electrical neuroimaging analysis revealed a surprise response at 158–270 ms upon omission of the expected sounds in the synchronous condition only. Control analyses ruled out that this effect was trivially based on expectation from the auditory temporal structure or on differences in heartbeat physiological signals. Implicit neural monitoring of temporal regularities across interoceptive and exteroceptive signals drives prediction of future events in auditory sequences.

Test-retest reliability of the magnetic mismatch negativity response to sound duration and omission deviants

Mismatch negativity (MMN) is a neurophysiological measure of auditory novelty detection that could serve as a translational biomarker of psychiatric disorders, such as schizophrenia. However, the replicability of its magnetoencephalographic (MEG) counterpart (MMNm) has been insufficiently addressed. In the current study, test-retest reliability of the MMNm response to both duration and omission deviants was evaluated over two MEG sessions in 16 healthy adults. MMNm amplitudes and latencies were obtained at both sensor- and source-level using a cortically-constrained minimum-norm approach. Intraclass correlations (ICC) were derived to assess stability of MEG responses over time. In addition, signal-to-noise ratios (SNR) and within-subject statistics were obtained in order to determine MMNm detectability in individual participants. ICC revealed robust values at both sensor- and source-level for both duration and omission MMNm amplitudes (ICC = 0.81-0.90), in particular in the right hemisphere, while moderate to strong values were obtained for duration MMNm and omission MMNm peak latencies (ICC = 0.74-0.88). Duration MMNm was robustly identified in individual participants with high SNR, whereas omission MMNm responses were only observed in half of the participants. Our data indicate that MMNm to unexpected duration changes and omitted sounds are highly reproducible, providing support for the use of MEG-parameters in basic and clinical research.

Laryngeal Features Are Phonetically Abstract: Mismatch Negativity Evidence from Arabic, English, and Russian

Many theories of phonology assume that the sound structure of language is made up of distinctive features, but there is considerable debate about how much articulatory detail distinctive features encode in long-term memory. Laryngeal features such as voicing provide a unique window into this question: while many languages have two-way contrasts that can be given a simple binary feature account [±VOICE], the precise articulatory details underlying these contrasts can vary significantly across languages. Here, we investigate a series of two-way voicing contrasts in English, Arabic, and Russian, three languages that implement their voicing contrasts very differently at the articulatory-phonetic level. In three event-related potential experiments contrasting English, Arabic, and Russian fricatives along with Russian stops, we observe a consistent pattern of asymmetric mismatch negativity (MMN) effects that is compatible with an articulatorily abstract and cross-linguistically uniform way of marking two-way voicing contrasts, as opposed to an articulatorily precise and cross-linguistically diverse way of encoding them. Regardless of whether a language is theorized to encode [VOICE] over [SPREAD GLOTTIS], the data is consistent with a universal marking of the [SPREAD GLOTTIS] feature.

No place for /h/: an ERP investigation of English fricative place features

The representational format of speech units in long-term memory is a topic of debate. We present novel event-related brain potential evidence from the Mismatch Negativity (MMN) paradigm that is compatible with abstract, non-redundant feature-based models like the Featurally Underspecified Lexicon (FUL). First, we show that the fricatives /s/ and /f/ display an asymmetric pattern of MMN responses, which is predicted if /f/ has a fully specified place of articulation ([Labial]) but /s/ does not ([Coronal], which is lexically underspecified). Second, we show that when /s/ and /h/ are contrasted, no such asymmetric MMN pattern occurs. The lack of asymmetry suggests both that (i) oral and laryngeal articulators are represented distinctly and that (ii) /h/ has no oral place of articulation in long-term memory. The lack of asymmetry between /s/ and /h/ is also in-line with traditional feature-geometric models of lexical representations.

Attention and prediction in human audition: a lesson from cognitive psychophysiology

Attention is a hypothetical mechanism in the service of perception that facilitates the processing of relevant information and inhibits the processing of irrelevant information. Prediction is a hypothetical mechanism in the service of perception that considers prior information when interpreting the sensorial input. Although both (attention and prediction) aid perception, they are rarely considered together. Auditory attention typically yields enhanced brain activity, whereas auditory prediction often results in attenuated brain responses. However, when strongly predicted sounds are omitted, brain responses to silence resemble those elicited by sounds. Studies jointly investigating attention and prediction revealed that these different mechanisms may interact, e.g. attention may magnify the processing differences between predicted and unpredicted sounds. Following the predictive coding theory, we suggest that prediction relates to predictions sent down from predictive models housed in higher levels of the processing hierarchy to lower levels and attention refers to gain modulation of the prediction error signal sent up to the higher level. As predictions encode contents and confidence in the sensory data, and as gain can be modulated by the intention of the listener and by the predictability of the input, various possibilities for interactions between attention and prediction can be unfolded. From this perspective, the traditional distinction between bottom-up/exogenous and top-down/endogenous driven attention can be revisited and the classic concepts of attentional gain and attentional trace can be integrated.

MMN and P300 are both modulated by the featured/featureless nature of deviant stimuli

OBJECTIVE

This study was designed to test the effect of the featured/featureless nature of deviant stimuli on mismatch negativity (MMN), P300 (P3a and P3b) and on behavioral discrimination performances.

METHODS

Ten healthy adults were submitted to stimuli contrasted by the presence or absence of a frequency sweep. Discrimination performances were collected during the neurophysiological sessions.

RESULTS

MMN, P3a and P3b were much larger for featured deviants than for featureless ones. Behavioral data (d', at ceiling level, and reaction times) were not affected by the featured/featureless nature of the deviant stimulus.

CONCLUSION

Behavioral results and MMN amplitudes are in accordance with our previous study, using the same design albeit in an ignore condition and with collection of the behavioral data deferred until after the neurophysiological sessions. The present study strengthens previous evidence suggesting that two mechanisms contribute to the MMN evoked by featured deviants: the memory comparison process and the adaptation/fresh-afferent phenomenons.

SIGNIFICANCE

We here demonstrate that the neurophysiological processes underlying P300 generation are also impacted by the featured/featureless nature of the deviant stimulus and that the dissociation from behavioral data, which are not impacted, is also observed when both types of data are recorded simultaneously.

Automatic detection of unattended changes in room acoustics

Previous research has shown that the human auditory system continuously monitors its acoustic environment, detecting a variety of irregularities (e.g., deviance from prior stimulation regularity in pitch, loudness, duration, and (perceived) sound source location). Detection of irregularities can be inferred from a component of the event-related brain potential (ERP), referred to as the mismatch negativity (MMN), even in conditions in which participants are instructed to ignore the auditory stimulation. The current study extends previous findings by demonstrating that auditory irregularities brought about by a change in room acoustics elicit a MMN in a passive oddball protocol (acoustic stimuli with differing room acoustics, that were otherwise identical, were employed as standard and deviant stimuli), in which participants watched a fiction movie (silent with subtitles). While the majority of participants reported no awareness for any changes in the auditory stimulation, only one out of 14 participants reported to have become aware of changing room acoustics or sound source location. Together, these findings suggest automatic monitoring of room acoustics.

Veto and Vacillation: A Neural Precursor of the Decision to Withhold Action

The capacity to inhibit a planned action gives human behavior its characteristic flexibility. How this mechanism operates and what factors influence a decision to act or not act remain relatively unexplored. We used EEG readiness potentials (RPs) to examine preparatory activity before each action of an ongoing sequence, in which one action was occasionally omitted. We compared RPs between sequences in which omissions were instructed by a rule (e.g., “omit every fourth action”) and sequences in which the participant themselves freely decided which action to omit. RP amplitude was reduced for actions that immediately preceded a voluntary omission but not a rule-based omission. We also used the regular temporal pattern of the action sequences to explore brain processes linked to omitting an action by time-locking EEG averages to the inferred time when an action would have occurred had it not been omitted. When omissions were instructed by a rule, there was a negative-going trend in the EEG, recalling the rising ramp of an RP. No such component was found for voluntary omissions. The results are consistent with a model in which spontaneously fluctuating activity in motor areas of the brain could bias “free” decisions to act or not.

The computational anatomy of psychosis

This paper considers psychotic symptoms in terms of false inferences or beliefs. It is based on the notion that the brain is an inference machine that actively constructs hypotheses to explain or predict its sensations. This perspective provides a normative (Bayes-optimal) account of action and perception that emphasizes probabilistic representations; in particular, the confidence or precision of beliefs about the world. We will consider hallucinosis, abnormal eye movements, sensory attenuation deficits, catatonia, and delusions as various expressions of the same core pathology: namely, an aberrant encoding of precision. From a cognitive perspective, this represents a pernicious failure of metacognition (beliefs about beliefs) that can confound perceptual inference. In the embodied setting of active (Bayesian) inference, it can lead to behaviors that are paradoxically more accurate than Bayes-optimal behavior. Crucially, this normative account is accompanied by a neuronally plausible process theory based upon hierarchical predictive coding. In predictive coding, precision is thought to be encoded by the post-synaptic gain of neurons reporting prediction error. This suggests that both pervasive trait abnormalities and florid failures of inference in the psychotic state can be linked to factors controlling post-synaptic gain - such as NMDA receptor function and (dopaminergic) neuromodulation. We illustrate these points using biologically plausible simulations of perceptual synthesis, smooth pursuit eye movements and attribution of agency - that all use the same predictive coding scheme and pathology: namely, a reduction in the precision of prior beliefs, relative to sensory evidence.

An Asymmetry in the Automatic Detection of the Presence or Absence of a Frequency Modulation within a Tone: A Mismatch Negativity Study

The infrequent occurrence of a transient feature (deviance; e.g., frequency modulation, FM) in one of the regular occurring sinusoidal tones (standards) elicits the deviance related mismatch negativity (MMN) component of the event-related brain potential. Based on a memory-based comparison, MMN reflects the mismatch between the representations of incoming and standard sounds. The present study investigated to what extent the infrequent exclusion of an FM is detected by the MMN system. For that purpose we measured MMN to deviances that either consisted of the exclusion or inclusion of an FM at an early or late position within the sound that was present or absent, respectively, in the standard. According to the information-content hypothesis, deviance detection relies on the difference in informational content of the deviant relative to that of the standard. As this difference between deviants with FM and standards without FM is the same as in the reversed case, comparable MMNs should be elicited to FM inclusions and exclusions. According to the feature-detector hypothesis, however, the deviance detection depends on the increased activation of feature detectors to additional sound features. Thus, rare exclusions of the FM should elicit no or smaller MMN than FM inclusions. In passive listening condition, MMN was obtained only for the early inclusion, but not for the exclusions nor for the late inclusion of an FM. This asymmetry in automatic deviance detection seems to partly reflect the contribution of feature detectors even though it cannot fully account for the missing MMN to late FM inclusions. Importantly, the behavioral deviance detection performance in the active listening condition did not reveal such an asymmetry, suggesting that the intentional detection of the deviants is based on the difference in informational content. On a more general level, the results partly support the “fresh-afferent” account or an extended memory-comparison based account of MMN.

Predictive coding under the free-energy principle

This paper considers prediction and perceptual categorization as an inference problem that is solved by the brain. We assume that the brain models the world as a hierarchy or cascade of dynamical systems that encode causal structure in the sensorium. Perception is equated with the optimization or inversion of these internal models, to explain sensory data. Given a model of how sensory data are generated, we can invoke a generic approach to model inversion, based on a free energy bound on the model's evidence. The ensuing free-energy formulation furnishes equations that prescribe the process of recognition, i.e. the dynamics of neuronal activity that represent the causes of sensory input. Here, we focus on a very general model, whose hierarchical and dynamical structure enables simulated brains to recognize and predict trajectories or sequences of sensory states. We first review hierarchical dynamical models and their inversion. We then show that the brain has the necessary infrastructure to implement this inversion and illustrate this point using synthetic birds that can recognize and categorize birdsongs.

Auditory size-deviant detection in adults and newborn infants

Auditory size perception refers to the ability to make accurate judgements of the size of a sound source based solely upon the sound emitted from the source. Electro-physiological and behavioural data were collected to test whether sound-source size parameters are detected from task-irrelevant sequences in adults and newborn infants. The mismatch negativity (MMN) obtained from adults indexed automatic detection of changes in size for voices, musical instruments and animal calls, regardless of whether the acoustic change indicated larger or smaller sources. Neonates detected changes in the size of a musical instrument. The data are consistent with the notion that auditory size-deviant detection in humans is an innate automatic process. This conclusion is compatible with the theory that the ability to assess the size of sound sources evolved because it provided selective advantage of being able to detect larger (more competent) suitors and larger (more dangerous) predators.

Cortical circuits for perceptual inference

This paper assumes that cortical circuits have evolved to enable inference about the causes of sensory input received by the brain. This provides a principled specification of what neural circuits have to achieve. Here, we attempt to address how the brain makes inferences by casting inference as an optimisation problem. We look at how the ensuing recognition dynamics could be supported by directed connections and message-passing among neuronal populations, given our knowledge of intrinsic and extrinsic neuronal connections. We assume that the brain models the world as a dynamic system, which imposes causal structure on the sensorium. Perception is equated with the optimisation or inversion of this internal model, to explain sensory input. Given a model of how sensory data are generated, we use a generic variational approach to model inversion to furnish equations that prescribe recognition; i.e., the dynamics of neuronal activity that represents the causes of sensory input. Here, we focus on a model whose hierarchical and dynamical structure enables simulated brains to recognise and predict sequences of sensory states. We first review these models and their inversion under a variational free-energy formulation. We then show that the brain has the necessary infrastructure to implement this inversion and present stimulations using synthetic birds that generate and recognise birdsongs.

ERP evaluation of auditory sensory memory systems in adults with intellectual disability

Auditory sensory memory stage can be functionally divided into two subsystems; transient-detector system and permanent feature-detector system (Naatanen, 1992). We assessed these systems in persons with intellectual disability by measuring event-related potentials (ERPs) N1 and mismatch negativity (MMN), which reflect the two auditory subsystems, respectively. Added to these, P3a (an ERP reflecting stage after sensory memory) was evaluated. Either synthesized vowels or simple tones were delivered during a passive oddball paradigm to adults with and without intellectual disability. ERPs were recorded from midline scalp sites (Fz, Cz, and Pz). Relative to control group, participants with the disability exhibited greater N1 latency and less MMN amplitude. The results for N1 amplitude and MMN latency were basically comparable between both groups. IQ scores in participants with the disability revealed no significant relation with N1 and MMN measures, whereas the IQ scores tended to increase significantly as P3a latency reduced. These outcomes suggest that persons with intellectual disability might own discrete malfunctions for the two detector systems in auditory sensory-memory stage. Moreover, the processes following sensory memory might be partly related to a determinant of mental development.

Comparator and non-comparator mechanisms of change detection in the context of speech--an ERP study

Automatic change detection reflects a cognitive memory-based comparison mechanism as well as a sensorial non-comparator mechanism based on differential states of refractoriness. The purpose of this study was to examine whether the comparator mechanism of the mismatch negativity component (MMN) is differentially affected by the lexical status of the deviant. Event-related potential (ERP) data was collected during an "oddball" paradigm designed to elicit the MMN from 15 healthy subjects that were involved in a counting task. Topography pattern analysis and source estimation were utilized to examine the deviance (deviants vs. standards), cognitive (deviants vs. control counterparts) and refractoriness (standards vs. control counterparts) effects elicited by standard-deviant pairs ("deh-day"; "day-deh"; "teh-tay") embedded within "oddball" blocks. Our results showed that when the change was salient regardless of lexical status (i.e., the /e:/ to /eI/ transition) the response tapped the comparator based-mechanism of the MMN which was located in the cuneus/posterior cingulate, reflected sensitivity to the novelty of the auditory object, appeared in the P2 latency range and mainly involved topography modulations. In contrast, when the novelty was low (i.e., the /eI/ to /e:/ transition) an acoustic change complex was elicited which involved strength modulations over the P1/N1 range and implicated the middle temporal gyrus. This result pattern also resembled the one displayed by the non-comparator mechanism. These findings suggest spatially and temporally distinct brain activities of comparator mechanisms of change detection in the context of speech.

Sensory and cognitive mechanisms of change detection in the context of speech

The aim of this study was to dissociate the contributions of memory-based (cognitive) and adaptation-based (sensory) mechanisms underlying deviance detection in the context of natural speech. Twenty healthy right-handed native speakers of English participated in an event-related design scan in which natural speech stimuli, /de:/ ("deh") and /deI/ ("day"); (/te:/ ("teh") and /teI/ ("tay") served as standards and deviants within functional magnetic resonance imaging event-related "oddball" paradigm designed to elicit the mismatch negativity component. Thus, "oddball" blocks could involve either a word deviant ("day") resulting in a "word advantage" effect, or a non-word deviant ("deh" or "tay"). We utilized an experimental protocol controlling for refractoriness similar to that used previously when deviance detection was studied in the context of tones. Results showed that the cognitive and sensory mechanisms of deviance detection were located in the anterior and posterior auditory cortices, respectively, as was previously found in the context of tones. The cognitive effect, that was most robust for the word deviant, diminished in the "oddball" condition. In addition, the results indicated that the lexical status of the speech stimulus interacts with acoustic factors exerting a top-down modulation of the extent to which novel sounds gain access to the subject's awareness through memory-based processes. Thus, the more salient the deviant stimulus is the more likely it is to be released from the effects of adaptation exerted by the posterior auditory cortex.

Electrophysiological evidence of different interpretative strategies in irony comprehension

We explore the hypothesis that induction of holistic or analytic strategies influences comprehension and processing of highly contextualized expressions of ordinary language, such as irony. Twenty undergraduate students were asked to categorize as coherent or incoherent a group of sentences. Each sentence completed a previous story, so that they could be ironical, literal or nonsensical endings. Participants were asked to evaluate whether each sentence was coherent or incoherent. Half of them were initially instructed to consider whether the sentences made sense (holistic condition); the other half were instructed to consider whether the sentences were congruent or incongruent (analytic condition). Behavioral responses and Event Related Potentials were registered during the experiment. Both behavioral and electrophysiological results allow clearly distinguishing between the holistic and the analytic strategies. The fact that the same set of stimuli elicits different ERP waveforms, depending on the strategy with which they are analyzed, suggests that different cognitive processes and different areas of the brain are operating in each case.

The mismatch negativity (MMN) in basic research of central auditory processing: a review

In the present article, the basic research using the mismatch negativity (MMN) and analogous results obtained by using the magnetoencephalography (MEG) and other brain-imaging technologies is reviewed. This response is elicited by any discriminable change in auditory stimulation but recent studies extended the notion of the MMN even to higher-order cognitive processes such as those involving grammar and semantic meaning. Moreover, MMN data also show the presence of automatic intelligent processes such as stimulus anticipation at the level of auditory cortex. In addition, the MMN enables one to establish the brain processes underlying the initiation of attention switch to, conscious perception of, sound change in an unattended stimulus stream.

Setting the Stage for Automatic Syntax Processing: The Mismatch Negativity as an Indicator of Syntactic Priming

The present study investigated the automaticity of morphosyntactic processes and processes of syntactic structure building using event-related brain potentials. Two experiments were conducted, which contrasted the impact of local subject-verb agreement violations (Experiment 1) and word category violations (Experiment 2) on the mismatch negativity, an early event-related brain potential component reflecting automatic auditory change detection. The two violation types were realized in two-word utterances comparable with regard to acoustic parameters and structural complexity. The grammaticality of the utterances modulated the mismatch negativity response in both experiments, suggesting that both types of syntactic violations were detected automatically within 200 msec after the violation point. However, the topographical distribution of the grammaticality effect varied as a function of violation type, which indicates that the brain mechanisms underlying the processing of subject-verb agreement and word category information may be functionally distinct even at this earliest stage of syntactic analysis. The findings are discussed against the background of studies investigating syntax processing beyond the level of two-word utterances.

Tracking speech comprehension in space and time

A fundamental challenge for the cognitive neuroscience of language is to capture the spatio-temporal patterns of brain activity that underlie critical functional components of the language comprehension process. We combine here psycholinguistic analysis, whole-head magnetoencephalography (MEG), the Mismatch Negativity (MMN) paradigm, and state-of-the-art source localization techniques (Equivalent Current Dipole and L1 Minimum-Norm Current Estimates) to locate the process of spoken word recognition at a specific moment in space and time. The magnetic MMN to words presented as rare "deviant stimuli" in an oddball paradigm among repetitive "standard" speech stimuli, peaked 100-150 ms after the information in the acoustic input, was sufficient for word recognition. The latency with which words were recognized corresponded to that of an MMN source in the left superior temporal cortex. There was a significant correlation (r = 0.7) of latency measures of word recognition in individual study participants with the latency of the activity peak of the superior temporal source. These results demonstrate a correspondence between the behaviorally determined recognition point for spoken words and the cortical activation in left posterior superior temporal areas. Both the MMN calculated in the classic manner, obtained by subtracting standard from deviant stimulus response recorded in the same experiment, and the identity MMN (iMMN), defined as the difference between the neuromagnetic responses to the same stimulus presented as standard and deviant stimulus, showed the same significant correlation with word recognition processes.

Expectancy effects on omission evoked potentials in musicians and non-musicians

Abstract An expanded omitted stimulus paradigm was investigated to determine whether expectancy would modulate the amplitude of the omission evoked potentials (OEPs). In addition, we examined the effects of musical expertise on OEPs. Trials started with 3-7 beats randomly and contained 5 omitted beats. Three types of trials (n = 90) were presented with 1, 2, or 3 beats occurring between omissions. A tap response at the end of each trial was used to determine timing accuracy. Clear OEPs were observed over midline sites. We found main omission effects with respect to an N150 and a P400 OEPs component, such that peak amplitudes diminished whenever the occurrence of an omitted stimulus could be expected. In addition, an N600 OEPs component emerged in response to expectedly omitted stimuli toward the end of each trial within the group of musicians. Thus, musical training seems to lead to more efficient and more refined processing of auditory temporal patterns.

Automatic Auditory Processing of English Words as Indexed by the Mismatch Negativity, Using a Multiple Deviant Paradigm

OBJECTIVE

The aim of this study was to investigate mismatch negativity (MMN) responses to a variety of speech stimuli (/de:/, /ge:/, /deI/ "day", and /geI/ "gay") in a multiple deviant paradigm. It was hypothesized that all speech stimulus contrasts in the multiple deviant paradigm, including the fine acoustic speech contrast [d/g], would elicit robust MMN responses and that consonant vowel (CV) real word deviants (e.g., "day" and "gay") would elicit larger MMN responses than CV nonword deviants (e.g., "de" and "ge") within and across experimental contrasts.

DESIGN

Ten healthy, right-handed, native English-speaking adults (23.4 +/- 2.27 yr) with normal hearing were presented with 12 blocks of stimuli, using a multiple deviant oddball paradigm. Each of the four speech stimuli were presented as standards (p = 0.7) in three blocks, with the remaining stimuli acting as deviants (p = 0.1 each). Subjects were also presented with the same stimuli in a behavioral discrimination task.

RESULTS

MMN responses to the fine acoustic speech contrast [d/g] (e.g., "de" versus "ge", "day" versus "gay") did not reach significance. However, a significant and larger MMN response was obtained at an earlier latency to the real word deviants among nonword standards with the same initial consonant (i.e., de-->day, ge-->gay) when compared with the responses to nonword deviants among word standards (day-->de, gay-->ge).

CONCLUSIONS

The results showed that MMN responses could be elicited by speech stimuli with large, single acoustic deviances, within a multiple deviant paradigm design. This result has positive clinical implications for the testing of subjects who may only tolerate short testing sessions (e.g., pathological populations) in that responses to a wider range of speech stimuli may be recorded without necessarily having to increase session length. The results also demonstrated that MMN responses were elicited by large, single acoustic deviances but not fine acoustic deviances within the speech stimuli. The poor results for the fine acoustic deviances support previous studies that have used single contrast paradigms and found that when carefully controlled methodological designs and strict methods of analysis are applied, robust responses to fine-grained CV syllable contrasts may be difficult to obtain. The enhanced MMN observed in response to the real word deviants among nonword standards may provide further evidence for the presence of long-term neural traces for words in the brain, however possible contextual effects limit the interpretation of these data. Further research is needed to investigate the ability of the MMN response to accurately reflect speech sounds with fine acoustic contrasts, as well as the ability of the MMN to reflect neural traces for words in the brain, before it can be reliably used as a clinical tool in the investigation of spoken word processing in pathological populations.

Rhythmic context influences the auditory evoked potentials of musicians and nonmusicians

In this study, we investigated how rhythms are processed in the brain by measuring both behaviourally obtained ratings and auditory evoked potentials (AEPs) from the EEG. We presented probe beats on seven positions within a test bar. Two bars of either a duple- or triple meter rhythm preceded probe beats. We hypothesised that sequential processing would lead to meter effects at the 1/3 and 1/2bar positions, whereas hierarchical processing would lead to context effects on the 1/3, 1/2 and 2/3bar positions. We found that metric contexts affected behavioural ratings. This effect was more pronounced for rhythmic experts. In addition, both the AEP P3a and P3b component could be identified. Though metric context affected the P3a amplitudes, group effects were less clear. We found that the AEP P3a component is sensitive to violation of temporal expectancies. In addition, behavioural data and P3a correlation coefficients (CCs) suggest that temporal patterns are processed sequentially in nonmusicians but are processed in a hierarchical way in rhythmic experts.

Temporal integration of sequential auditory events: silent period in sound pattern activates human planum temporale

Temporal integration is a fundamental process that the brain carries out to construct coherent percepts from serial sensory events. This process critically depends on the formation of memory traces reconciling past with present events and is particularly important in the auditory domain where sensory information is received both serially and in parallel. It has been suggested that buffers for transient auditory memory traces reside in the auditory cortex. However, previous studies investigating "echoic memory" did not distinguish between brain response to novel auditory stimulus characteristics on the level of basic sound processing and a higher level involving matching of present with stored information. Here we used functional magnetic resonance imaging in combination with a regular pattern of sounds repeated every 100 ms and deviant interspersed stimuli of 100-ms duration, which were either brief presentations of louder sounds or brief periods of silence, to probe the formation of auditory memory traces. To avoid interaction with scanner noise, the auditory stimulation sequence was implemented into the image acquisition scheme. Compared to increased loudness events, silent periods produced specific neural activation in the right planum temporale and temporoparietal junction. Our findings suggest that this area posterior to the auditory cortex plays a critical role in integrating sequential auditory events and is involved in the formation of short-term auditory memory traces. This function of the planum temporale appears to be fundamental in the segregation of simultaneous sound sources.

Automatic processing of grammar in the human brain as revealed by the mismatch negativity

The Mismatch Negativity (MMN), a neurophysiological indicator of cognitive processing, was used to investigate grammatical processes in the absence of focused attention to language. Subjects instructed to watch a silent video film and to ignore speech stimuli heard grammatical and ungrammatical spoken word strings that were physically identical up to a divergence point where they differed between each other by a minimal acoustic event, the presence or the absence of a final -s sound. The sentence we come was presented as a rare deviant stimulus against the background of frequently occurring ungrammatical strings, and, in a different experiment, the ungrammatical string *we comes was the deviant in the reverse design. To control for effects related to differences between the critical words, come and comes, control conditions were used in which the same words were presented out of linguistic context. At 100-150 ms after the divergence point, the ungrammatical deviant stimulus elicited a larger MMN than the correct sentence at left-anterior recording sites. This difference was not seen under the out-of-context conditions. In the time range 100-400 ms after stimulus divergence, a spatiotemporal pattern of grammatically related effects was documented by statistically significant interactions of the word and context variables. Minimum-Norm Current Estimates of the cortical sources of the grammaticality effects revealed a main source in the left frontal cortex. We use a neurobiological model of serial order processing to provide a tentative explanation for the data.

MMN in the visual modality: a review

The mismatch negativity (MMN) component is an event-related potential (ERP) that can be elicited by any change in the acoustic environment, and it is related to memory-based, automatic processing mechanisms, and attentional capture processes. This component is well defined in the auditory modality. However, there is still a great controversy about its existence in the visual modality. This paper reviews the studies that are relevant with regard to memory-based, automatic deviance detection ERPs in the visual system. The paper discusses the main strengths and limitations of those studies and suggests what directions should be taken for future research.

Measuring duration mismatch negativity

OBJECTIVE

Automatic comparisons of sound duration in auditory sensory memory are typically investigated by comparing event-related potentials (ERPs) to standard and deviant stimuli presented in oddball blocks. Deviants elicit mismatch negativity (MMN). This procedure might overestimate an MMN contribution reflecting automatic sensory memory processes because of differential states of refractoriness of respectively recruited neural populations [Neuroreport 1996;7:3005; Psychophysiology 2001;38:723]. Here, memory-comparison-based Duration MMN contributions were investigated using various experimental protocols.

METHODS

Memory-comparison-based first-order Duration MMN was investigated using 4 blocked conditions: (a) descending Deviant (100 ms, P=0.14), 150 ms Standard; (b) reverse ascending Deviant (150 ms), 100 ms Standard; (c) Control comprised of 7 equiprobable durations between 25 and 175 ms; and additionally (d) equiprobable tones between 100 and 400 ms. Using the former 3 conditions, Deviants, Standards and Controls were physically identical.

RESULTS

Comparing Deviants and Controls excluded potential refractoriness effects, and a decomposition of memory-comparison-based MMN and residual MMN was demonstrated. Genuine Duration MMN was also obtained in the deviant-standard-reverse comparison.

CONCLUSIONS

Using a blocked control condition yielded equivalent results to reversing the role of deviant and standard in two separate oddball blocks. Using the reverse ascending deviant condition is thus sufficient as a control.

The internal auditory clock: what can evoked potentials reveal about the analysis of temporal sound patterns, and abnormal states of consciousness?

Whereas in vision a large amount of information may in theory be extracted from instantaneous images, sound exists only in its temporal extent, and most of its information is contained in the pattern of changes over time. The "echoic memory" is a pre-attentive auditory sensory store in which sounds are apparently retained in full temporal detail for a period of a few seconds. From the long-latency auditory evoked potentials to spectro-temporal modulation of complex harmonic tones, at least two automatic sound analysis processes can be identified whose time constants suggest participation of the echoic memory. When a steady tone changes its pitch or timbre, "change-type" CP1, CN1 and CP2 potentials are maximally recorded near the vertex. These potentials appear to reflect a process concerned with the distribution of sound energy across the frequency spectrum. When, on the other hand, changes occur in the temporal pattern of tones (in which individual pitch changes are occurring at a rate sufficiently rapid for the C-potentials to be refractory), a large mismatch negativity (or MN1) and following positivity (MP2) are generated. The amplitude of these potentials is influenced by the degree of regularity of the pattern, larger responses being generated to a "deviant" tone when the pitch and time of occurrence of the "standards" are fully specified by the preceding pattern. At the sudden cessation of changes, on resumption of a steady pitch, a mismatch response is generated whose latency is determined with high precision (in the order of a few milliseconds) by the anticipated time of the next change, which did not in fact occur. The mismatch process, therefore, functions as spectro-temporal auditory pattern analyser, whose consequences are manifested each time the pattern changes. Since calibration of the passage of time is essential for all conscious and subconscious behaviour, is it possible that some states of unconsciousness may be directly due to disruption of internal "clocks"? Abnormal mismatch potentials may provide a manifestation of a disordered auditory time-sense, sometimes being abolished in comatose patients while the C-potentials and similar responses to the onset of tones are preserved. Both C- and M-potentials were usually found to be preserved, however, in patients who had emerged from coma and were capable of discriminating sounds. Substantially intact responses were also recorded from three patients who were functionally in a "vegetative" state. The C- and M-potentials were once again dissociated in a group of patients with multiple sclerosis, only the mismatch potentials being found to be significantly delayed. This subclinical impairment of a memory-based process responsible for the detection of change in temporal sound patterns may be related to defects in other memory domains such as working memory.

Memory traces for inflectional affixes as shown by mismatch negativity

Mismatch negativity (MMN), an index of experience-dependent memory traces, was used to investigate the processing of grammatical affixes in the human brain. The MMN was elicited by either a verb stem or an inflected verb form, phonetic contrasts being identical in both conditions. The topography of the mismatch responses showed clear left-hemispheric laterality in both conditions. However, the MMN to the inflected form occurred later than that for the stem. Furthermore, the inflected stimulus produced MMN maximal in centroparietal sites, whereas stem-elicited MMN was more profound at more frontal sites. We suggest that these features of the MMN to inflected form indicate delayed activation of left-lateralized perisylvian cell assemblies that function as cortical memory traces of inflectional affixes.

Responses of Human Auditory Association Cortex to the Omission of an Expected Acoustic Event

Unexpected auditory events initiate a complex set of event-related potentials (ERPs) that vary in their latency and anatomical localization. Such "mismatch" responses include active responses to the omission of an expected event or the omission of elements in expected stimulus composites. Here we describe intracranial recordings of middle-latency ERPs elicited by the omission of an auditory event. We first presented a sequence of tones at regular temporal intervals and the tone was omitted 20% of the time. In a second condition, we presented a sequence of tone pairs and the second tone of the pair was omitted 20% of the time. These two conditions are complementary in that the single tone conformed to the expectancy in one condition, but violated the expectancy in the other. All patients demonstrated localized cortical responses to missing tones that were topographically similar to the responses evoked by actual tones. Responses to both actual and omitted tones were observed bilaterally in the vicinity of the temporal--parietal junction, where we also obtained midlatency ERPs to a variety of other auditory stimuli. Responses that appeared to be selective for the nonoccurrence of expected tones were also observed in a number of subjects. We interpret these effects in terms of processes associated with the comparison of sensory inputs to the contents of a short-term auditory memory. Such a system could automatically detect deviant auditory events, and provide input to higher-level, task-dependent cognitive processes.