Independence of early speech processing from word meaning

The Temporal Dynamics of Stop Consonant Perception: Evidence from Context Effects

Empirical evidence and theoretical models suggest that phonetic category perception involves two stages of auditory and phonetic processing. However, few studies examined the time course of these two processing stages. With brief stop consonant segments as context stimuli, this study examined the temporal dynamics of stop consonant perception by varying the inter-stimulus interval between context and target stimuli. The results suggest that phonetic category activation of stop consonants may appear before 100 ms of processing time. Furthermore, the activation of phonetic categories resulted in contrast context effects on identifying the target stop continuum; the auditory processing of stop consonants resulted in a different context effect from those caused by phonetic category activation. The findings provide further evidence for the two-stage model of speech perception and reveal the time course of auditory and phonetic processing.

The effect of topic familiarity and volatility of auditory scene on selective auditory attention

Selective auditory attention has been shown to modulate the cortical representation of speech. This effect has been well documented in acoustically more challenging environments. However, the influence of top-down factors, in particular topic familiarity, on this process remains unclear, despite evidence that semantic information can promote speech-in-noise perception. Apart from individual features forming a static listening condition, dynamic and irregular changes of auditory scenes-volatile listening environments-have been less studied. To address these gaps, we explored the influence of topic familiarity and volatile listening on the selective auditory attention process during dichotic listening using electroencephalography. When stories with unfamiliar topics were presented, participants' comprehension was severely degraded. However, their cortical activity selectively tracked the speech of the target story well. This implies that topic familiarity hardly influences the speech tracking neural index, possibly when the bottom-up information is sufficient. However, when the listening environment was volatile and the listeners had to re-engage in new speech whenever auditory scenes altered, the neural correlates of the attended speech were degraded. In particular, the cortical response to the attended speech and the spatial asymmetry of the response to the left and right attention were significantly attenuated around 100-200 ms after the speech onset. These findings suggest that volatile listening environments could adversely affect the modulation effect of selective attention, possibly by hampering proper attention due to increased perceptual load.

Categorical perception and influence of attention on neural consistency in response to speech sounds in adults with dyslexia

Developmental dyslexia is a common neurodevelopmental disorder that is associated with alterations in the behavioral and neural processing of speech sounds, but the scope and nature of that association is uncertain. It has been proposed that more variable auditory processing could underlie some of the core deficits in this disorder. In the current study, magnetoencephalography (MEG) data were acquired from adults with and without dyslexia while they passively listened to or actively categorized tokens from a /ba/-/da/ consonant continuum. We observed no significant group difference in active categorical perception of this continuum in either of our two behavioral assessments. During passive listening, adults with dyslexia exhibited neural responses that were as consistent as those of typically reading adults in six cortical regions associated with auditory perception, language, and reading. However, they exhibited significantly less consistency in the left supramarginal gyrus, where greater inconsistency correlated significantly with worse decoding skills in the group with dyslexia. The group difference in the left supramarginal gyrus was evident only when neural data were binned with a high temporal resolution and was only significant during the passive condition. Interestingly, consistency significantly improved in both groups during active categorization versus passive listening. These findings suggest that adults with dyslexia exhibit typical levels of neural consistency in response to speech sounds with the exception of the left supramarginal gyrus and that this consistency increases during active versus passive perception of speech sounds similarly in the two groups.

Spatiotemporal target selection for intracranial neural decoding of abstract and concrete semantics

Decoding the inner representation of a word meaning from human cortical activity is a substantial challenge in the development of speech brain-machine interfaces (BMIs). The semantic aspect of speech is a novel target of speech decoding that may enable versatile communication platforms for individuals with impaired speech ability; however, there is a paucity of electrocorticography studies in this field. We decoded the semantic representation of a word from single-trial cortical activity during an imageability-based property identification task that required participants to discriminate between the abstract and concrete words. Using high gamma activity in the language-dominant hemisphere, a support vector machine classifier could discriminate the 2-word categories with significantly high accuracy (73.1 ± 7.5%). Activities in specific time components from two brain regions were identified as significant predictors of abstract and concrete dichotomy. Classification using these feature components revealed that comparable prediction accuracy could be obtained based on a spatiotemporally targeted decoding approach. Our study demonstrated that mental representations of abstract and concrete word processing could be decoded from cortical high gamma activities, and the coverage of implanted electrodes and time window of analysis could be successfully minimized. Our findings lay the foundation for the future development of semantic-based speech BMIs.

The Precentral Gyrus Contributions to the Early Time-Course of Grapheme-to-Phoneme Conversion

As part of silent reading models, visual orthographic information is transduced into an auditory phonological code in a process of grapheme-to-phoneme conversion (GPC). This process is often identified with lateral temporal-parietal regions associated with auditory phoneme encoding. However, the role of articulatory phonemic representations and the precentral gyrus in GPC is ambiguous. Though the precentral gyrus is implicated in many functional MRI studies of reading, it is not clear if the time course of activity in this region is consistent with the precentral gyrus being involved in GPC. We recorded cortical electrophysiology during a bimodal match/mismatch task from eight patients with perisylvian subdural electrodes to examine the time course of neural activity during a task that necessitated GPC. Patients made a match/mismatch decision between a 3-letter string and the following auditory bi-phoneme. We characterized the distribution and timing of evoked broadband high gamma (70-170 Hz) as well as phase-locking between electrodes. The precentral gyrus emerged with a high concentration of broadband high gamma responses to visual and auditory language as well as mismatch effects. The pars opercularis, supramarginal gyrus, and superior temporal gyrus were also involved. The precentral gyrus showed strong phase-locking with the caudal fusiform gyrus during letter-string presentation and with surrounding perisylvian cortex during the bimodal visual-auditory comparison period. These findings hint at a role for precentral cortex in transducing visual into auditory codes during silent reading.

Microscale Physiological Events on the Human Cortical Surface

Despite ongoing advances in our understanding of local single-cellular and network-level activity of neuronal populations in the human brain, extraordinarily little is known about their "intermediate" microscale local circuit dynamics. Here, we utilized ultra-high-density microelectrode arrays and a rare opportunity to perform intracranial recordings across multiple cortical areas in human participants to discover three distinct classes of cortical activity that are not locked to ongoing natural brain rhythmic activity. The first included fast waveforms similar to extracellular single-unit activity. The other two types were discrete events with slower waveform dynamics and were found preferentially in upper cortical layers. These second and third types were also observed in rodents, nonhuman primates, and semi-chronic recordings from humans via laminar and Utah array microelectrodes. The rates of all three events were selectively modulated by auditory and electrical stimuli, pharmacological manipulation, and cold saline application and had small causal co-occurrences. These results suggest that the proper combination of high-resolution microelectrodes and analytic techniques can capture neuronal dynamics that lay between somatic action potentials and aggregate population activity. Understanding intermediate microscale dynamics in relation to single-cell and network dynamics may reveal important details about activity in the full cortical circuit.

Chasing language through the brain: Successive parallel networks

OBJECTIVE

To describe the spatio-temporal dynamics and interactions during linguistic and memory tasks.

METHODS

Event-related electrocorticographic (ECoG) spectral patterns obtained during cognitive tasks from 26 epilepsy patients (aged: 9-60 y) were analyzed in order to examine the spatio-temporal patterns of activation of cortical language areas. ECoGs (1024 Hz/channel) were recorded from 1567 subdural electrodes and 510 depth electrodes chronically implanted over or within the frontal, parietal, occipital and/or temporal lobes as part of their surgical work-up for intractable seizures. Six language/memory tasks were performed, which required responding verbally to auditory or visual word stimuli. Detailed analysis of electrode locations allowed combining results across patients.

RESULTS

Transient increases in induced ECoG gamma power (70-100 Hz) were observed in response to hearing words (central superior temporal gyrus), reading text and naming pictures (occipital and fusiform cortex) and speaking (pre-central, post-central and sub-central cortex).

CONCLUSIONS

Between these activations there was widespread spatial divergence followed by convergence of gamma activity that reliably identified cortical areas associated with task-specific processes.

SIGNIFICANCE

The combined dataset supports the concept of functionally-specific locally parallel language networks that are widely distributed, partially interacting in succession to serve the cognitive and behavioral demands of the tasks.

Semantic Context Enhances the Early Auditory Encoding of Natural Speech

Speech perception involves the integration of sensory input with expectations based on the context of that speech. Much debate surrounds the issue of whether or not prior knowledge feeds back to affect early auditory encoding in the lower levels of the speech processing hierarchy, or whether perception can be best explained as a purely feedforward process. Although there has been compelling evidence on both sides of this debate, experiments involving naturalistic speech stimuli to address these questions have been lacking. Here, we use a recently introduced method for quantifying the semantic context of speech and relate it to a commonly used method for indexing low-level auditory encoding of speech. The relationship between these measures is taken to be an indication of how semantic context leading up to a word influences how its low-level acoustic and phonetic features are processed. We record EEG from human participants (both male and female) listening to continuous natural speech and find that the early cortical tracking of a word's speech envelope is enhanced by its semantic similarity to its sentential context. Using a forward modeling approach, we find that prediction accuracy of the EEG signal also shows the same effect. Furthermore, this effect shows distinct temporal patterns of correlation depending on the type of speech input representation (acoustic or phonological) used for the model, implicating a top-down propagation of information through the processing hierarchy. These results suggest a mechanism that links top-down prior information with the early cortical entrainment of words in natural, continuous speech.SIGNIFICANCE STATEMENT During natural speech comprehension, we use semantic context when processing information about new incoming words. However, precisely how the neural processing of bottom-up sensory information is affected by top-down context-based predictions remains controversial. We address this discussion using a novel approach that indexes a word's similarity to context and how well a word's acoustic and phonetic features are processed by the brain at the time of its utterance. We relate these two measures and show that lower-level auditory tracking of speech improves for words that are more related to their preceding context. These results suggest a mechanism that links top-down prior information with bottom-up sensory processing in the context of natural, narrative speech listening.

Increased variability of stimulus-driven cortical responses is associated with genetic variability in children with and without dyslexia

Individuals with dyslexia exhibit increased brainstem variability in response to sound. It is unknown as to whether increased variability extends to neocortical regions associated with audition and reading, extends to visual stimuli, and whether increased variability characterizes all children with dyslexia or, instead, a specific subset of children. We evaluated the consistency of stimulus-evoked neural responses in children with (N = 20) or without dyslexia (N = 12) as measured by magnetoencephalography (MEG). Approximately half of the children with dyslexia had significantly higher levels of variability in cortical responses to both auditory and visual stimuli in multiple nodes of the reading network. There was a significant and positive relationship between the number of risk alleles at rs6935076 in the dyslexia-susceptibility gene KIAA0319 and the degree of neural variability in primary auditory cortex across all participants. This gene has been linked with neural variability in rodents and in typical readers. These findings indicate that unstable representations of auditory and visual stimuli in auditory and other reading-related neocortical regions are present in a subset of children with dyslexia and support the link between the gene KIAA0319 and the auditory neural variability across children with or without dyslexia.

Where did language come from? Precursor mechanisms in nonhuman primates

At first glance, the monkey brain looks like a smaller version of the human brain. Indeed, the anatomical and functional architecture of the cortical auditory system in monkeys is very similar to that of humans, with dual pathways segregated into a ventral and a dorsal processing stream. Yet, monkeys do not speak. Repeated attempts to pin this inability on one particular cause have failed. A closer look at the necessary components of language, according to Darwin, reveals that all of them got a significant boost during evolution from nonhuman to human primates. The vocal-articulatory system, in particular, has developed into the most sophisticated of all human sensorimotor systems with about a dozen effectors that, in combination with each other, result in an auditory communication system like no other. This sensorimotor network possesses all the ingredients of an internal model system that permits the emergence of sequence processing, as required for phonology and syntax in modern languages.

Sub-millimeter ECoG pitch in human enables higher fidelity cognitive neural state estimation

Electrocorticography (ECoG), electrophysiological recording from the pial surface of the brain, is a critical measurement technique for clinical neurophysiology, basic neurophysiology studies, and demonstrates great promise for the development of neural prosthetic devices for assistive applications and the treatment of neurological disorders. Recent advances in device engineering are poised to enable orders of magnitude increase in the resolution of ECoG without comprised measurement quality. This enhancement in cortical sensing enables the observation of neural dynamics from the cortical surface at the micrometer scale. While these technical capabilities may be enabling, the extent to which finer spatial scale recording enhances functionally relevant neural state inference is unclear. We examine this question by employing a high-density and low impedance 400 μm pitch microECoG (μECoG) grid to record neural activity from the human cortical surface during cognitive tasks. By applying machine learning techniques to classify task conditions from the envelope of high-frequency band (70-170Hz) neural activity collected from two study participants, we demonstrate that higher density grids can lead to more accurate binary task condition classification. When controlling for grid area and selecting task informative sub-regions of the complete grid, we observed a consistent increase in mean classification accuracy with higher grid density; in particular, 400 μm pitch grids outperforming spatially sub-sampled lower density grids up to 23%. We also introduce a modeling framework to provide intuition for how spatial properties of measurements affect the performance gap between high and low density grids. To our knowledge, this work is the first quantitative demonstration of human sub-millimeter pitch cortical surface recording yielding higher-fidelity state estimation relative to devices at the millimeter-scale, motivating the development and testing of μECoG for basic and clinical neurophysiology as well as towards the realization of high-performance neural prostheses.

Interaction in Spoken Word Recognition Models: Feedback Helps

Human perception, cognition, and action requires fast integration of bottom-up signals with top-down knowledge and context. A key theoretical perspective in cognitive science is the interactive activation hypothesis: forward and backward flow in bidirectionally connected neural networks allows humans and other biological systems to approximate optimal integration of bottom-up and top-down information under real-world constraints. An alternative view is that online feedback is neither necessary nor helpful; purely feed forward alternatives can be constructed for any feedback system, and online feedback could not improve processing and would preclude veridical perception. In the domain of spoken word recognition, the latter view was apparently supported by simulations using the interactive activation model, TRACE, with and without feedback: as many words were recognized more quickly without feedback as were recognized faster with feedback, However, these simulations used only a small set of words and did not address a primary motivation for interaction: making a model robust in noise. We conducted simulations using hundreds of words, and found that the majority were recognized more quickly with feedback than without. More importantly, as we added noise to inputs, accuracy and recognition times were better with feedback than without. We follow these simulations with a critical review of recent arguments that online feedback in interactive activation models like TRACE is distinct from other potentially helpful forms of feedback. We conclude that in addition to providing the benefits demonstrated in our simulations, online feedback provides a plausible means of implementing putatively distinct forms of feedback, supporting the interactive activation hypothesis.

Prediction, Bayesian inference and feedback in speech recognition

Speech perception involves prediction, but how is that prediction implemented? In cognitive models prediction has often been taken to imply that there is feedback of activation from lexical to pre-lexical processes as implemented in interactive-activation models (IAMs). We show that simple activation feedback does not actually improve speech recognition. However, other forms of feedback can be beneficial. In particular, feedback can enable the listener to adapt to changing input, and can potentially help the listener to recognise unusual input, or recognise speech in the presence of competing sounds. The common feature of these helpful forms of feedback is that they are all ways of optimising the performance of speech recognition using Bayesian inference. That is, listeners make predictions about speech because speech recognition is optimal in the sense captured in Bayesian models.

From perceptual to lexico-semantic analysis--cortical plasticity enabling new levels of processing

Certain kinds of stimuli can be processed on multiple levels. While the neural correlates of different levels of processing (LOPs) have been investigated to some extent, most of the studies involve skills and/or knowledge already present when performing the task. In this study we specifically sought to identify neural correlates of an evolving skill that allows the transition from perceptual to a lexico-semantic stimulus analysis. Eighteen participants were trained to decode 12 letters of Morse code that were presented acoustically inside and outside of the scanner environment. Morse code was presented in trains of three letters while brain activity was assessed with fMRI. Participants either attended to the stimulus length (perceptual analysis), or evaluated its meaning distinguishing words from nonwords (lexico-semantic analysis). Perceptual and lexico-semantic analyses shared a mutual network comprising the left premotor cortex, the supplementary motor area (SMA) and the inferior parietal lobule (IPL). Perceptual analysis was associated with a strong brain activation in the SMA and the superior temporal gyrus bilaterally (STG), which remained unaltered from pre and post training. In the lexico-semantic analysis post learning, study participants showed additional activation in the left inferior frontal cortex (IFC) and in the left occipitotemporal cortex (OTC), regions known to be critically involved in lexical processing. Our data provide evidence for cortical plasticity evolving with a learning process enabling the transition from perceptual to lexico-semantic stimulus analysis. Importantly, the activation pattern remains task-related LOP and is thus the result of a decision process as to which LOP to engage in.

Laminar profile of spontaneous and evoked theta: Rhythmic modulation of cortical processing during word integration

Theta may play a central role during language understanding and other extended cognitive processing, providing an envelope for widespread integration of participating cortical areas. We used linear microelectrode arrays in epileptics to define the circuits generating theta in inferotemporal, perirhinal, entorhinal, prefrontal and anterior cingulate cortices. In all locations, theta was generated by excitatory current sinks in middle layers which receive predominantly feedforward inputs, alternating with sinks in superficial layers which receive mainly feedback/associative inputs. Baseline and event-related theta were generated by indistinguishable laminar profiles of transmembrane currents and unit-firing. Word presentation could reset theta phase, permitting theta to contribute to late event-related potentials, even when theta power decreases relative to baseline. Limited recordings during sentence reading are consistent with rhythmic theta activity entrained by a given word modulating the neural background for the following word. These findings show that theta occurs spontaneously, and can be momentarily suppressed, reset and synchronized by words. Theta represents an alternation between feedforward/divergent and associative/convergent processing modes that may temporally organize sustained processing and optimize the timing of memory formation. We suggest that words are initially encoded via a ventral feedforward stream which is lexicosemantic in the anteroventral temporal lobe; its arrival may trigger a widespread theta rhythm which integrates the word within a larger context.

Dynamic encoding of speech sequence probability in human temporal cortex

Sensory processing involves identification of stimulus features, but also integration with the surrounding sensory and cognitive context. Previous work in animals and humans has shown fine-scale sensitivity to context in the form of learned knowledge about the statistics of the sensory environment, including relative probabilities of discrete units in a stream of sequential auditory input. These statistics are a defining characteristic of one of the most important sequential signals humans encounter: speech. For speech, extensive exposure to a language tunes listeners to the statistics of sound sequences. To address how speech sequence statistics are neurally encoded, we used high-resolution direct cortical recordings from human lateral superior temporal cortex as subjects listened to words and nonwords with varying transition probabilities between sound segments. In addition to their sensitivity to acoustic features (including contextual features, such as coarticulation), we found that neural responses dynamically encoded the language-level probability of both preceding and upcoming speech sounds. Transition probability first negatively modulated neural responses, followed by positive modulation of neural responses, consistent with coordinated predictive and retrospective recognition processes, respectively. Furthermore, transition probability encoding was different for real English words compared with nonwords, providing evidence for online interactions with high-order linguistic knowledge. These results demonstrate that sensory processing of deeply learned stimuli involves integrating physical stimulus features with their contextual sequential structure. Despite not being consciously aware of phoneme sequence statistics, listeners use this information to process spoken input and to link low-level acoustic representations with linguistic information about word identity and meaning.

EEG decoding of spoken words in bilingual listeners: from words to language invariant semantic-conceptual representations

Spoken word recognition and production require fast transformations between acoustic, phonological, and conceptual neural representations. Bilinguals perform these transformations in native and non-native languages, deriving unified semantic concepts from equivalent, but acoustically different words. Here we exploit this capacity of bilinguals to investigate input invariant semantic representations in the brain. We acquired EEG data while Dutch subjects, highly proficient in English listened to four monosyllabic and acoustically distinct animal words in both languages (e.g., “paard”–“horse”). Multivariate pattern analysis (MVPA) was applied to identify EEG response patterns that discriminate between individual words within one language (within-language discrimination) and generalize meaning across two languages (across-language generalization). Furthermore, employing two EEG feature selection approaches, we assessed the contribution of temporal and oscillatory EEG features to our classification results. MVPA revealed that within-language discrimination was possible in a broad time-window (~50–620 ms) after word onset probably reflecting acoustic-phonetic and semantic-conceptual differences between the words. Most interestingly, significant across-language generalization was possible around 550–600 ms, suggesting the activation of common semantic-conceptual representations from the Dutch and English nouns. Both types of classification, showed a strong contribution of oscillations below 12 Hz, indicating the importance of low frequency oscillations in the neural representation of individual words and concepts. This study demonstrates the feasibility of MVPA to decode individual spoken words from EEG responses and to assess the spectro-temporal dynamics of their language invariant semantic-conceptual representations. We discuss how this method and results could be relevant to track the neural mechanisms underlying conceptual encoding in comprehension and production.

Timing the impact of literacy on visual processing

Learning to read requires the acquisition of an efficient visual procedure for quickly recognizing fine print. Thus, reading practice could induce a perceptual learning effect in early vision. Using functional magnetic resonance imaging (fMRI) in literate and illiterate adults, we previously demonstrated an impact of reading acquisition on both high- and low-level occipitotemporal visual areas, but could not resolve the time course of these effects. To clarify whether literacy affects early vs. late stages of visual processing, we measured event-related potentials to various categories of visual stimuli in healthy adults with variable levels of literacy, including completely illiterate subjects, early-schooled literate subjects, and subjects who learned to read in adulthood (ex-illiterates). The stimuli included written letter strings forming pseudowords, on which literacy is expected to have a major impact, as well as faces, houses, tools, checkerboards, and false fonts. To evaluate the precision with which these stimuli were encoded, we studied repetition effects by presenting the stimuli in pairs composed of repeated, mirrored, or unrelated pictures from the same category. The results indicate that reading ability is correlated with a broad enhancement of early visual processing, including increased repetition suppression, suggesting better exemplar discrimination, and increased mirror discrimination, as early as ∼ 100-150 ms in the left occipitotemporal region. These effects were found with letter strings and false fonts, but also were partially generalized to other visual categories. Thus, learning to read affects the magnitude, precision, and invariance of early visual processing.

Neural language processing in adolescent first-language learners

The relation between the timing of language input and development of neural organization for language processing in adulthood has been difficult to tease apart because language is ubiquitous in the environment of nearly all infants. However, within the congenitally deaf population are individuals who do not experience language until after early childhood. Here, we investigated the neural underpinnings of American Sign Language (ASL) in 2 adolescents who had no sustained language input until they were approximately 14 years old. Using anatomically constrained magnetoencephalography, we found that recently learned signed words mainly activated right superior parietal, anterior occipital, and dorsolateral prefrontal areas in these 2 individuals. This spatiotemporal activity pattern was significantly different from the left fronto-temporal pattern observed in young deaf adults who acquired ASL from birth, and from that of hearing young adults learning ASL as a second language for a similar length of time as the cases. These results provide direct evidence that the timing of language experience over human development affects the organization of neural language processing.

Differential activation of human core, non-core and auditory-related cortex during speech categorization tasks as revealed by intracranial recordings

Speech perception requires that sounds be transformed into speech-related objects with lexical and semantic meaning. It is unclear at what level in the auditory pathways this transformation emerges. Primary auditory cortex has been implicated in both representation of acoustic sound attributes and sound objects. While non-primary auditory cortex located on the posterolateral superior temporal gyrus (PLST) is clearly involved in acoustic-to-phonetic pre-lexical representations, it is unclear what role this region plays in auditory object formation. Additional data support the importance of prefrontal cortex in the formation of auditory objects, while other data would implicate this region in auditory object selection. To help clarify the respective roles of auditory and auditory-related cortex in the formation and selection of auditory objects, we examined high gamma activity simultaneously recorded directly from Heschl's gyrus (HG), PLST and prefrontal cortex, while subjects performed auditory semantic detection tasks. Subjects were patients undergoing evaluation for treatment of medically intractable epilepsy. We found that activity in posteromedial HG and early activity on PLST was robust to sound stimuli regardless of their context, and minimally modulated by tasks. Later activity on PLST could be strongly modulated by semantic context, but not by behavioral performance. Activity within prefrontal cortex also was related to semantic context, and did co-vary with behavior. We propose that activity in posteromedial HG and early activity on PLST primarily reflect the representation of spectrotemporal sound attributes. Later activity on PLST represents a pre-lexical processing stage and is an intermediate step in the formation of word objects. Activity in prefrontal cortex appears directly involved in word object selection. The roles of other auditory and auditory-related cortical areas in the formation of word objects remain to be explored.

Surface-based mixed effects multilevel analysis of grouped human electrocorticography

Electrocorticography (ECoG) in humans yields data with unmatched spatio-temporal resolution that provides novel insights into cognitive operations. However, the broader application of ECoG has been confounded by difficulties in accurately depicting individual data and performing statistically valid population-level analyses. To overcome these limitations, we developed methods for accurately registering ECoG data to individual cortical topology. We integrated this technique with surface-based co-registration and a mixed-effects multilevel analysis (MEMA) to control for variable cortical surface anatomy and sparse coverage across patients, as well as intra- and inter-subject variability. We applied this surface-based MEMA (SB-MEMA) technique to a face-recognition task dataset (n=22). Compared against existing techniques, SB-MEMA yielded results much more consistent with individual data and with meta-analyses of face-specific activation studies. We anticipate that SB-MEMA will greatly expand the role of ECoG in studies of human cognition, and will enable the generation of population-level brain activity maps and accurate multimodal comparisons.

Dynamic speech representations in the human temporal lobe

Speech perception requires rapid integration of acoustic input with context-dependent knowledge. Recent methodological advances have allowed researchers to identify underlying information representations in primary and secondary auditory cortex and to examine how context modulates these representations. We review recent studies that focus on contextual modulations of neural activity in the superior temporal gyrus (STG), a major hub for spectrotemporal encoding. Recent findings suggest a highly interactive flow of information processing through the auditory ventral stream, including influences of higher-level linguistic and metalinguistic knowledge, even within individual areas. Such mechanisms may give rise to more abstract representations, such as those for words. We discuss the importance of characterizing representations of context-dependent and dynamic patterns of neural activity in the approach to speech perception research.

Neural stages of spoken, written, and signed word processing in beginning second language learners

We combined magnetoencephalography (MEG) and magnetic resonance imaging (MRI) to examine how sensory modality, language type, and language proficiency interact during two fundamental stages of word processing: (1) an early word encoding stage, and (2) a later supramodal lexico-semantic stage. Adult native English speakers who were learning American Sign Language (ASL) performed a semantic task for spoken and written English words, and ASL signs. During the early time window, written words evoked responses in left ventral occipitotemporal cortex, and spoken words in left superior temporal cortex. Signed words evoked activity in right intraparietal sulcus that was marginally greater than for written words. During the later time window, all three types of words showed significant activity in the classical left fronto-temporal language network, the first demonstration of such activity in individuals with so little second language (L2) instruction in sign. In addition, a dissociation between semantic congruity effects and overall MEG response magnitude for ASL responses suggested shallower and more effortful processing, presumably reflecting novice L2 learning. Consistent with previous research on non-dominant language processing in spoken languages, the L2 ASL learners also showed recruitment of right hemisphere and lateral occipital cortex. These results demonstrate that late lexico-semantic processing utilizes a common substrate, independent of modality, and that proficiency effects in sign language are comparable to those in spoken language.