A Hierarchical Generative Framework of Language Processing: Linking Language Perception, Interpretation, and Production Abnormalities in Schizophrenia

Spared bottom-up but impaired top-down interactive effects during naturalistic language processing in schizophrenia: evidence from the visual-world paradigm

BACKGROUND

People with schizophrenia process language in unusual ways, but the causes of these abnormalities are unclear. In particular, it has proven difficult to empirically disentangle explanations based on impairments in the top-down processing of higher level information from those based on the bottom-up processing of lower level information.

METHODS

To distinguish these accounts, we used visual-world eye tracking, a paradigm that measures spoken language processing during real-world interactions. Participants listened to and then acted out syntactically ambiguous spoken instructions (e.g. 'tickle the frog with the feather', which could either specify how to tickle a frog, or which frog to tickle). We contrasted how 24 people with schizophrenia and 24 demographically matched controls used two types of lower level information (prosody and lexical representations) and two types of higher level information (pragmatic and discourse-level representations) to resolve the ambiguous meanings of these instructions. Eye tracking allowed us to assess how participants arrived at their interpretation in real time, while recordings of participants' actions measured how they ultimately interpreted the instructions.

RESULTS

We found a striking dissociation in participants' eye movements: the two groups were similarly adept at using lower level information to immediately constrain their interpretations of the instructions, but only controls showed evidence of fast top-down use of higher level information. People with schizophrenia, nonetheless, did eventually reach the same interpretations as controls.

CONCLUSIONS

These data suggest that language abnormalities in schizophrenia partially result from a failure to use higher level information in a top-down fashion, to constrain the interpretation of language as it unfolds in real time.

Normal categorical perception to syllable-like stimuli in long term and in first episode schizophrenia

Schizophrenia is associated with deficits in language processing that are evident even at first-episode. However, there is debate as to how early in the processing stream the linguistic deficits appear. We measured categorical processing of artificial syllables that varied in voice-onset time (VOT), and how sensory biasing impacts categorical perception. VOT varied in 5 ms increments from 0 ms (strong /ba/) to 40 ms (strong /pa/). Participants chose whether a syllable sounded more like /ba/ or /pa/. Twenty-two individuals with long-term schizophrenia (Sz) were compared to 21 controls (HCSz), and 17 individuals at their first-episode of schizophrenia (FE) were compared to 19 controls (HCFE). There were three conditions: equiprobable - each syllable had an equal probability of being presented; /ba/-biased - 0 ms VOT (strong /ba/) presented 70% of the time; /pa/-biased - 40 ms VOT (strong /pa/) presented 70% of the time. All groups showed categorical perception and category shifts during biased conditions. Sz and FE were statistically indistinguishable from controls in the point of categorical shift, slope of their response function, and the VOT needed to reliably perceive /pa/. Together, this suggests intact ability to map acoustic stimuli to phonetic categories when based on timing differences in voiced information, both early and late in the disease.

NMDA Receptor Antagonist Effects on Speech-Related Mismatch Negativity and Its Underlying Oscillatory and Source Activity in Healthy Humans

Background: Previous studies in schizophrenia have consistently shown that deficits in the generation of the auditory mismatch negativity (MMN) – a pre-attentive, event-related potential (ERP) typically elicited by changes to simple sound features – are linked to N-methyl-D-aspartate (NMDA) receptor hypofunction. Concomitant with extensive language dysfunction in schizophrenia, patients also exhibit MMN deficits to changes in speech but their relationship to NMDA-mediated neurotransmission is not clear. Accordingly, our study aimed to investigate speech MMNs in healthy humans and their underlying electrophysiological mechanisms in response to NMDA antagonist treatment. We also evaluated the relationship between baseline MMN/electrocortical activity and emergent schizophrenia-like symptoms associated with NMDA receptor blockade.

Methods: In a sample of 18 healthy volunteers, a multi-feature Finnish language paradigm incorporating changes in syllables, vowels and consonant stimuli was used to assess the acute effects of the NMDA receptor antagonist ketamine and placebo on the MMN. Further, measures of underlying neural activity, including evoked theta power, theta phase locking and source-localized current density in cortical regions of interest were assessed. Subjective symptoms were assessed with the Clinician Administered Dissociative States Scale (CADSS).

Results: Participants exhibited significant ketamine-induced increases in psychosis-like symptoms and depending on temporal or frontal recording region, co-occurred with reductions in MMN generation in response to syllable frequency/intensity, vowel duration, across vowel and consonant deviants. MMN attenuation was associated with decreases in evoked theta power, theta phase locking and diminished current density in auditory and inferior frontal (language-related cortical) regions. Baseline (placebo) MMN and underlying electrophysiological features associated with the processing of changes in syllable intensity correlated with the degree of psychotomimetic response to ketamine.

Conclusion: Ketamine-induced impairments in healthy human speech MMNs and their underlying electrocortical mechanisms closely resemble those observed in schizophrenia and support a model of dysfunctional NMDA receptor-mediated neurotransmission of language processing deficits in schizophrenia.

Combining CDP-choline and galantamine, an optimized α7 nicotinic strategy, to ameliorate sensory gating to speech stimuli in schizophrenia

Neural α7 nicotinic acetylcholine receptor (nAChR) expression and functioning deficits have been extensively associated with cognitive and early sensory gating (SG) impairments in schizophrenia (SCZ) patients and their relatives. SG, the suppression of irrelevant and redundant stimuli, is measured in a conditioning-testing (S₁-S₂) paradigm eliciting electroencephalography-derived P50 event-related potentials (ERPs), the S₂ amplitudes of which are typically suppressed relative to S₁. Despite extensive reports of nicotine-related improvements and several decades of research, an efficient nicotinic treatment has yet to be approved for SCZ. Following reports of SG improvements in low P50 suppressing SCZ patients and healthy participants with the α7 agonist, CDP-choline, this pilot study examined the combined modulatory effect of CDP-choline (500 mg) and galantamine (16 mg), a nAChR positive allosteric modulator and acetylcholinesterase inhibitor, on SG to speech stimuli in twenty-four SCZ patients in a randomized, double-blind and placebo-controlled design. As expected, in low P50 suppressors CDP-choline/galantamine (vs. Placebo) improved rP50 and dP50 scores by increasing inhibitory mechanisms as reflected by S₂P50 amplitude reductions. Results also suggest a moderating role for auditory verbal hallucinations in treatment response. These preliminary findings provide supportive evidence for the involvement of α7 nAChR activity in speech gating in SCZ and support additional trials, examining different dose combinations and repeated doses of this optimized and personalized targeted α7 cholinergic treatment for SG dysfunction in subgroups of SCZ patients.

Multimodal neuroimaging evidence for looser lexico-semantic networks in schizophrenia:Evidence from masked indirect semantic priming

It has been hypothesized that schizophrenia is characterized by overly broad automatic activity within lexico-semantic networks. We used two complementary neuroimaging techniques, Magnetoencephalography (MEG) and functional Magnetic Resonance Imaging (fMRI), in combination with a highly automatic indirect semantic priming paradigm, to spatiotemporally localize this abnormality in the brain. Eighteen people with schizophrenia and 20 demographically-matched control participants viewed target words ("bell") preceded by directly related ("church"), indirectly related ("priest"), or unrelated ("hammer") prime words in MEG and fMRI sessions. To minimize top-down processing, the prime was masked, the target appeared only 140 ms after prime onset, and participants simply monitored for words within a particular semantic category that appeared in filler trials. Both techniques revealed a significantly larger automatic indirect priming effect in people with schizophrenia than in control participants. MEG temporally localized this enhanced effect to the N400 time window (300-500 ms) - the critical stage of accessing meaning from words. fMRI spatially localized the effect to the left temporal fusiform cortex, which plays a role in mapping of orthographic word-form on to meaning. There was no evidence of an enhanced automatic direct semantic priming effect in the schizophrenia group. These findings provide converging neural evidence for abnormally broad highly automatic lexico-semantic activity in schizophrenia. We argue that, rather than arising from an unconstrained spread of automatic activation across semantic memory, this broader automatic lexico-semantic activity stems from looser mappings between the form and meaning of words.

The language profile of formal thought disorder

Formal thought disorder (FTD) is clinically manifested as disorganized speech, but there have been only few investigations of its linguistic properties. We examined how disturbance of thought may relate to the referential function of language as expressed in the use of noun phrases (NPs) and the complexity of sentence structures. We used a comic strip description task to elicit language samples from 30 participants with schizophrenia (SZ), 15 with moderate or severe FTD (SZ + FTD), and 15 minimal or no FTD (SZ−FTD), as well as 15 first-degree relatives of people with SZ (FDRs) and 15 neurotypical controls (NC). We predicted that anomalies in the normal referential use of NPs, sub-divided into definite and indefinite NPs, would identify FTD; and also that FTD would also be linked to reduced linguistic complexity as specifically measured by the number of embedded clauses and of grammatical dependents. Participants with SZ + FTD produced more referential anomalies than NC and produced the fewest definite NPs, while FDRs produced the most and thus also differed from NC. When referential anomalies were classed according to the NP type in which they occurred, the SZ + FTD group produced more anomalies in definite NPs than NC. Syntactic errors did not distinguish groups, but the SZ + FTD group exhibited significantly less syntactic complexity than non-SZ groups. Exploratory regression analyses suggested that production of definite NPs distinguished the two SZ groups. These results demonstrate that FTD can be identified in specific grammatical patterns which provide new targets for detection, intervention, and neurobiological studies.

Deficits in nominal reference identify thought disordered speech in a narrative production task

Formal thought disorder (TD) is a neuropathology manifest in formal language dysfunction, but few behavioural linguistic studies exist. These have highlighted problems in the domain of semantics and more specifically of reference. Here we aimed for a more complete and systematic linguistic model of TD, focused on (i) a more in-depth analysis of anomalies of reference as depending on the grammatical construction type in which they occur, and (ii) measures of formal grammatical complexity and errors. Narrative speech obtained from 40 patients with schizophrenia, 20 with TD and 20 without, and from 14 healthy controls matched on pre-morbid IQ, was rated blindly. Results showed that of 10 linguistic variables annotated, 4 showed significant differences between groups, including the two patient groups. These all concerned mis-uses of noun phrases (NPs) for purposes of reference, but showed sensitivity to how NPs were classed: definite and pronominal forms of reference were more affected than indefinite and non-pronominal (lexical) NPs. None of the measures of formal grammatical complexity and errors distinguished groups. We conclude that TD exhibits a specific and differentiated linguistic profile, which can illuminate TD neuro-cognitively and inform future neuroimaging studies, and can have clinical utility as a linguistic biomarker.

The underlying mechanism of deficits of speech comprehension and hallucinations in Chinese patients with schizophrenia

Sentence context and fundamental frequency (F0) contours are important factors to speech perception and comprehension. In Chinese-Mandarin, lexical tones can be distinguished by the F0 contours. Previous studies found healthy people could use the cue of context to recover the phonological representations of lexical tones from the altered tonal patterns to comprehend the sentences in quiet condition, but can not in noise environment. Lots of research showed that patients with schizophrenia have deficits of speech perception and comprehension. However, it is unclear how context and F0 contours influence speech perception and comprehension in patients with schizophrenia. This study detected the contribution of context and lexical tone to sentence comprehension in four types of sentences by manipulating the context and F0 contours in 32 patients with schizophrenia and 33 healthy controls. The results showed that (1) in patients with schizophrenia, the interaction between context and F0 contour was not significant, which was significant in healthy controls; (2) the scores of sentences with two types of sentences with flattened F0 contours were negatively correlated with hallucination trait scores; (3) the patients with schizophrenia showed significantly lower scores on the intelligibility of sentences in all conditions, which were negatively correlated with PANSS-P. The patients with schizophrenia couldn't use the cue of context to recover the phonological representations of lexical tones from the altered tonal patterns when they comprehend the sentences, inner noise may be the underlying mechanism for the deficits of speech perception and comprehension.

When Proactivity Fails: An Electrophysiological Study of Establishing Reference in Schizophrenia

BACKGROUND

Schizophrenia is characterized by abnormalities in referential communication, which may be linked to more general deficits in proactive cognitive control. We used event-related potentials to probe the timing and nature of the neural mechanisms engaged as people with schizophrenia linked pronouns to their preceding referents during word-by-word sentence comprehension.

METHODS

We measured event-related potentials to pronouns in two-clause sentences in 16 people with schizophrenia and 20 demographically matched control participants. Our design crossed the number of potential referents (1-referent, 2-referent) with whether the pronoun matched the gender of its preceding referent(s) (matching, mismatching). This gave rise to four conditions: 1) 1-referent matching ("Edward took courses in accounting but he . . ."); 2) 2-referent matching ("Edward and Phillip took courses but he . . . "); 3) 1-referent mismatching ("Edward took courses in accounting but she . . ."); and 4) 2-referent mismatching ("Edward and Phillip took courses but she . . .").

RESULTS

Consistent with previous findings, healthy control participants produced a larger left anteriorly distributed negativity between 400 and 600 ms to 2-referent matching than to 1-referent matching pronouns (the "Nref effect"). In contrast, people with schizophrenia produced a larger centroposterior positivity effect between 600 and 800 ms. Both patient and control groups produced a larger positivity between 400 and 800 ms to mismatching than to matching pronouns.

CONCLUSIONS

These findings suggest that proactive mechanisms of referential processing, reflected by the Nref effect, are impaired in schizophrenia, while reactive mechanisms, reflected by the positivity effects, are relatively spared. Indeed, patients may compensate for proactive deficits by retroactively engaging with context to influence the processing of inputs at a later stage of analysis.

Abnormal neural hierarchy in processing of verbal information in patients with schizophrenia

Previous research indicates abnormal comprehension of verbal information in patients with schizophrenia. Yet the neural mechanism underlying the breakdown of verbal information processing in schizophrenia is poorly understood. Imaging studies in healthy populations have shown a network of brain areas involved in hierarchical processing of verbal information over time. Here, we identified critical aspects of this hierarchy, examining patients with schizophrenia. Using functional magnetic resonance imaging, we examined various levels of information comprehension elicited by naturally presented verbal stimuli; from a set of randomly shuffled words to an intact story. Specifically, patients with first episode schizophrenia (N = 15), their non-manifesting siblings (N = 14) and healthy controls (N = 15) listened to a narrated story and randomly scrambled versions of it. To quantify the degree of dissimilarity between the groups, we adopted an inter-subject correlation (inter-SC) approach, which estimates differences in synchronization of neural responses within and between groups. The temporal topography found in healthy and siblings groups were consistent with our previous findings - high synchronization in responses from early sensory toward high order perceptual and cognitive areas. In patients with schizophrenia, stimuli with short and intermediate temporal scales evoked a typical pattern of reliable responses, whereas story condition (long temporal scale) revealed robust and widespread disruption of the inter-SCs. In addition, the more similar the neural activity of patients with schizophrenia was to the average response in the healthy group, the less severe the positive symptoms of the patients. Our findings suggest that system-level neural indication of abnormal verbal information processing in schizophrenia reflects disease manifestations.

Priming production: Neural evidence for enhanced automatic semantic activity preceding language production in schizophrenia

Introduction

Lexico-semantic disturbances are considered central to schizophrenia. Clinically, their clearest manifestation is in language production. However, most studies probing their underlying mechanisms have used comprehension or categorization tasks. Here, we probed automatic semantic activity prior to language production in schizophrenia using event-related potentials (ERPs).

Methods

19 people with schizophrenia and 16 demographically-matched healthy controls named target pictures that were very quickly preceded by masked prime words. To probe automatic semantic activity prior to production, we measured the N400 ERP component evoked by these targets. To determine the origin of any automatic semantic abnormalities, we manipulated the type of relationship between prime and target such that they overlapped in (a) their semantic features (semantically related, e.g. "cake" preceding a < picture of a pie >, (b) their initial phonemes (phonemically related, e.g. "stomach" preceding a < picture of a starfish >), or (c) both their semantic features and their orthographic/phonological word form (identity related, e.g. "socks" preceding a < picture of socks >). For each of these three types of relationship, the same targets were paired with unrelated prime words (counterbalanced across lists). We contrasted ERPs and naming times to each type of related target with its corresponding unrelated target.

Results

People with schizophrenia showed abnormal N400 modulation prior to naming identity related (versus unrelated) targets: whereas healthy control participants produced a smaller amplitude N400 to identity related than unrelated targets, patients showed the opposite pattern, producing a larger N400 to identity related than unrelated targets. This abnormality was specific to the identity related targets. Just like healthy control participants, people with schizophrenia produced a smaller N400 to semantically related than to unrelated targets, and showed no difference in the N400 evoked by phonemically related and unrelated targets. There were no differences between the two groups in the pattern of naming times across conditions.

Conclusion

People with schizophrenia can show abnormal neural activity associated with automatic semantic processing prior to language production. The specificity of this abnormality to the identity related targets suggests that that, rather than arising from abnormalities of either semantic features or lexical form alone, it may stem from disruptions of mappings (connections) between the meaning of words and their form.

Action (verb) fluency deficits in schizophrenia spectrum disorders: linking language, cognition and interpersonal functioning

Deficits in action (verb) fluency have previously been reported in schizophrenia spectrum disorders. The degree to which this reflects difficulties generating verbs in different semantic categories is unknown. Here, action fluency responses of 46 patients with schizophrenia spectrum disorders and 76 healthy controls were classified as action or mental state verbs, using well-established taxonomies. The word length, frequency, age of acquisition, valence and concreteness of the verbs produced were also examined. Participants also completed measures of cognitive function, and clinical symptoms. Independent inter-rater agreement of semantic categorization was high. The percentage of action verbs produced was significantly lower in patients than controls, whilst the percentage of mental state verbs produced did not differ. Patients' action verbs were: significantly less concrete; positively correlated with memory and intelligence; and negatively correlated with interpersonal symptoms. Impaired action verb, but intact mental state verb generation is consistent with the neural separability of these processes.

Implications of Information Theory for Computational Modeling of Schizophrenia

Information theory provides a formal framework within which information processing and its disorders can be described. However, information theory has rarely been applied to modeling aspects of the cognitive neuroscience of schizophrenia. The goal of this article is to highlight the benefits of an approach based on information theory, including its recent extensions, for understanding several disrupted neural goal functions as well as related cognitive and symptomatic phenomena in schizophrenia. We begin by demonstrating that foundational concepts from information theory—such as Shannon information, entropy, data compression, block coding, and strategies to increase the signal-to-noise ratio—can be used to provide novel understandings of cognitive impairments in schizophrenia and metrics to evaluate their integrity. We then describe more recent developments in information theory, including the concepts of infomax, coherent infomax, and coding with synergy, to demonstrate how these can be used to develop computational models of schizophrenia-related failures in the tuning of sensory neurons, gain control, perceptual organization, thought organization, selective attention, context processing, predictive coding, and cognitive control. Throughout, we demonstrate how disordered mechanisms may explain both perceptual/cognitive changes and symptom emergence in schizophrenia. Finally, we demonstrate that there is consistency between some information-theoretic concepts and recent discoveries in neurobiology, especially involving the existence of distinct sites for the accumulation of driving input and contextual information prior to their interaction. This convergence can be used to guide future theory, experiment, and treatment development.

Mapping the Speech Code: Cortical Responses Linking the Perception and Production of Vowels

The acoustic realization of speech is constrained by the physical mechanisms by which it is produced. Yet for speech perception, the degree to which listeners utilize experience derived from speech production has long been debated. In the present study, we examined how sensorimotor adaptation during production may affect perception, and how this relationship may be reflected in early vs. late electrophysiological responses. Participants first performed a baseline speech production task, followed by a vowel categorization task during which EEG responses were recorded. In a subsequent speech production task, half the participants received shifted auditory feedback, leading most to alter their articulations. This was followed by a second, post-training vowel categorization task. We compared changes in vowel production to both behavioral and electrophysiological changes in vowel perception. No differences in phonetic categorization were observed between groups receiving altered or unaltered feedback. However, exploratory analyses revealed correlations between vocal motor behavior and phonetic categorization. EEG analyses revealed correlations between vocal motor behavior and cortical responses in both early and late time windows. These results suggest that participants' recent production behavior influenced subsequent vowel perception. We suggest that the change in perception can be best characterized as a mapping of acoustics onto articulation.

Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events

Since the early 2000s, several ERP studies have challenged the assumption that we always use syntactic contextual information to influence semantic processing of incoming words, as reflected by the N400 component. One approach for explaining these findings is to posit distinct semantic and syntactic processing mechanisms, each with distinct time courses. While this approach can explain specific datasets, it cannot account for the wider body of findings. I propose an alternative explanation: a dynamic generative framework in which our goal is to infer the underlying event that best explains the set of inputs encountered at any given time. Within this framework, combinations of semantic and syntactic cues with varying reliabilities are used as evidence to weight probabilistic hypotheses about this event. I further argue that the computational principles of this framework can be extended to understand how we infer situation models during discourse comprehension, and intended messages during spoken communication.

What do we mean by prediction in language comprehension?

We consider several key aspects of prediction in language comprehension: its computational nature, the representational level(s) at which we predict, whether we use higher level representations to predictively pre-activate lower level representations, and whether we 'commit' in any way to our predictions, beyond pre-activation. We argue that the bulk of behavioral and neural evidence suggests that we predict probabilistically and at multiple levels and grains of representation. We also argue that we can, in principle, use higher level inferences to predictively pre-activate information at multiple lower representational levels. We also suggest that the degree and level of predictive pre-activation might be a function of the expected utility of prediction, which, in turn, may depend on comprehenders' goals and their estimates of the relative reliability of their prior knowledge and the bottom-up input. Finally, we argue that all these properties of language understanding can be naturally explained and productively explored within a multi-representational hierarchical actively generative architecture whose goal is to infer the message intended by the producer, and in which predictions play a crucial role in explaining the bottom-up input.