Emotional self-other voice processing in schizophrenia and its relationship with hallucinations: ERP evidence

Vocal Emotion Recognition in Autism: Behavioral Performance and Event-Related Potential (ERP) Response

Autistic youth display difficulties in emotion recognition, yet little research has examined behavioral and neural indices of vocal emotion recognition (VER). The current study examines behavioral and event-related potential (N100, P200, Late Positive Potential [LPP]) indices of VER in autistic and non-autistic youth. Participants (N = 164) completed an emotion recognition task, the Diagnostic Analyses of Nonverbal Accuracy (DANVA-2) which included VER, during EEG recording. The LPP amplitude was larger in response to high intensity VER, and social cognition predicted VER errors. Verbal IQ, not autism, was related to VER errors. An interaction between VER intensity and social communication impairments revealed these impairments were related to larger LPP amplitudes during low intensity VER. Taken together, differences in VER may be due to higher order cognitive processes, not basic, early perception (N100, P200), and verbal cognitive abilities may underlie behavioral, yet occlude neural, differences in VER processing.

Blindness influences emotional authenticity perception in voices: Behavioral and ERP evidence

The ability to distinguish spontaneous from volitional emotional expressions is an important social skill. How do blind individuals perceive emotional authenticity? Unlike sighted individuals, they cannot rely on facial and body language cues, relying instead on vocal cues alone. Here, we combined behavioral and ERP measures to investigate authenticity perception in laughter and crying in individuals with early- or late-blindness onset. Early-blind, late-blind, and sighted control participants (n = 17 per group, N = 51) completed authenticity and emotion discrimination tasks while EEG data were recorded. The stimuli consisted of laughs and cries that were either spontaneous or volitional. The ERP analysis focused on the N1, P2, and late positive potential (LPP). Behaviorally, early-blind participants showed intact authenticity perception, but late-blind participants performed worse than controls. There were no group differences in the emotion discrimination task. In brain responses, all groups were sensitive to laughter authenticity at the P2 stage, and to crying authenticity at the early LPP stage. Nevertheless, only early-blind participants were sensitive to crying authenticity at the N1 and middle LPP stages, and to laughter authenticity at the early LPP stage. Furthermore, early-blind and sighted participants were more sensitive than late-blind ones to crying authenticity at the P2 and late LPP stages. Altogether, these findings suggest that early blindness relates to facilitated brain processing of authenticity in voices, both at early sensory and late cognitive-evaluative stages. Late-onset blindness, in contrast, relates to decreased sensitivity to authenticity at behavioral and brain levels.

From communication dysfunction to treatment options in serious mental illness

The Commentary covers research focused on language dysfunction in schizophrenia, and more broadly in communication dysfunction in this disorder, which I have examined with a variety of both behavioral and imaging methodologies. It briefly outlines how further progress can be achieved in pursuing the goal of a comprehensive understanding of its underlying causes. Possible therapeutic approaches are also briefly discussed.

Bone conduction facilitates self-other voice discrimination

One's own voice is one of the most important and most frequently heard voices. Although it is the sound we associate most with ourselves, it is perceived as strange when played back in a recording. One of the main reasons is the lack of bone conduction that is inevitably present when hearing one's own voice while speaking. The resulting discrepancy between experimental and natural self-voice stimuli has significantly impeded self-voice research, rendering it one of the least investigated aspects of self-consciousness. Accordingly, factors that contribute to self-voice perception remain largely unknown. In a series of three studies, we rectified this ecological discrepancy by augmenting experimental self-voice stimuli with bone-conducted vibrotactile stimulation that is present during natural self-voice perception. Combining voice morphing with psychophysics, we demonstrate that specifically self-other but not familiar-other voice discrimination improved for stimuli presented using bone as compared with air conduction. Furthermore, our data outline independent contributions of familiarity and acoustic processing to separating the own from another's voice: although vocal differences increased general voice discrimination, self-voices were more confused with familiar than unfamiliar voices, regardless of their acoustic similarity. Collectively, our findings show that concomitant vibrotactile stimulation improves auditory self-identification, thereby portraying self-voice as a fundamentally multi-modal construct.

Visual processing deficits in patients with schizophrenia spectrum and bipolar disorders and associations with psychotic symptoms, and intellectual abilities

Objective

Low-level sensory disruption is hypothesized as a precursor to clinical and cognitive symptoms in severe mental disorders. We compared visual discrimination performance in patients with schizophrenia spectrum disorder or bipolar disorder with healthy controls, and investigated associations with clinical symptoms and IQ.

Methods

Patients with schizophrenia spectrum disorder (n = 32), bipolar disorder (n = 55) and healthy controls (n = 152) completed a computerized visual discrimination task. Participants responded whether the latter of two consecutive grids had higher or lower spatial frequency, and discrimination thresholds were estimated using an adaptive maximum likelihood procedure. Case-control differences in threshold were assessed using linear regression, F-test and post-hoc pair-wise comparisons. Linear models were used to test for associations between visual discrimination threshold and psychotic symptoms derived from the PANSS and IQ assessed using the Matrix Reasoning and Vocabulary subtests from the Wechsler Abbreviated Scale of Intelligence (WASI).

Results

Robust regression revealed a significant main effect of diagnosis on discrimination threshold (robust F = 6.76, p = .001). Post-hoc comparisons revealed that patients with a schizophrenia spectrum disorder (mean = 14%, SD = 0.08) had higher thresholds compared to healthy controls (mean = 10.8%, SD = 0.07, β = 0.35, t = 3.4, p = .002), as did patients with bipolar disorder (12.23%, SD = 0.07, β = 0.21, t = 2.42, p = .04). There was no significant difference between bipolar disorder and schizophrenia (β = -0.14, t = -1.2, p = .45). Linear models revealed negative associations between IQ and threshold across all participants when controlling for diagnostic group (β = -0.3, t = -3.43, p = .0007). This association was found within healthy controls (t = -3.72, p = .0003) and patients with bipolar disorder (t = -2.53, p = .015), and no significant group by IQ interaction on threshold (F = 0.044, p = .97). There were no significant associations between PANSS domain scores and discrimination threshold.

Conclusion

Patients with schizophrenia spectrum or bipolar disorders exhibited higher visual discrimination thresholds than healthy controls, supporting early visual deficits among patients with severe mental illness. Discrimination threshold was negatively associated with IQ among healthy controls and bipolar disorder patients. These findings elucidate perception-related disease mechanisms in severe mental illness, which warrants replication in independent samples.

Attention and emotion shape self-voice prioritization in speech processing

Both self-voice and emotional speech are salient signals that are prioritized in perception. Surprisingly, self-voice perception has been investigated to a lesser extent than the self-face. Therefore, it remains to be clarified whether self-voice prioritization is boosted by emotion, and whether self-relevance and emotion interact differently when attention is focused on who is speaking vs. what is being said. Thirty participants listened to 210 prerecorded words spoken in one's own or an unfamiliar voice and differing in emotional valence in two tasks, manipulating the attention focus on either speaker identity or speech emotion. Event-related potentials (ERP) of the electroencephalogram (EEG) informed on the temporal dynamics of self-relevance, emotion, and attention effects. Words spoken in one's own voice elicited a larger N1 and Late Positive Potential (LPP), but smaller N400. Identity and emotion interactively modulated the P2 (self-positivity bias) and LPP (self-negativity bias). Attention to speaker identity modulated more strongly ERP responses within 600 ms post-word onset (N1, P2, N400), whereas attention to speech emotion altered the late component (LPP). However, attention did not modulate the interaction of self-relevance and emotion. These findings suggest that the self-voice is prioritized for neural processing at early sensory stages, and that both emotion and attention shape self-voice prioritization in speech processing. They also confirm involuntary processing of salient signals (self-relevance and emotion) even in situations in which attention is deliberately directed away from those cues. These findings have important implications for a better understanding of symptoms thought to arise from aberrant self-voice monitoring such as auditory verbal hallucinations.

Hypersensitivity to passive voice hearing in hallucination proneness

Voices are a complex and rich acoustic signal processed in an extensive cortical brain network. Specialized regions within this network support voice perception and production and may be differentially affected in pathological voice processing. For example, the experience of hallucinating voices has been linked to hyperactivity in temporal and extra-temporal voice areas, possibly extending into regions associated with vocalization. Predominant self-monitoring hypotheses ascribe a primary role of voice production regions to auditory verbal hallucinations (AVH). Alternative postulations view a generalized perceptual salience bias as causal to AVH. These theories are not mutually exclusive as both ascribe the emergence and phenomenology of AVH to unbalanced top-down and bottom-up signal processing. The focus of the current study was to investigate the neurocognitive mechanisms underlying predisposition brain states for emergent hallucinations, detached from the effects of inner speech. Using the temporal voice area (TVA) localizer task, we explored putative hypersalient responses to passively presented sounds in relation to hallucination proneness (HP). Furthermore, to avoid confounds commonly found in in clinical samples, we employed the Launay-Slade Hallucination Scale (LSHS) for the quantification of HP levels in healthy people across an experiential continuum spanning the general population. We report increased activation in the right posterior superior temporal gyrus (pSTG) during the perception of voice features that positively correlates with increased HP scores. In line with prior results, we propose that this right-lateralized pSTG activation might indicate early hypersensitivity to acoustic features coding speaker identity that extends beyond own voice production to perception in healthy participants prone to experience AVH.

Attention to voices is increased in non-clinical auditory verbal hallucinations irrespective of salience

Alterations in the processing of vocal emotions have been associated with both clinical and non-clinical auditory verbal hallucinations (AVH), suggesting that changes in the mechanisms underpinning voice perception contribute to AVH. These alterations seem to be more pronounced in psychotic patients with AVH when attention demands increase. However, it remains to be clarified how attention modulates the processing of vocal emotions in individuals without clinical diagnoses who report hearing voices but no related distress. Using an active auditory oddball task, the current study clarified how emotion and attention interact during voice processing as a function of AVH proneness, and examined the contributions of stimulus valence and intensity. Participants with vs. without non-clinical AVH were presented with target vocalizations differing in valence (neutral; positive; negative) and intensity (55 decibels (dB); 75 dB). The P3b amplitude was larger in response to louder (vs. softer) vocal targets irrespective of valence, and in response to negative (vs. neutral) vocal targets irrespective of intensity. Of note, the P3b amplitude was globally increased in response to vocal targets in participants reporting AVH, and failed to be modulated by valence and intensity in these participants. These findings suggest enhanced voluntary attention to changes in vocal expressions but reduced discrimination of salient and non-salient cues. A decreased sensitivity to salience cues of vocalizations could contribute to increased cognitive control demands, setting the stage for an AVH.

External speech processing and auditory verbal hallucinations: A systematic review of functional neuroimaging studies

It has been documented that individuals who hear auditory verbal hallucinations (AVH) exhibit diminished capabilities in processing external speech. While functional neuroimaging studies have attempted to characterise the cortical regions and networks facilitating these deficits in a bid to understand AVH, considerable methodological heterogeneity has prevented a consensus being reached. The current systematic review investigated the neurobiological underpinnings of external speech processing deficits in voice-hearers in 38 studies published between January 1990 to June 2020. AVH-specific deviations in the activity and lateralisation of the temporal auditory regions were apparent when processing speech sounds, words and sentences. During active or affective listening tasks, functional connectivity changes arose within the language, limbic and default mode networks. However, poor study quality and lack of replicable results plague the field. A detailed list of recommendations has been provided to improve the quality of future research on this topic.

Multimodal assessment of communicative-pragmatic features in schizophrenia: a machine learning approach

An impairment in pragmatic communication is a core feature of schizophrenia, often associated with difficulties in social interactions. The pragmatic deficits regard various pragmatic phenomena, e.g., direct and indirect communicative acts, deceit, irony, and include not only the use of language but also other expressive means such as non-verbal/extralinguistic modalities, e.g., gestures and body movements, and paralinguistic cues, e.g., prosody and tone of voice. The present paper focuses on the identification of those pragmatic features, i.e., communicative phenomena and expressive modalities, that more reliably discriminate between individuals with schizophrenia and healthy controls. We performed a multimodal assessment of communicative-pragmatic ability, and applied a machine learning approach, specifically a Decision Tree model, with the aim of identifying the pragmatic features that best separate the data into the two groups, i.e., individuals with schizophrenia and healthy controls, and represent their configuration. The results indicated good overall performance of the Decision Tree model, with mean Accuracy of 82%, Sensitivity of 76%, and Precision of 91%. Linguistic irony emerged as the most relevant pragmatic phenomenon in distinguishing between the two groups, followed by violation of the Gricean maxims, and then extralinguistic deceitful and sincere communicative acts. The results are discussed in light of the pragmatic theoretical literature, and their clinical relevance in terms of content and design of both assessment and rehabilitative training.

Abnormal ERPs and Brain Dynamics Mediate Basic Self Disturbance in Schizophrenia: A Review of EEG and MEG Studies

Background: Interest in disordered sense of self in schizophrenia has recently re-emerged in the literature. It has been proposed that there is a basic self disturbance, underlying the diagnostic symptoms of schizophrenia, in which the person's sense of being a bounded individual continuous through time loses stability. This disturbance has been documented phenomenologically and at the level of cognitive tasks. However, the neural correlates of basic self disorder in schizophrenia are poorly understood. Methods: A search of PubMed was used to identify studies on self and schizophrenia that reported EEG or MEG data. Results: Thirty-three studies were identified, 32 using EEG and one using MEG. Their operationalizations of the self were divided into six paradigms: self-monitoring for errors, proprioception, self-other integration, self-referential processing, aberrant salience, and source monitoring. Participants with schizophrenia were less accurate on self-referential processing tasks and had slower response times across most studies. Event-related potential amplitudes differed across many early and late components, with reduced N100 suppression in source monitoring paradigms being the most replicated finding. Several studies found differences in one or more frequency band, but no coherent overall finding emerged in this area. Various other measures of brain dynamics also showed differences in single studies. Only some of the study designs were adequate to establish a causal relationship between the self and EEG or MEG measures. Conclusion: The broad range of changes suggests a global self disturbance at the neuronal level, possibly carried over from the resting state. Further studies that successfully isolate self-related effects are warranted to better understand the temporal-dynamic and spatial-topographic basis of self disorder and its relationship to basic self disturbance on the phenomenological level.

The Self and Its Prolonged Intrinsic Neural Timescale in Schizophrenia

Schizophrenia (SCZ) can be characterized as a basic self-disorder that is featured by abnormal temporal integration on phenomenological (experience) and psychological (information processing) levels. Temporal integration on the neuronal level can be measured by the brain's intrinsic neural timescale using the autocorrelation window (ACW) and power-law exponent (PLE). Our goal was to relate intrinsic neural timescales (ACW, PLE), as a proxy of temporal integration on the neuronal level, to temporal integration related to self-disorder on psychological (Enfacement illusion task in electroencephalography) and phenomenological (Examination of Anomalous Self-Experience [EASE]) levels. SCZ participants exhibited prolonged ACW and higher PLE during the self-referential task (Enfacement illusion), but not during the non-self-referential task (auditory oddball). The degree of ACW/PLE change during task relative to rest was significantly reduced in self-referential task in SCZ. A moderation model showed that low and high ACW/PLE exerted differential impact on the relationship of self-disorder (EASE) and negative symptoms (PANSS). In sum, we demonstrate abnormal prolongation in intrinsic neural timescale during self-reference in SCZ including its relation to basic self-disorder and negative symptoms. Our results point to abnormal relation of self and temporal integration at the core of SCZ constituting a "common currency" of neuronal, psychological, and phenomenological levels.

Pragmatics, Theory of Mind and executive functions in schizophrenia: Disentangling the puzzle using machine learning

OBJECTIVE

Schizophrenia is associated with a severe impairment in the communicative-pragmatic domain. Recent research has tried to disentangle the relationship between communicative impairment and other domains usually impaired in schizophrenia, i.e. Theory of Mind (ToM) and cognitive functions. However, the results are inconclusive and this relationship is still unclear. Machine learning (ML) provides novel opportunities for studying complex relationships among phenomena and representing causality among multiple variables. The present research explored the potential of applying ML, specifically Bayesian network (BNs) analysis, to characterize the relationship between cognitive, ToM and pragmatic abilities in individuals with schizophrenia and healthy controls, and to identify the cognitive and pragmatic abilities that are most informative in discriminating between schizophrenia and controls.

METHODS

We provided a comprehensive assessment of different aspects of pragmatic performance, i.e. linguistic, extralinguistic, paralinguistic, contextual and conversational, ToM and cognitive functions, i.e. Executive Functions (EF)-selective attention, planning, inhibition, cognitive flexibility, working memory and speed processing-and general intelligence, in a sample of 32 individuals with schizophrenia and 35 controls.

RESULTS

The results showed that the BNs classifier discriminated well between patients with schizophrenia and healthy controls. The network structure revealed that only pragmatic Linguistic ability directly influenced the classification of patients and controls, while diagnosis determined performance on ToM, Extralinguistic, Paralinguistic, Selective Attention, Planning, Inhibition and Cognitive Flexibility tasks. The model identified pragmatic, ToM and cognitive abilities as three distinct domains independent of one another.

CONCLUSION

Taken together, our results confirmed the importance of considering pragmatic linguistic impairment as a core dysfunction in schizophrenia, and demonstrated the potential of applying BNs in investigating the relationship between pragmatic ability and cognition.

Voice patterns in schizophrenia: A systematic review and Bayesian meta-analysis

Voice atypicalities have been a characteristic feature of schizophrenia since its first definitions. They are often associated with core negative symptoms such as flat affect and alogia, and with the social impairments seen in the disorder. This suggests that voice atypicalities may represent a marker of clinical features and social functioning in schizophrenia. We systematically reviewed and meta-analyzed the evidence for distinctive acoustic patterns in schizophrenia, as well as their relation to clinical features. We identified 46 articles, including 55 studies with a total of 1254 patients with schizophrenia and 699 healthy controls. Summary effect sizes (Hedges'g and Pearson's r) estimates were calculated using multilevel Bayesian modeling. We identified weak atypicalities in pitch variability (g = -0.55) related to flat affect, and stronger atypicalities in proportion of spoken time, speech rate, and pauses (g's between -0.75 and -1.89) related to alogia and flat affect. However, the effects were mostly modest (with the important exception of pause duration) compared to perceptual and clinical judgments, and characterized by large heterogeneity between studies. Moderator analyses revealed that tasks with a more demanding cognitive and social component showed larger effects both in contrasting patients and controls and in assessing symptomatology. In conclusion, studies of acoustic patterns are a promising but, yet unsystematic avenue for establishing markers of schizophrenia. We outline recommendations towards more cumulative, open, and theory-driven research.

Auditory Predictions and Prediction Errors in Response to Self-Initiated Vowels

It has been suggested that speech production is accomplished by an internal forward model, reducing processing activity directed to self-produced speech in the auditory cortex. The current study uses an established N1-suppression paradigm comparing self- and externally initiated natural speech sounds to answer two questions: (1) Are forward predictions generated to process complex speech sounds, such as vowels, initiated via a button press? (2) Are prediction errors regarding self-initiated deviant vowels reflected in the corresponding ERP components? Results confirm an N1-suppression in response to self-initiated speech sounds. Furthermore, our results suggest that predictions leading to the N1-suppression effect are specific, as self-initiated deviant vowels do not elicit an N1-suppression effect. Rather, self-initiated deviant vowels elicit an enhanced N2b and P3a compared to externally generated deviants, externally generated standard, or self-initiated standards, again confirming prediction specificity. Results show that prediction errors are salient in self-initiated auditory speech sounds, which may lead to more efficient error correction in speech production.

Self-voice perception and its relationship with hallucination predisposition

Introduction: Auditory verbal hallucinations (AVH) are a core symptom of psychotic disorders such as schizophrenia but are also reported in 10-15% of the general population. Impairments in self-voice recognition are frequently reported in schizophrenia and associated with the severity of AVH, particularly when the self-voice has a negative quality. However, whether self-voice processing is also affected in nonclinical voice hearers remains to be specified. Methods: Thirty-five nonclinical participants varying in hallucination predisposition based on the Launay-Slade Hallucination Scale, listened to prerecorded words and vocalisations differing in identity (self/other) and emotional quality. In Experiment 1, participants indicated whether words were spoken in their own voice, another voice, or whether they were unsure (recognition task). They were also asked whether pairs of words/vocalisations were uttered by the same or by a different speaker (discrimination task). In Experiment 2, participants judged the emotional quality of the words/vocalisations. Results: In Experiment 1, hallucination predisposition affected voice discrimination and recognition, irrespective of stimulus valence. Hallucination predisposition did not affect the evaluation of the emotional valence of words/vocalisations (Experiment 2). Conclusions: These findings suggest that nonclinical participants with high HP experience altered voice identity processing, whereas HP does not affect the perception of vocal emotion. Specific alterations in self-voice perception in clinical and nonclinical voice hearers may establish a core feature of the psychosis continuum.

Temporal Features of Psychological and Physical Self-Representation: An ERP Study

Psychological and physical-self are two important aspects of self-concept. Although a growing number of behavioral and neuroimaging studies have investigated the cognitive mechanism and neural substrate underlying psychological and physical-self-representation, most of the existing research on psychological and physical-self-representation had been done in isolation. The present study aims to examine the electrophysiological responses to both psychological (one’s own name) and physical (one’s own voice) self-related stimuli in a uniform paradigm. Event-related potentials (ERPs) were recorded for subjects’ own and others’ names uttered by subjects’ own or others’ voice (own voice-own name, own voice-other’s name, other’s voice-own name, other’s voice-other’s name) while subjects performed an auditory passive oddball task. The results showed that one’s own name elicited smaller P2 and larger P3 amplitudes than those of other’s names, irrespective of the voice identity. However, no differences were observed between self and other’s voice during the P2 and P3 stages. Moreover, an obvious interaction effect was observed between voice content and voice identity at the N400 stage that the subject’s own voice elicited a larger parietal N400 amplitude than other’s voice in other name condition but not in own name condition. Taken together, these findings suggested that psychological (one’s own name) and physical (one’s own voice) self-representation induced distinct electrophysiological response patterns in auditory-cognitive processing.

Voice-selective prediction alterations in nonclinical voice hearers

Auditory verbal hallucinations (AVH) are a cardinal symptom of psychosis but also occur in 6–13% of the general population. Voice perception is thought to engage an internal forward model that generates predictions, preparing the auditory cortex for upcoming sensory feedback. Impaired processing of sensory feedback in vocalization seems to underlie the experience of AVH in psychosis, but whether this is the case in nonclinical voice hearers remains unclear. The current study used electroencephalography (EEG) to investigate whether and how hallucination predisposition (HP) modulates the internal forward model in response to self-initiated tones and self-voices. Participants varying in HP (based on the Launay-Slade Hallucination Scale) listened to self-generated and externally generated tones or self-voices. HP did not affect responses to self vs. externally generated tones. However, HP altered the processing of the self-generated voice: increased HP was associated with increased pre-stimulus alpha power and increased N1 response to the self-generated voice. HP did not affect the P2 response to voices. These findings confirm that both prediction and comparison of predicted and perceived feedback to a self-generated voice are altered in individuals with AVH predisposition. Specific alterations in the processing of self-generated vocalizations may establish a core feature of the psychosis continuum.