"The Sound of Fear": assessing vocal fundamental frequency as a physiological indicator of social anxiety disorder

Associations of Personality, Physical and Mental Health with Voice Range Profiles

OBJECTIVES

There is evidence in the literature that voice characteristics are linked to mental and physical health. The aim of this explorative study was to determine associations between voice parameters measured by a voice range profile (VRP) and personality, mental and physical health.

STUDY DESIGN

Cross-sectional population-based study.

METHODS

As part of the LIFE-Adult-Study, 2639 individuals aged 18-80 years, randomly sampled from the general population, completed both speaking and singing voice tasks and answered questionnaires on depression, anxiety, life satisfaction, personality and quality of life. The voice parameters used were fundamental frequency, sound pressure level, their ranges and maximum phonation time. The associations were examined with the help of correlation and regression analyses.

RESULTS

Wider ranges between the lowest and highest frequency, between the lowest and highest sound pressure level and longer maximum phonation time were significantly correlated with extraversion and quality of life in both sexes, as well as openness and agreeableness in women. Smaller ranges and shorter maximum phonation time were significantly correlated with depression. Neuroticism in men was inversely correlated with the maximum phonation time. In the speaking VRP, the associations for sound pressure level were more pronounced than for the fundamental frequency. The same was true in reverse for the singing VRP. Few associations were found for anxiety, life satisfaction and conscientiousness.

CONCLUSIONS

Weak associations between voice parameters derived from the VRP and mental and physical health, as well as personality were seen in this exploratory study. The results indicate that the VRP measurements in a clinical context are not significantly affected by these parameters and thus are a robust measurement method for voice parameters.

Multidisciplinary characterization of embarrassment through behavioral and acoustic modeling

Embarrassment is a social emotion that shares many characteristics with social anxiety (SA). Most people experience embarrassment in their daily lives, but it is quite overlooked in research. We characterized embarrassment through an interdisciplinary approach, introducing a behavioral paradigm and applying machine learning approaches, including acoustic analyses. 33 participants wrote about an embarrassing experience and then, without knowing it prior, had to read it out loud to the conductor. Embarrassment was then examined using two different approaches: Firstly, from a subjective view, with self-report measures from the participants. Secondly, from an objective, machine-learning approach, in which trained models tested the robustness of our embarrassment data set (i.e., prediction accuracy), and then described embarrassment in a dimensional (i.e., dimension: valence, arousal, dominance; VAD) and categorical (i.e., comparing embarrassment to other emotional states) way. The subjective rating of embarrassment was increased after participants read their stories out loud, and participants with higher SA scores experienced higher embarrassment than participants with lower SA scores. The state of embarrassment was predicted with 86.4% as the best of the unweighted average recall rates. While the simple VAD dimensional analyses did not differentiate between the state of embarrassment and the references, the complex emotional category analyses characterized embarrassment as closer to boredom, a neutral state, and less of sadness. Combining an effective behavioral paradigm and advanced acoustic modeling, we characterized the emotional state of embarrassment, and the identified characteristics could be used as a biomarker to assess SA.

Speech Detection via Respiratory Inductance Plethysmography, Thoracic Impedance, Accelerometers, and Gyroscopes: A Machine Learning-Informed Comparative Study

Speech production interferes with the measurement of changes in cardiac vagal activity during acute stress by attenuating the expected drop in heart rate variability. Speech also induces cardiac sympathetic changes similar to those induced by psychological stress. In the laboratory, confounding of physiological stress reactivity by speech may be controlled experimentally. In ambulatory assessments, however, detection of speech episodes would be necessary to separate the physiological effects of psychosocial stress from those of speech. Using machine learning (https://osf.io/bk9nf), we trained and tested speech classification models on data from 56 participants (ages 18-39) under controlled laboratory conditions. They were equipped with privacy-secure wearables measuring thoracoabdominal respiratory inductance plethysmography (RIP from a single and a dual-band set-up), thoracic impedance pneumography, and an upper sternum positioned unit with triaxial accelerometers and gyroscopes. Following an 80/20 train-test split, nested cross-validations were run with the machine learning algorithms XGBoost, gradient boosting, random forest, and logistic regression on the training set to get generalized performance estimates. Speech classification by the best model per method was then validated in the test set. Speech versus no-speech classification performance (AUC) for both nested cross-validation and test set predictions was excellent for thorax-abdomen RIP (nested cross-validation: 96.6%, test set prediction: 98.5%), thorax-only RIP (97.5%, 99.1%), impedance (97.0%, 97.8%), and accelerometry (99.3%, 99.6%). The sternal accelerometer method outperformed others. These open-access models leveraging biosignals have the potential to also work in daily life settings. This could enhance the trustworthiness of ambulatory psychophysiology, by enabling detection of speech and controlling for its confounding effects on physiology.

Mutual eye gaze and vocal pitch in relation to social anxiety and depression: A virtual interaction task

BACKGROUND

Individuals with high social interaction anxiety (SIA) and depression often behave submissively in social settings. Few studies have simultaneously examined the associations between objectively assessed submissive behaviors and SIA or depression, despite their high comorbidity and unknown mechanisms regarding submissiveness.

METHODS

A sample of 45 young adults self-reported trait SIA and depression, state positive/negative affect (PA/NA) before and after a virtual social interaction. Participants engaged in a four-minute conversation with a confederate who was trained to behave neutrally. Mutual eye gaze, via eye-tracking, and vocal pitch were assessed throughout the interaction.

RESULTS

Depression and SIA were positively correlated with NA, poorer self-rated performance, and vocal pitch. Highly socially anxious women engaged in less mutual eye gaze than highly socially anxious men. Also, vocal pitch was inversely associated with mutual eye gaze and positively related to NA and (nonsignificantly) to self-ratings of poor performance. Finally, our data partially replicated past research on the use of vocal pitch during social stress to detect social anxiety disorder.

LIMITATIONS

The current sample is relatively homogenous in educational attainment, age, and race. All research confederates were women. Future research should examine whether these archival data replicate with the latest telecommunication technologies.

CONCLUSION

Our findings highlight nuanced relationships among SIA, depression, emotions, self-perceptions, and biobehavioral indicators of submissive behavior-in response to an ambiguously negative/positive social interaction. Sex/gender may interact with these effects, emphasizing considerations for research method designs.

Interpersonal dynamics of vocal fundamental frequency in couples: Depressive symptoms, anxiety symptoms, and relationship distress

Given the bidirectional association between psychopathology and relationship distress, an in-depth understanding of couples' interaction processes that contribute to psychopathology is needed. This study examined the interpersonal dynamics of vocally-encoded emotional arousal (fundamental frequency, f0) during couple conversations and their associations with depressive symptoms, anxiety symptoms, and relationship distress. Data from eight samples were pooled (N = 404 couples) to examine (a) overall trajectories of f0 across the interaction and (b) moment-by-moment intraindividual changes in and interpersonal reactivity to partners' f0. Multilevel growth models and repeated-measures actor-partner interdependence models demonstrated that individuals with more severe depression showed more synchronizing reactivity to their partners' f0 on a moment-by-moment basis, and their overall baseline level of f0 was lower. More severe relationship distress was associated with more steeply increasing trajectories of f0 and with greater synchronizing reactivity to partners' f0. Relative differences in depressive symptoms between the two members of a couple were associated with interpersonal dynamics of f0 as well. There were no associations with anxiety symptoms. Thus, depressive symptoms were associated with characteristic interpersonal dynamics of vocally-encoded emotional arousal; yet, most consistent associations emerged for relationship distress, which future studies on individual psychopathology should take into account.

Deconstructing demographic bias in speech-based machine learning models for digital health

Introduction

Machine learning (ML) algorithms have been heralded as promising solutions to the realization of assistive systems in digital healthcare, due to their ability to detect fine-grain patterns that are not easily perceived by humans. Yet, ML algorithms have also been critiqued for treating individuals differently based on their demography, thus propagating existing disparities. This paper explores gender and race bias in speech-based ML algorithms that detect behavioral and mental health outcomes.

Methods

This paper examines potential sources of bias in the data used to train the ML, encompassing acoustic features extracted from speech signals and associated labels, as well as in the ML decisions. The paper further examines approaches to reduce existing bias via using the features that are the least informative of one's demographic information as the ML input, and transforming the feature space in an adversarial manner to diminish the evidence of the demographic information while retaining information about the focal behavioral and mental health state.

Results

Results are presented in two domains, the first pertaining to gender and race bias when estimating levels of anxiety, and the second pertaining to gender bias in detecting depression. Findings indicate the presence of statistically significant differences in both acoustic features and labels among demographic groups, as well as differential ML performance among groups. The statistically significant differences present in the label space are partially preserved in the ML decisions. Although variations in ML performance across demographic groups were noted, results are mixed regarding the models' ability to accurately estimate healthcare outcomes for the sensitive groups.

Discussion

These findings underscore the necessity for careful and thoughtful design in developing ML models that are capable of maintaining crucial aspects of the data and perform effectively across all populations in digital healthcare applications.

Re-assessing the assessment of fears of positive and negative evaluation: Scale development and psychometric evaluation of the Bivalent Fear of Evaluation Scale (BFOES)

The bivalent fear of evaluation (BFOE) model of social anxiety divides fear of evaluation into two distinct valences: fear of positive evaluation (FPE) and fear of negative evaluation (FNE). However, there is evidence that the two most widely utilized and psychometrically supported measures of FNE and FPE contain items which are ambiguous with regard to valence of evaluative fear. To formally address this, the BFOE Scale (BFOES) was developed, by merging items from measures of FNE and FPE into a single scale with an integrated response format. The present studies examined the psychometric profile of the BFOES across a large pooled archival dataset (N = 2216), which included approximately 10 % (n = 224) patients with social anxiety disorder (SAD). The factorial validity, internal consistency, and construct validity of the BFOES were examined. Additionally, item response theory analyses were employed for the purpose of merging items from self-report scales which utilized different Likert-type response formats. Results from both studies provided support for the psychometric profile of the BFOES. The implications of the BFOES for the assessment of social anxiety, and theoretical models of fear of evaluation and SAD, are discussed.

Multimodal Mental Health Digital Biomarker Analysis From Remote Interviews Using Facial, Vocal, Linguistic, and Cardiovascular Patterns

OBJECTIVE

Psychiatric evaluation suffers from subjectivity and bias, and is hard to scale due to intensive professional training requirements. In this work, we investigated whether behavioral and physiological signals, extracted from tele-video interviews, differ in individuals with psychiatric disorders.

METHODS

Temporal variations in facial expression, vocal expression, linguistic expression, and cardiovascular modulation were extracted from simultaneously recorded audio and video of remote interviews. Averages, standard deviations, and Markovian process-derived statistics of these features were computed from 73 subjects. Four binary classification tasks were defined: detecting 1) any clinically-diagnosed psychiatric disorder, 2) major depressive disorder, 3) self-rated depression, and 4) self-rated anxiety. Each modality was evaluated individually and in combination.

RESULTS

Statistically significant feature differences were found between psychiatric and control subjects. Correlations were found between features and self-rated depression and anxiety scores. Heart rate dynamics provided the best unimodal performance with areas under the receiver-operator curve (AUROCs) of 0.68-0.75 (depending on the classification task). Combining multiple modalities provided AUROCs of 0.72-0.82.

CONCLUSION

Multimodal features extracted from remote interviews revealed informative characteristics of clinically diagnosed and self-rated mental health status.

SIGNIFICANCE

The proposed multimodal approach has the potential to facilitate scalable, remote, and low-cost assessment for low-burden automated mental health services.

Multimodal mental health assessment with remote interviews using facial, vocal, linguistic, and cardiovascular patterns

Objective

The current clinical practice of psychiatric evaluation suffers from subjectivity and bias, and requires highly skilled professionals that are often unavailable or unaffordable. Objective digital biomarkers have shown the potential to address these issues. In this work, we investigated whether behavioral and physiological signals, extracted from remote interviews, provided complimentary information for assessing psychiatric disorders.

Methods

Time series of multimodal features were derived from four conceptual modes: facial expression, vocal expression, linguistic expression, and cardiovascular modulation. The features were extracted from simultaneously recorded audio and video of remote interviews using task-specific and foundation models. Averages, standard deviations, and hidden Markov model-derived statistics of these features were computed from 73 subjects. Four binary classification tasks were defined: detecting 1) any clinically-diagnosed psychiatric disorder, 2) major depressive disorder, 3) self-rated depression, and 4) self-rated anxiety. Each modality was evaluated individually and in combination.

Results

Statistically significant feature differences were found between controls and subjects with mental health conditions. Correlations were found between features and self-rated depression and anxiety scores. Visual heart rate dynamics achieved the best unimodal performance with areas under the receiver-operator curve (AUROCs) of 0.68-0.75 (depending on the classification task). Combining multiple modalities achieved AUROCs of 0.72-0.82. Features from task-specific models outperformed features from foundation models.

Conclusion

Multimodal features extracted from remote interviews revealed informative characteristics of clinically diagnosed and self-rated mental health status.

Significance

The proposed multimodal approach has the potential to facilitate objective, remote, and low-cost assessment for low-burden automated mental health services.

Body exposure and vocal analysis: validation of fundamental frequency as a correlate of emotional arousal and valence

Introduction

Vocal analysis of fundamental frequency (f0) represents a suitable index to assess emotional activation. However, although f0 has often been used as an indicator of emotional arousal and different affective states, its psychometric properties are unclear. Specifically, there is uncertainty regarding the validity of the indices of f0_mean and f0_{variabilitymeasures} (f0_dispersion, f0_range, and f0_SD) and whether higher or lower f0 indices are associated with higher arousal in stressful situations. The present study therefore aimed to validate f0 as a marker of vocally encoded emotional arousal, valence, and body-related distress during body exposure as a psychological stressor.

Methods

N = 73 female participants first underwent a 3-min, non-activating neutral reference condition, followed by a 7-min activating body exposure condition. Participants completed questionnaires on affect (i.e., arousal, valence, body-related distress), and their voice data and heart rate (HR) were recorded continuously. Vocal analyses were performed using Praat, a program for extracting paralinguistic measures from spoken audio.

Results

The results revealed no effects for f0 and state body dissatisfaction or general affect. F0_mean correlated positively with self-reported arousal and negatively with valence, but was not correlated with HR_mean/maximum. No correlations with any measure were found for any f0_{variabililtymeasures}.

Discussion

Given the promising findings regarding f0_mean for arousal and valence and the inconclusive findings regarding f0 as a marker of general affect and body-related distress, it may be assumed that f0_mean represents a valid global marker of emotional arousal and valence rather than of concrete body-related distress. In view of the present findings regarding the validity of f0, it may be suggested that f0_mean, but not f0_{variabilitymeasures}, can be used to assess emotional arousal and valence in addition to self-report measures, which is less intrusive than conventional psychophysiological measures.

Multimodal Remote Research on Social Anxiety Using a New Teleconferencing Paradigm

Background

Social anxiety is a prevalent mental health concern. Models of social anxiety incorporate multifaceted components from cognitive appraisals to attention as factors maintaining the disorder. Multimodal research investigating multiple facets of social anxiety simultaneously offers an important avenue to advance our understanding of the disorder.

Methods

The current study tested a novel, internet-based simulated teleconferencing interaction social stressor challenge and included the collection of self-report, eye-tracking, and auditory vocal data during the challenge. Participants (N = 262) completed two interactions. The pre-recorded male and female audience members (assigned to display interest or uninterest) discussed a topic and then prompted the participant to speak on that topic.

Results

Fidelity indices revealed that most participants rated the interactions with the simulated audience as realistic; reported heightened subjective distress during the simulated teleconferencing interactions; and correctly rated audience members' level of interest. As predicted, social anxiety predicted participants' subjective distress during the simulated teleconferencing interactions. Findings from audio vocal and eye tracking analyses largely corresponded to prior research - indicating that social anxiety influences audio vocal responses and patterns of attention during social stressors.

Conclusions

Taken together, these findings suggest that the simulated teleconferencing interaction framework introduced here offers a potentially useful approach for the remote investigation of mechanisms underpinning social anxiety.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10608-023-10371-y.

Taking a Closer Look at Social Performance in Childhood Social Anxiety Disorder: Biopsychosocial Context Considerations and Effects of Cognitive Behavior Therapy

Models of social anxiety disorder (SAD) describe shortfalls in child social performance, whereas empirically, children often show a deficit only in subjective and not objective performance. We examined social performance in relation to possible changes (before and after cognitive behavior therapy [CBT] including social skills training) and to an objective parameter (vocal arousal). Children with SAD were expected to subjectively judge their behavior as less competent than healthy control (HC) children despite a lack of objective differences. Children receiving CBT were expected to show a change in subjective and objective social performance in comparison to children waiting for treatment. Exploratory correlation analyses were used to disentangle the relation between social performance and vocal arousal. One hundred and nineteen children (64 with and 55 without SAD; aged 9-13 years) completed a Trier Social Stress Test (TSST). Children with SAD participated in a second TSST after CBT or waiting. Performance was assessed by self-report and by blinded observers. Vocal arousal was analyzed by audio recording. Children with SAD were objectively assessed as more socially competent than HC children; subjectively, children with SAD showed lower social performance. CBT showed no effect on subjective or objective performance ratings. Vocal arousal did not correlate with social performance. Results need to be considered carefully, as psychometric problems appeared that had not been considered in previous studies. The surprising lack of CBT effects suggests a need to focus on cognitions surrounding social performance. Further, social skills training should not be a standard SAD treatment component but used only if necessary.

Acoustic and Linguistic Features of Impromptu Speech and Their Association With Anxiety: Validation Study

BACKGROUND

The measurement and monitoring of generalized anxiety disorder requires frequent interaction with psychiatrists or psychologists. Access to mental health professionals is often difficult because of high costs or insufficient availability. The ability to assess generalized anxiety disorder passively and at frequent intervals could be a useful complement to conventional treatment and help with relapse monitoring. Prior work suggests that higher anxiety levels are associated with features of human speech. As such, monitoring speech using personal smartphones or other wearable devices may be a means to achieve passive anxiety monitoring.

OBJECTIVE

This study aims to validate the association of previously suggested acoustic and linguistic features of speech with anxiety severity.

METHODS

A large number of participants (n=2000) were recruited and participated in a single web-based study session. Participants completed the Generalized Anxiety Disorder 7-item scale assessment and provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test. Acoustic and linguistic speech features were a priori selected based on the existing speech and anxiety literature, along with related features. Associations between speech features and anxiety levels were assessed using age and personal income as covariates.

RESULTS

Word count and speaking duration were negatively correlated with anxiety scores (r=-0.12; P<.001), indicating that participants with higher anxiety scores spoke less. Several acoustic features were also significantly (P<.05) associated with anxiety, including the mel-frequency cepstral coefficients, linear prediction cepstral coefficients, shimmer, fundamental frequency, and first formant. In contrast to previous literature, second and third formant, jitter, and zero crossing rate for the z score of the power spectral density acoustic features were not significantly associated with anxiety. Linguistic features, including negative-emotion words, were also associated with anxiety (r=0.10; P<.001). In addition, some linguistic relationships were sex dependent. For example, the count of words related to power was positively associated with anxiety in women (r=0.07; P=.03), whereas it was negatively associated with anxiety in men (r=-0.09; P=.01).

CONCLUSIONS

Both acoustic and linguistic speech measures are associated with anxiety scores. The amount of speech, acoustic quality of speech, and gender-specific linguistic characteristics of speech may be useful as part of a system to screen for anxiety, detect relapse, or monitor treatment.

Exploring the Interpersonal Level of Music Performance Anxiety: Online Listener's Accuracy in Detecting Performer Anxiety

Music performance anxiety (MPA) affects musicians at various stages of a performance, from its preparation until the aftermath of its delivery. Given the commonality and potentially grave consequences of MPA, it is understandable that much attention has been paid to the musician experiencing it. Consequently, we have learned a great deal about the intrapersonal level of MPA: how to measure it, treatments, experimental manipulations, and subjective experiences. However, MPA may also manifest at an interpersonal level by influencing how the performance is perceived. Yet, this has not yet been measured. This exploratory online study focuses on the listener’s perception of anxiety and compares it to the musician’s actual experienced anxiety. Forty-eight participants rated the amount of perceived anxiety of a pianist performing two pieces of contrasting difficulty in online-recital and practice conditions. Participants were presented with two stimulus modality conditions of the performance: audiovisual and audio-only. The listener’s perception of anxiety and its similarity to the musician’s experienced anxiety varies depending on variables such as the piece performed, the stimulus modality, as well as interactions between these variables and the listener’s musical background. We discuss the implications for performance and future research on the interpersonal level of MPA.

Screening for Generalized Anxiety Disorder from Acoustic and Linguistic Features of Impromptu Speech: Prediction Model Evaluation Study (Preprint)

Background

Frequent interaction with mental health professionals is required to screen, diagnose, and track mental health disorders. However, high costs and insufficient access can make frequent interactions difficult. The ability to assess a mental health disorder passively and at frequent intervals could be a useful complement to the conventional treatment. It may be possible to passively assess clinical symptoms with high frequency by characterizing speech alterations collected using personal smartphones or other wearable devices. The association between speech features and mental health disorders can be leveraged as an objective screening tool.

Objective

This study aimed to evaluate the performance of a model that predicts the presence of generalized anxiety disorder (GAD) from acoustic and linguistic features of impromptu speech on a larger and more generalizable scale than prior studies did.

Methods

A total of 2000 participants were recruited, and they participated in a single web-based session. They completed the Generalized Anxiety Disorder-7 item scale assessment and provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test. We used the linguistic and acoustic features that were found to be associated with anxiety disorders in previous studies along with demographic information to predict whether participants fell above or below the screening threshold for GAD based on the Generalized Anxiety Disorder-7 item scale threshold of 10. Separate models for each sex were also evaluated. We reported the mean area under the receiver operating characteristic (AUROC) from a repeated 5-fold cross-validation to evaluate the performance of the models.

Results

A logistic regression model using only acoustic and linguistic speech features achieved a significantly greater prediction accuracy than a random model did (mean AUROC 0.57, SD 0.03; P<.001). When separately assessing samples from female participants, we observed a mean AUROC of 0.55 (SD 0.05; P=.01). The model constructed from the samples from male participants achieved a mean AUROC of 0.57 (SD 0.07; P=.002). The mean AUROC increased to 0.62 (SD 0.03; P<.001) on the all-sample data set when demographic information (age, sex, and income) was included, indicating the importance of demographics when screening for anxiety disorders. The performance also increased for the female sample to a mean of 0.62 (SD 0.04; P<.001) when using demographic information (age and income). An increase in performance was not observed when demographic information was added to the model constructed from the male samples.

Conclusions

A logistic regression model using acoustic and linguistic speech features, which have been suggested to be associated with anxiety disorders in prior studies, can achieve above-random accuracy for predicting GAD. Importantly, the addition of basic demographic variables further improves model performance, suggesting a role for speech and demographic information to be used as automated, objective screeners of GAD.

Klang der Depression

Hintergrund

Die Depression ist neben kognitiven, emotionalen, sozialen und psychomotorischen Beeinträchtigungen durch spezifische stimmliche Merkmale gekennzeichnet. Bisher existieren erst wenige Studien, die diese in klinischen Kontexten und in Abgrenzung zu Gesunden untersuchten.

Fragestellung

Untersucht wurde der Zusammenhang zwischen Depressivität und paraverbalen Merkmalen bei depressiven Patient_innen und gesunden Probanden.

Methode

In einem mehrschrittigen Verfahren wurden die Audioinhalte von Anamneseinterviews mit Depressiven (n = 15) und Gesunden (n = 15) mit einer Software annotiert und transkribiert. Die paraverbalen Merkmale Grundfrequenz der Stimme, Spannweite der Stimme, Sprechgeschwindigkeit und Pausenlänge wurden automatisiert bestimmt. Mithilfe hierarchisch linearer Modelle wurde der Einfluss der Gruppenzugehörigkeit, Depressivität, Ängstlichkeit sowie psychischer und körperlicher Gesundheit auf die paraverbalen Merkmale analysiert.

Ergebnisse

Es zeigte sich ein Zusammenhang zwischen Depressivität und der Sprechgeschwindigkeit. Zwischen der Spannweite der Grundfrequenz, der Pausenlänge und Depressivität konnten tendenzielle Zusammenhänge gefunden werden. Depressive Patient_innen sind im Vergleich zu Gesunden durch eine monotone Sprache, eine geringe Sprechgeschwindigkeit und längere Pausen gekennzeichnet. Sprechgeschwindigkeit und Pausenlänge waren ebenfalls mit Ängstlichkeit assoziiert.

Diskussion

Sprechgeschwindigkeit, Pausenlänge und Spannweite der Grundfrequenz scheinen relevante Indikatoren für Depressivität/ggf. Ängstlichkeit zu sein. Die Spannweite der Grundfrequenz ist eher depressionsspezifisch, während Pausenlänge und Sprechgeschwindigkeit mit Depressivität und Ängstlichkeit assoziiert sind. Zukünftige Studien sollten die Zusammenhänge in größeren Stichproben verschiedener klinischer Störungsbilder untersuchen.

Measuring neuropsychiatric symptoms in patients with early cognitive decline using speech analysis

BACKGROUND

Certain neuropsychiatric symptoms (NPS), namely apathy, depression, and anxiety demonstrated great value in predicting dementia progression, representing eventually an opportunity window for timely diagnosis and treatment. However, sensitive and objective markers of these symptoms are still missing. Therefore, the present study aims to investigate the association between automatically extracted speech features and NPS in patients with mild neurocognitive disorders.

METHODS

Speech of 141 patients aged 65 or older with neurocognitive disorder was recorded while performing two short narrative speech tasks. NPS were assessed by the neuropsychiatric inventory. Paralinguistic markers relating to prosodic, formant, source, and temporal qualities of speech were automatically extracted, correlated with NPS. Machine learning experiments were carried out to validate the diagnostic power of extracted markers.

RESULTS

Different speech variables are associated with specific NPS; apathy correlates with temporal aspects, and anxiety with voice quality-and this was mostly consistent between male and female after correction for cognitive impairment. Machine learning regressors are able to extract information from speech features and perform above baseline in predicting anxiety, apathy, and depression scores.

CONCLUSIONS

Different NPS seem to be characterized by distinct speech features, which are easily extractable automatically from short vocal tasks. These findings support the use of speech analysis for detecting subtypes of NPS in patients with cognitive impairment. This could have great implications for the design of future clinical trials as this cost-effective method could allow more continuous and even remote monitoring of symptoms.

Vocal features obtained through automated methods in verbal fluency tasks can aid the identification of mixed episodes in bipolar disorder

There is a lack of consensus on the diagnostic thresholds that could improve the detection accuracy of bipolar mixed episodes in clinical settings. Some studies have shown that voice features could be reliable biomarkers of manic and depressive episodes compared to euthymic states, but none thus far have investigated whether they could aid the distinction between mixed and non-mixed acute bipolar episodes. Here we investigated whether vocal features acquired via verbal fluency tasks could accurately classify mixed states in bipolar disorder using machine learning methods. Fifty-six patients with bipolar disorder were recruited during an acute episode (19 hypomanic, 8 mixed hypomanic, 17 with mixed depression, 12 with depression). Nine different trials belonging to four conditions of verbal fluency tasks-letter, semantic, free word generation, and associational fluency-were administered. Spectral and prosodic features in three conditions were selected for the classification algorithm. Using the leave-one-subject-out (LOSO) strategy to train the classifier, we calculated the accuracy rate, the F1 score, and the Matthews correlation coefficient (MCC). For depression versus mixed depression, the accuracy and F1 scores were high, i.e., respectively 0.83 and 0.86, and the MCC was of 0.64. For hypomania versus mixed hypomania, accuracy and F1 scores were also high, i.e., 0.86 and 0.75, respectively, and the MCC was of 0.57. Given the high rates of correctly classified subjects, vocal features quickly acquired via verbal fluency tasks seem to be reliable biomarkers that could be easily implemented in clinical settings to improve diagnostic accuracy.

Association between acoustic speech features and non-severe levels of anxiety and depression symptoms across lifespan

BACKGROUND

Several studies have investigated the acoustic effects of diagnosed anxiety and depression. Anxiety and depression are not characteristics of the typical aging process, but minimal or mild symptoms can appear and evolve with age. However, the knowledge about the association between speech and anxiety or depression is scarce for minimal/mild symptoms, typical of healthy aging. As longevity and aging are still a new phenomenon worldwide, posing also several clinical challenges, it is important to improve our understanding of non-severe mood symptoms' impact on acoustic features across lifetime. The purpose of this study was to determine if variations in acoustic measures of voice are associated with non-severe anxiety or depression symptoms in adult population across lifetime.

METHODS

Two different speech tasks (reading vowels in disyllabic words and describing a picture) were produced by 112 individuals aged 35-97. To assess anxiety and depression symptoms, the Hospital Anxiety Depression Scale (HADS) was used. The association between the segmental and suprasegmental acoustic parameters and HADS scores were analyzed using the linear multiple regression technique.

RESULTS

The number of participants with presence of anxiety or depression symptoms is low (>7: 26.8% and 10.7%, respectively) and non-severe (HADS-A: 5.4 ± 2.9 and HADS-D: 4.2 ± 2.7, respectively). Adults with higher anxiety symptoms did not present significant relationships associated with the acoustic parameters studied. Adults with increased depressive symptoms presented higher vowel duration, longer total pause duration and short total speech duration. Finally, age presented a positive and significant effect only for depressive symptoms, showing that older participants tend to have more depressive symptoms.

CONCLUSIONS

Non-severe depression symptoms can be related to some acoustic parameters and age. Depression symptoms can be explained by acoustic parameters even among individuals without severe symptom levels.

Vocal arousal: a physiological correlate of body distress in women with overweight and obesity

PURPOSE

Numerous studies highlight the relevance of body image in the development and maintenance of overweight and obesity mostly using self-reported data. Given the importance of physiological assessment methods, the present study aimed at investigating vocally encoded emotional arousal as a correlate of body distress in women with overweight and obesity.

METHODS

Cognitions of women with overweight and obesity (OW; n = 22) as well as normal weight controls (NW; n = 22) were assessed by means of a thought-sampling procedure during a mirror exposure and a control condition. Fundamental frequency (f0) as a marker of vocally encoded emotional arousal as well as verbalized body-related cognitions were analyzed during this experimental task.

RESULTS

A stronger increase in f0 between the control and the mirror exposure condition was found in OW compared to NW. Furthermore, there were significant positive correlations between vocally encoded emotional arousal and various measures of body image.

CONCLUSION

The findings support the utility of vocally encoded emotional arousal as an objective physiological correlate of the evaluative dimension of body image in women with overweight and obesity.

LEVEL OF EVIDENCE

Level I, experimental study.

Fundamental frequency during cognitive preparation and its impact on therapy outcome for panic disorder with Agoraphobia

BACKGROUND

Cognitive preparation plays a crucial role in CBT with exposure for panic disorder and agoraphobia. High emotional arousal while developing the exposure rationale might impair patients' cognitive capacities for processing information about treatment and impede therapeutic outcome.

OBJECTIVE

This study investigates whether patients' vocally encoded emotional arousal, assessed by fundamental frequency (f₀), during rationale development is associated with premature treatment dropout, insight into the rationale, and symptom reduction.

METHODS

Patients' (N = 197, mean age 36.1 years, 79.2% female) f₀ during rationale development was measured based on treatment videos from a randomized controlled trial of CBT for panic disorder and agoraphobia. Insight was rater assessed. Symptom severity was self- and rater assessed at the beginning and end of therapy.

RESULTS

Higher f₀ mean during rationale development was associated with lower probability of insight and less reduction in avoidance behavior. f₀ was not associated with dropout. Insight was associated with lower probability of dropout and partially mediated the association between f₀ and avoidance reduction.

DISCUSSION

This study highlights the importance of emotional arousal during cognitive preparation for exposure. Therapists should ensure that patients are not too highly aroused while learning about the exposure rationale as an important step in treatment.

Automated assessment of psychiatric disorders using speech: A systematic review

OBJECTIVE

There are many barriers to accessing mental health assessments including cost and stigma. Even when individuals receive professional care, assessments are intermittent and may be limited partly due to the episodic nature of psychiatric symptoms. Therefore, machine-learning technology using speech samples obtained in the clinic or remotely could one day be a biomarker to improve diagnosis and treatment. To date, reviews have only focused on using acoustic features from speech to detect depression and schizophrenia. Here, we present the first systematic review of studies using speech for automated assessments across a broader range of psychiatric disorders.

METHODS

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We included studies from the last 10 years using speech to identify the presence or severity of disorders within the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). For each study, we describe sample size, clinical evaluation method, speech-eliciting tasks, machine learning methodology, performance, and other relevant findings.

RESULTS

1395 studies were screened of which 127 studies met the inclusion criteria. The majority of studies were on depression, schizophrenia, and bipolar disorder, and the remaining on post-traumatic stress disorder, anxiety disorders, and eating disorders. 63% of studies built machine learning predictive models, and the remaining 37% performed null-hypothesis testing only. We provide an online database with our search results and synthesize how acoustic features appear in each disorder.

CONCLUSION

Speech processing technology could aid mental health assessments, but there are many obstacles to overcome, especially the need for comprehensive transdiagnostic and longitudinal studies. Given the diverse types of data sets, feature extraction, computational methodologies, and evaluation criteria, we provide guidelines for both acquiring data and building machine learning models with a focus on testing hypotheses, open science, reproducibility, and generalizability.

LEVEL OF EVIDENCE

3a.

"Fear guides the eyes of the beholder": Assessing gaze avoidance in social anxiety disorder via covert eye tracking of dynamic social stimuli

Gaze avoidance is an important feature of social anxiety disorder (SAD) and may serve as a biobehavioral marker of SAD. The purpose of the present study was to replicate and extend findings on gaze avoidance in SAD via eye tracking during a computerized social simulation. Patients with SAD (n = 27) and a (sub)sample of demographically-matched healthy controls (HC; n = 22) completed a computerized, dynamic social simulation task involving video clips of actors giving positive and negative social feedback to the participant. All participants were unknowingly eye tracked during the simulation, and post-study consent was obtained to examine responses. Consistent with the bivalent fear of evaluation (BFOE) model of social anxiety, fear of positive evaluation related systematically to state anxiety in response to positive social feedback, and fear of negative evaluation related systematically to state anxiety in response to negative social feedback. Moreover, compared to HCs, SAD patients exhibited significantly greater global gaze avoidance in response to both the positive and negative video clips. Our results provide strong additional support for gaze avoidance as a biobehavioral marker of SAD, as well as additional support for the BFOE model. Implications for the assessment and treatment of SAD are discussed.

Cognitive-Emotional Involvement During Mirror Exposure Is Not Accompanied by Physiological Activation in Binge Eating Disorder

Body image interventions have been shown to reduce self-reported cognitive-emotional facets of body image disturbance in binge eating disorder (BED). However, more objective assessment methods are required to evaluate the effects of these interventions. Therefore, the present study aimed at investigating the usefulness of vocally encoded emotional arousal as physiological correlate of body dissatisfaction during mirror exposure in women with BED. Women with BED (n = 60) and weight-matched controls (CG; n = 60) participated in an experimental thought-sampling procedure including a mirror exposure and a control condition in a repeated-measures design. Fundamental frequency as a vocal correlate of emotional arousal as well as negative, neutral, and positive body-related cognitions during both conditions were analyzed. In line with our hypotheses, the BED group verbalized more negative, and less positive and neutral body-related cognitions during the mirror exposure condition compared to the CG. Contrary to our hypotheses, though, there was a stronger increase in physiological arousal between the control and the mirror exposure condition in the CG relative to the BED group. Furthermore, a significant negative correlation between fundamental frequency and the severity of cognitive-emotional body image disturbances emerged. The findings indicate a cognitive-emotional over-involvement with physical appearance during mirror exposure in women with BED compared to weight-matched controls in the absence of a corresponding physiological pattern. Results are discussed in terms of an impaired ability of women with BED to show adequate physiological responses to body-related stress. In addition, methodological recommendations for future research are presented.

A Weakly Supervised Learning Framework for Detecting Social Anxiety and Depression

Although social anxiety and depression are common, they are often underdiagnosed and undertreated, in part due to difficulties identifying and accessing individuals in need of services. Current assessments rely on client self-report and clinician judgment, which are vulnerable to social desirability and other subjective biases. Identifying objective, nonburdensome markers of these mental health problems, such as features of speech, could help advance assessment, prevention, and treatment approaches. Prior research examining speech detection methods has focused on fully supervised learning approaches employing strongly labeled data. However, strong labeling of individuals high in symptoms or state affect in speech audio data is impractical, in part because it is not possible to identify with high confidence which regions of a long speech indicate the person's symptoms or affective state. We propose a weakly supervised learning framework for detecting social anxiety and depression from long audio clips. Specifically, we present a novel feature modeling technique named NN2Vec that identifies and exploits the inherent relationship between speakers' vocal states and symptoms/affective states. Detecting speakers high in social anxiety or depression symptoms using NN2Vec features achieves F-1 scores 17% and 13% higher than those of the best available baselines. In addition, we present a new multiple instance learning adaptation of a BLSTM classifier, named BLSTM-MIL. Our novel framework of using NN2Vec features with the BLSTM-MIL classifier achieves F-1 scores of 90.1% and 85.44% in detecting speakers high in social anxiety and depression symptoms.

Perceptions of Competence, Strength, and Age Influence Voters to Select Leaders with Lower-Pitched Voices

Voters prefer leaders with lower-pitched voices because they are perceived as stronger, having greater physical prowess, more competent, and having greater integrity. An alternative hypothesis that has yet to be tested is that lower-pitched voices are perceived as older and thus wiser and more experienced. Here the relationships between candidate voice pitch, candidate age, and electoral success are examined with two experiments. Study 1 tests whether voters discriminate on candidate age. The results show that male and female candidates in their 40s and 50s, the time in the lifecycle when voice pitch is at its lowest, are preferred over candidates in their 30s, 60s, and 70s. Study 2 shows that the preference for leaders with lower-pitched voices correlates with the perception that speakers with lower voices are stronger, more competent, and older, but the influence of perception of age on vote choice is the weakest of the three.

Being "in" or "out" of the game: subjective and acoustic reactions to exclusion and popularity in social anxiety

Social Anxiety (SA) has been shown to be associated with compensatory deficits in pro-social behavior following exclusion and with failure to capitalize on social success. We assessed the subjective and expressive responses of high (n = 48) and low (n = 56) socially anxious individuals to exclusion, acceptance, and popularity induced by a participation in an online ball-tossing game. Before the manipulation, participants read aloud neutral and command utterances. Following the manipulation, participants rated their mood and cognitions and re-read the utterances. Acoustic properties (fundamental frequency-mF0, vocal intensity) of these utterances were analyzed. We found greater differences in self-esteem between high and low socially anxious individuals following the exclusion condition, as compared to the acceptance condition. Among low socially anxious individuals, exclusion promoted increased vocal confidence, as indicated by decreased mF0 and increased vocal intensity in uttering commands; High socially anxious individuals exhibited an opposite reaction, responding to exclusion by decreased vocal confidence. Following popularity, high SA was associated with decreased enhancement in mood and self-esteem in women but not in men. Consistent with evolutionary and interpersonal accounts of SA, we highlight the importance of examining the effects of SA and gender on events indicating unambiguous and unanimous social acceptance. Examining reactivity to changes in belongingness may have important implications for understanding the core mechanisms of SA.

More than a face: a unified theoretical perspective on nonverbal social cue processing in social anxiety

Processing of nonverbal social cues (NVSCs) is essential to interpersonal functioning and is particularly relevant to models of social anxiety. This article provides a review of the literature on NVSC processing from the perspective of social rank and affiliation biobehavioral systems (ABSs), based on functional analysis of human sociality. We examine the potential of this framework for integrating cognitive, interpersonal, and evolutionary accounts of social anxiety. We argue that NVSCs are uniquely suited to rapid and effective conveyance of emotional, motivational, and trait information and that various channels are differentially effective in transmitting such information. First, we review studies on perception of NVSCs through face, voice, and body. We begin with studies that utilized information processing or imaging paradigms to assess NVSC perception. This research demonstrated that social anxiety is associated with biased attention to, and interpretation of, emotional facial expressions (EFEs) and emotional prosody. Findings regarding body and posture remain scarce. Next, we review studies on NVSC expression, which pinpointed links between social anxiety and disturbances in eye gaze, facial expressivity, and vocal properties of spontaneous and planned speech. Again, links between social anxiety and posture were understudied. Although cognitive, interpersonal, and evolutionary theories have described different pathways to social anxiety, all three models focus on interrelations among cognition, subjective experience, and social behavior. NVSC processing and production comprise the juncture where these theories intersect. In light of the conceptualizations emerging from the review, we highlight several directions for future research including focus on NVSCs as indexing reactions to changes in belongingness and social rank, the moderating role of gender, and the therapeutic opportunities offered by embodied cognition to treat social anxiety.