Evaluating voice samples as a potential source of information about personality

Speech-based personality prediction using deep learning with acoustic and linguistic embeddings

This study introduces a novel method for predicting the Big Five personality traits through the analysis of speech samples, advancing the field of computational personality assessment. We collected data from 2045 participants who completed a self-reported Big Five personality questionnaire and provided free-form speech samples by introducing themselves without constraints on content. Using pre-trained convolutional neural networks and transformer-based models, we extracted embeddings representing both acoustic features (e.g., tone, pitch, rhythm) and linguistic content from the speech samples. These embeddings were combined and input into gradient boosted tree models to predict personality traits. Our results indicate that personality traits can be effectively predicted from speech, with correlation coefficients between predicted scores and self-reported scores ranging from 0.26 (extraversion) to 0.39 (neuroticism), and from 0.39 to 0.60 for disattenuated correlations. Intraclass correlations show moderate to high consistency in our model's predictions. This approach captures the subtle ways in which personality traits are expressed through both how people speak and what they say. Our findings underscore the potential of voice-based assessments as a complementary tool in psychological research, providing new insights into the connection between speech and personality.

Feasibility of Big Data Analytics to Assess Personality Based on Voice Analysis

(1) Background: As far back as the 1930s, it was already thought that gestures, clothing, speech, posture, and gait could express an individual's personality. Different research programs, some focused on linguistic cues, were launched, though results were inconsistent. The development of new speech analysis technology and the generalization of big data analysis have created an opportunity to test the predictive power of voice features on personality dimensions. This study aims to explore the feasibility of an automatic personality assessment system in the context of personnel selection. (2) Methods: One hundred participants were recorded during an individual interview for voice analysis. They also completed the NEO-FFI and were required to ask and collect the assessment of their personality by a close significant other. Furthermore, an expert estimated participants' personality dimensions based on the viewing of the recorded interviews. (3) Results: Results showed there are specific voice features related to the externalization of individuals' personalities (predictions ranging from 0.3 to 0.4). Voice features also predicted significant others' estimations and expert ratings of the target individual's personality, though the features were not exactly the same. (4) Conclusions: It is noteworthy that predictions were made based on voice recordings obtained using ordinary devices in controlled but not restricted speech situations, which may make such an approach a promising tool for personality assessment in contexts such as personnel selection.

Predictive modeling of neuroticism in depressed and non-depressed cohorts using voice features

BACKGROUND

Neuroticism's impact on psychopathological and physical health issues has significant public health implications. Multiple studies confirm its predictive effect on suicide risk among depressed patients. However, previous research lacks a standardized criterion for assessing neuroticism through speech, often relying on simple features (such as pitch, loudness and MFCCs). This study aims to improve upon this by extracting features using advanced pre-trained speaker embedding models (i-vector and x-vector extractors). Additionally, unlike prior studies utilizing general population data, we explore neuroticism prediction in depressed and non-depressed subgroups.

METHODS

We collected edited discourse data from clinical interviews of 3580 depressed individuals and 4016 healthy individuals from the CONVERGE study. Instead of solely extracting Low-Level Acoustic Descriptors, we incorporated i-vector and x-vector features. We compared the performance of three different features in predicting neuroticism and explored their combination to enhance model accuracy.

RESULTS

The SVR model, combining three speech features with downscaled features to 300, exhibited the highest performance in predicting neuroticism scores. It achieved a coefficient of determination (R-squared) of 0.3 or higher and a correlation of 0.56 between predicted and actual values. The predictive classification accuracy of speech features for neuroticism in specific populations (healthy and depressed) exceeded 60 %.

LIMITATIONS

This study included only women.

CONCLUSION

Combining diverse speech features enhances the predictive capacity of models using speech features to assess neuroticism, particularly in specific populations. This study lays the foundation for future exploration of speech features in neuroticism prediction.