Personality Research and Assessment in the Era of Machine Learning

Detecting Narcissism From Older Adults' Daily Language Use: A Machine Learning Approach

OBJECTIVES

Narcissism has been associated with poorer quality social connections in late life, yet less is known about how narcissism is associated with older adults' daily social interactions. This study explored the associations between narcissism and older adults' language use throughout the day.

METHODS

Participants aged 65-89 (N = 281) wore electronically activated recorders which captured ambient sound for 30 s every 7 min across 5-6 days. Participants also completed the Narcissism Personality Inventory-16 scale. We used Linguistic Inquiry and Word Count to extract 81 linguistic features from sound snippets and applied a supervised machine learning algorithm (random forest) to evaluate the strength of links between narcissism and each linguistic feature.

RESULTS

The random forest model showed that the top 5 linguistic categories that displayed the strongest associations with narcissism were first-person plural pronouns (e.g., we), words related to achievement (e.g., win, success), to work (e.g., hiring, office), to sex (e.g., erotic, condom), and that signal desired state (e.g., want, need).

DISCUSSION

Narcissism may be demonstrated in everyday life via word use in conversation. More narcissistic individuals may have poorer quality social connections because their communication conveys an emphasis on self and achievement rather than affiliation or topics of interest to the other party.

Psychology of Changeability: Basic Principles of Description of Processual Nature of Personality

The article reveals the basic principles of the processual approach to the study of personality, which have a natural scientific foundation and are based on the ideas of the philosophy of instability of I. Prigogine. The developed processual approach is designed to overcome the opposition of variability and stability of personality, and to explain how the personality remains sustainable, being in constant change. This question, formulated by Mischel, continues to be debated in modern theoretical and methodological studies, maintaining the controversy between supporters of structural and dynamic paradigms of personality research. The significant role of the theory of non-equilibrium systems for understanding personality changeability is revealed in connection with explanation of its processual nature, when the leading role is played not by the variety of elements and their dynamics, but by self-organization of personality components. The processuality of personality determines its ability to move to new levels of functioning, to become more complex, to unpredictably change structurally and meaningfully in an infinite variety of options. The processual nature of personality focuses attention of a researcher on the potentially possible, when the object of research is not the existing, but the emerging. The methodological principles for describing the processual nature of personality are the principle of contextuality, revealing the sensitivity of its subsystems to fluctuations, the principle of multiplicity (uncertainty) of states, explaining the growth of non-adaptive forms and variability in critical situations and turning points, the principle of historicity, defining events as a starting point of imbalance and consistency, the principles of complementarity and wholeness, describing the dialectic of sustainability and changeability at different levels of functioning (three contexts of personality existence: situational, life and existential).

Using passive sensor data to probe associations of social structure with changes in personality: A synthesis of network analysis and machine learning

Social network analysis (SNA) is an increasingly popular and effective tool for modeling psychological phenomena. Through application to the personality literature, social networks, in conjunction with passive, non-invasive sensing technologies, have begun to offer powerful insight into personality state variability. Resultant constructions of social networks can be utilized alongside machine learning-based frameworks to uniquely model personality states. Accordingly, this work leverages data from a previously published study to combine passively collected wearable sensor information on face-to-face, workplace social interactions with ecological momentary assessments of personality state. Data from 54 individuals across six weeks was used to explore the relative importance of 26 unique structural and nodal social network features in predicting individual changes in each of the Big Five (5F) personality states. Changes in personality state were operationalized by calculating the weekly root mean square of successive differences (RMSSD) in 5F state scores measured daily via self-report. Using only SNA-derived features from wearable sensor data, boosted tree-based machine learning models explained, on average, approximately 28-30% of the variance in individual personality state change. Model introspection implicated egocentric features as the most influential predictors across 5F-specific models, with network efficiency, constraint, and effective size measures among the most important. Feature importance profiles for each 5F model partially echoed previous empirical findings. Results support future efforts focusing on egocentric components of SNA and suggest particular investment in exploring efficiency measures to model personality fluctuations within the workplace setting.

A Failed Cross-Validation Study on the Relationship between LIWC Linguistic Indicators and Personality: Exemplifying the Lack of Generalizability of Exploratory Studies

(1) Background: Previous meta-analytic research found small to moderate relationships between the Big Five personality traits and different linguistic computational indicators. However, previous studies included multiple linguistic indicators to predict personality from an exploratory framework. The aim of this study was to conduct a cross-validation study analyzing the relationships between language indicators and personality traits to test the generalizability of previous results; (2) Methods: 643 Spanish undergraduate students were tasked to write a self-description in 500 words (which was evaluated with the LIWC) and to answer a standardized Big Five questionnaire. Two different analytical approaches using multiple linear regression were followed: first, using the complete data and, second, by conducting different cross-validation studies; (3) Results: The results showed medium effect sizes in the first analytical approach. On the contrary, it was found that language and personality relationships were not generalizable in the cross-validation studies; (4) Conclusions: We concluded that moderate effect sizes could be obtained when the language and personality relationships were analyzed in single samples, but it was not possible to generalize the model estimates to other samples. Thus, previous exploratory results found on this line of research appear to be incompatible with a nomothetic approach.

Grouped feature importance and combined features effect plot

Interpretable machine learning has become a very active area of research due to the rising popularity of machine learning algorithms and their inherently challenging interpretability. Most work in this area has been focused on the interpretation of single features in a model. However, for researchers and practitioners, it is often equally important to quantify the importance or visualize the effect of feature groups. To address this research gap, we provide a comprehensive overview of how existing model-agnostic techniques can be defined for feature groups to assess the grouped feature importance, focusing on permutation-based, refitting, and Shapley-based methods. We also introduce an importance-based sequential procedure that identifies a stable and well-performing combination of features in the grouped feature space. Furthermore, we introduce the combined features effect plot, which is a technique to visualize the effect of a group of features based on a sparse, interpretable linear combination of features. We used simulation studies and real data examples to analyze, compare, and discuss these methods.

Predicting romantic interest during early relationship development: A preregistered investigation using machine learning

There are massive literatures on initial attraction and established relationships. But few studies capture early relationship development: the interstitial period in which people experience rising and falling romantic interest for partners who could—but often do not—become sexual or dating partners. In this study, 208 single participants reported on 1,065 potential romantic partners across 7,179 data points over 7 months. In stage 1, we used random forests (a type of machine learning) to estimate how well different classes of variables (e.g., individual differences vs. target-specific constructs) predicted participants’ romantic interest in these potential partners. We also tested (and found only modest support for) the perceiver × target moderation account of compatibility: the meta-theoretical perspective that some types of perceivers experience greater romantic interest for some types of targets. In stage 2, we used multilevel modeling to depict predictors retained by the random-forests models; robust (positive) main effects emerged for many variables, including sociosexuality, sex drive, perceptions of the partner’s positive attributes (e.g., attractive and exciting), attachment features (e.g., proximity seeking), and perceived interest. Finally, we found no support for ideal partner preference-matching effects on romantic interest. The discussion highlights the need for new models to explain the origin of romantic compatibility.

Predicting Personality and Psychological Distress Using Natural Language Processing: A Study Protocol

Background

Self-report multiple choice questionnaires have been widely utilized to quantitatively measure one's personality and psychological constructs. Despite several strengths (e.g., brevity and utility), self-report multiple choice questionnaires have considerable limitations in nature. With the rise of machine learning (ML) and Natural language processing (NLP), researchers in the field of psychology are widely adopting NLP to assess psychological construct to predict human behaviors. However, there is a lack of connections between the work being performed in computer science and that of psychology due to small data sets and unvalidated modeling practices.

Aims

The current article introduces the study method and procedure of phase II which includes the interview questions for the five-factor model (FFM) of personality developed in phase I. This study aims to develop the interview (semi-structured) and open-ended questions for the FFM-based personality assessments, specifically designed with experts in the field of clinical and personality psychology (phase 1), and to collect the personality-related text data using the interview questions and self-report measures on personality and psychological distress (phase 2). The purpose of the study includes examining the relationship between natural language data obtained from the interview questions, measuring the FFM personality constructs, and psychological distress to demonstrate the validity of the natural language-based personality prediction.

Methods

Phase I (pilot) study was conducted to fifty-nine native Korean adults to acquire the personality-related text data from the interview (semi-structured) and open-ended questions based on the FFM of personality. The interview questions were revised and finalized with the feedback from the external expert committee, consisting of personality and clinical psychologists. Based on the established interview questions, a total of 300 Korean adults will be recruited using a convenience sampling method via online survey. The text data collected from interviews will be analyzed using the natural language processing. The results of the online survey including demographic data, depression, anxiety, and personality inventories will be analyzed together in the model to predict individuals' FFM of personality and the level of psychological distress (phase 2).

Beyond English: Considering Language and Culture in Psychological Text Analysis

The paper discusses the role of language and culture in the context of quantitative text analysis in psychological research. It reviews current automatic text analysis methods and approaches from the perspective of the unique challenges that can arise when going beyond the default English language. Special attention is paid to closed-vocabulary approaches and related methods (and Linguistic Inquiry and Word Count in particular), both from the perspective of cross-cultural research where the analytic process inherently consists of comparing phenomena across cultures and languages and the perspective of generalizability beyond the language and the cultural focus of the original investigation. We highlight the need for a more universal and flexible theoretical and methodological grounding of current research, which includes the linguistic, cultural, and situational specifics of communication, and we provide suggestions for procedures that can be implemented in future studies and facilitate psychological text analysis across languages and cultures.

Detecting Careless Responding in Survey Data Using Stochastic Gradient Boosting

Careless responding is a bias in survey responses that disregards the actual item content, constituting a threat to the factor structure, reliability, and validity of psychological measurements. Different approaches have been proposed to detect aberrant responses such as probing questions that directly assess test-taking behavior (e.g., bogus items), auxiliary or paradata (e.g., response times), or data-driven statistical techniques (e.g., Mahalanobis distance). In the present study, gradient boosted trees, a state-of-the-art machine learning technique, are introduced to identify careless respondents. The performance of the approach was compared with established techniques previously described in the literature (e.g., statistical outlier methods, consistency analyses, and response pattern functions) using simulated data and empirical data from a web-based study, in which diligent versus careless response behavior was experimentally induced. In the simulation study, gradient boosting machines outperformed traditional detection mechanisms in flagging aberrant responses. However, this advantage did not transfer to the empirical study. In terms of precision, the results of both traditional and the novel detection mechanisms were unsatisfactory, although the latter incorporated response times as additional information. The comparison between the results of the simulation and the online study showed that responses in real-world settings seem to be much more erratic than can be expected from the simulation studies. We critically discuss the generalizability of currently available detection methods and provide an outlook on future research on the detection of aberrant response patterns in survey research.

Pitfalls of Machine Learning-Based Personnel Selection

Abstract. In recent years, machine learning (ML) modeling (often referred to as artificial intelligence) has become increasingly popular for personnel selection purposes. Numerous organizations use ML-based procedures for screening large candidate pools, while some companies try to automate the hiring process as far as possible. Since ML models can handle large sets of predictor variables and are therefore able to incorporate many different data sources (often more than common procedures can consider), they promise a higher predictive accuracy and objectivity in selecting the best candidate than traditional personal selection processes. However, there are some pitfalls and challenges that have to be taken into account when using ML for a sensitive issue as personnel selection. In this paper, we address these major challenges – namely the definition of a valid criterion, transparency regarding collected data and decision mechanisms, algorithmic fairness, changing data conditions, and adequate performance evaluation – and discuss some recommendations for implementing fair, transparent, and accurate ML-based selection algorithms.

Explainable AI for Psychological Profiling from Behavioral Data: An Application to Big Five Personality Predictions from Financial Transaction Records

Every step we take in the digital world leaves behind a record of our behavior; a digital footprint. Research has suggested that algorithms can translate these digital footprints into accurate estimates of psychological characteristics, including personality traits, mental health or intelligence. The mechanisms by which AI generates these insights, however, often remain opaque. In this paper, we show how Explainable AI (XAI) can help domain experts and data subjects validate, question, and improve models that classify psychological traits from digital footprints. We elaborate on two popular XAI methods (rule extraction and counterfactual explanations) in the context of Big Five personality predictions (traits and facets) from financial transactions data (N = 6408). First, we demonstrate how global rule extraction sheds light on the spending patterns identified by the model as most predictive for personality, and discuss how these rules can be used to explain, validate, and improve the model. Second, we implement local rule extraction to show that individuals are assigned to personality classes because of their unique financial behavior, and there exists a positive link between the model’s prediction confidence and the number of features that contributed to the prediction. Our experiments highlight the importance of both global and local XAI methods. By better understanding how predictive models work in general as well as how they derive an outcome for a particular person, XAI promotes accountability in a world in which AI impacts the lives of billions of people around the world.

How do you feel during the COVID-19 pandemic? A survey using psychological and linguistic self-report measures, and machine learning to investigate mental health, subjective experience, personality, and behaviour during the COVID-19 pandemic among university students

Background

The WHO has raised concerns about the psychological consequences of the current COVID-19 pandemic, negatively affecting health across societies, cultures and age-groups.

Methods

This online survey study investigated mental health, subjective experience, and behaviour (health, learning/teaching) among university students studying in Egypt or Germany shortly after the first pandemic lockdown in May 2020. Psychological assessment included stable personality traits, self-concept and state-like psychological variables related to (a) mental health (depression, anxiety), (b) pandemic threat perception (feelings during the pandemic, perceived difficulties in describing, identifying, expressing emotions), (c) health (e.g., worries about health, bodily symptoms) and behaviour including perceived difficulties in learning. Assessment methods comprised self-report questions, standardized psychological scales, psychological questionnaires, and linguistic self-report measures. Data analysis comprised descriptive analysis of mental health, linguistic analysis of self-concept, personality and feelings, as well as correlational analysis and machine learning. N = 220 (107 women, 112 men, 1 = other) studying in Egypt or Germany provided answers to all psychological questionnaires and survey items.

Results

Mean state and trait anxiety scores were significantly above the cut off scores that distinguish between high versus low anxious subjects. Depressive symptoms were reported by 51.82% of the student sample, the mean score was significantly above the screening cut off score for risk of depression. Worries about health (mental and physical health) and perceived difficulties in identifying feelings, and difficulties in learning behaviour relative to before the pandemic were also significant. No negative self-concept was found in the linguistic descriptions of the participants, whereas linguistic descriptions of feelings during the pandemic revealed a negativity bias in emotion perception. Machine learning (exploratory) predicted personality from the self-report data suggesting relations between personality and subjective experience that were not captured by descriptive or correlative data analytics alone.

Conclusion

Despite small sample sizes, this multimethod survey provides important insight into mental health of university students studying in Egypt or Germany and how they perceived the first COVID-19 pandemic lockdown in May 2020. The results should be continued with larger samples to help develop psychological interventions that support university students across countries and cultures to stay psychologically resilient during the pandemic.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40359-021-00574-x.

Natural Language Analysis and the Psychology of Verbal Behavior: The Past, Present, and Future States of the Field

Throughout history, scholars and laypeople alike have believed that our words contain subtle clues about what we are like as people, psychologically speaking. However, the ways in which language has been used to infer psychological processes has seen dramatic shifts over time and, with modern computational technologies and digital data sources, we are on the verge of a massive revolution in language analysis research. In this article, we discuss the past and current states of research at the intersection of language analysis and psychology, summarizing the central successes and shortcomings of psychological text analysis to date. We additionally outline and discuss a critical need for language analysis practitioners in the social sciences to expand their view of verbal behavior. Lastly, we discuss the trajectory of interdisciplinary research on language and the challenges of integrating analysis methods across paradigms, recommending promising future directions for the field along the way.