Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy

Predicting neuroticism with open-ended response using natural language processing

Introduction

With rapid advancements in natural language processing (NLP), predicting personality using this technology has become a significant research interest. In personality prediction, exploring appropriate questions that elicit natural language is particularly important because questions determine the context of responses. This study aimed to predict levels of neuroticism-a core psychological trait known to predict various psychological outcomes-using responses to a series of open-ended questions developed based on the five-factor model of personality. This study examined the model's accuracy and explored the influence of item content in predicting neuroticism.

Methods

A total of 425 Korean adults were recruited and responded to 18 open-ended questions about their personalities, along with the measurement of the Five-Factor Model traits. In total, 30,576 Korean sentences were collected. To develop the prediction models, the pre-trained language model KoBERT was used. Accuracy, F1 Score, Precision, and Recall were calculated as evaluation metrics.

Results

The results showed that items inquiring about social comparison, unintended harm, and negative feelings performed better in predicting neuroticism than other items. For predicting depressivity, items related to negative feelings, social comparison, and emotions showed superior performance. For dependency, items related to unintended harm, social dominance, and negative feelings were the most predictive.

Discussion

We identified items that performed better at neuroticism prediction than others. Prediction models developed based on open-ended questions that theoretically aligned with neuroticism exhibited superior predictive performance.

Question-based computational language approach outperforms rating scales in quantifying emotional states

Psychological constructs are commonly quantified with closed-ended rating scales. However, recent advancements in natural language processing (NLP) enable the quantification of open-ended language responses. Here we demonstrate that descriptive word responses analyzed using NLP show higher accuracy in categorizing emotional states compared to traditional rating scales. One group of participants (N = 297) generated narratives related to depression, anxiety, satisfaction, or harmony, summarized them with five descriptive words, and rated them using rating scales. Another group (N = 434) evaluated these narratives (with descriptive words and rating scales) from the author's perspective. The descriptive words were quantified using NLP, and machine learning was used to categorize the responses into the corresponding emotional states. The results showed a significantly higher number of accurate categorizations of the narratives based on descriptive words (64%) than on rating scales (44%), questioning the notion that rating scales are more precise in measuring emotional states than language-based measures.

When I am sixty-four… evaluating language markers of well-being in healthy aging narratives

Natural language use is a promising candidate for the development of innovative measures of well-being to complement self-report measures. The type of words individuals use can reveal important psychological processes that underlie well-being across the lifespan. In this preregistered, cross-sectional study, we propose a conceptual model of language markers of well-being and use written narratives about healthy aging (N = 701) and computerized text analysis (LIWC) to empirically validate the model. As hypothesized, we identified a model with three groups of language markers (reflecting affective, evaluative, and social processes). Initial validation with established self-report scales (N = 30 subscales) showed that these language markers reliably predict core components of well-being and underlying processes. Our results support the concurrent validity of the conceptual language model and allude to the added benefits of language-based measures, which are thought to reflect less conscious processes of well-being. Future research is needed to continue validating language markers of well-being across the lifespan in a theoretically informed and contextualized way, which will lay the foundation for inferring people's well-being from their natural language use.

Predicting attitudes toward ambiguity using natural language processing on free descriptions for open-ended question measurements

Individual traits and reactions to ambiguity differ and are conceptualized in terms of an individual's attitudes toward ambiguity or ambiguity tolerance. The development of natural language processing technology has made it possible to measure mental states and reactions through open-ended questions, rather than predefined numerical rating scales, which have traditionally been the dominant method in psychological research. This study presented three ambiguity-related situations and responses collected online from 591 participants in an open-ended format. After the analysis with bidirectional encoder representations from transformers, correlations were calculated using scores from the numerical evaluation by conventional questionnaire, and a significant moderate positive correlation was found. Therefore, this study found that attitudes toward ambiguity can be measured using an open-ended response method of reporting everyday life states. It is a novel methodology that can be expanded to other scales in psychology and can potentially be used in educational and clinical situations where participants can be asked to respond with minimal burden.

Language-based EMA assessments help understand problematic alcohol consumption

BACKGROUND

Unhealthy alcohol consumption is a severe public health problem. But low to moderate alcohol consumption is associated with high subjective well-being, possibly because alcohol is commonly consumed socially together with friends, who often are important for subjective well-being. Disentangling the health and social complexities of alcohol behavior has been difficult using traditional rating scales with cross-section designs. We aim to better understand these complexities by examining individuals' everyday affective subjective well-being language, in addition to rating scales, and via both between- and within-person designs across multiple weeks.

METHOD

We used daily language and ecological momentary assessment on 908 US restaurant workers (12692 days) over two-week intervals. Participants were asked up to three times a day to "describe your current feelings", rate their emotions, and report their alcohol behavior in the past 24 hours, including if they were drinking alone or with others.

RESULTS

Both between and within individuals, language-based subjective well-being predicted alcohol behavior more accurately than corresponding rating scales. Individuals self-reported being happier on days when drinking more, with language characteristic of these days predominantly describing socializing with friends. Between individuals (over several weeks), subjective well-being correlated much more negatively with drinking alone (r = -.29) than it did with total drinking (r = -.10). Aligned with this, people who drank more alone generally described their feelings as sad, stressed and anxious and drinking alone days related to nervous and annoyed language as well as a lower reported subjective well-being.

CONCLUSIONS

Individuals' daily subjective well-being, as measured via language, in part, explained the social aspects of alcohol drinking. Further, being alone explained this relationship, such that drinking alone was associated with lower subjective well-being.

Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment

In this narrative review, we survey recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models to be poised for changing standardized psychological assessment. Artificial intelligence has been undergoing a purported "paradigm shift" initiated by new machine learning models, large language models (e.g., BERT, LAMMA, and that behind ChatGPT). These models have led to unprecedented accuracy over most computerized language processing tasks, from web searches to automatic machine translation and question answering, while their dialogue-based forms, like ChatGPT have captured the interest of over a million users. The success of the large language model is mostly attributed to its capability to numerically represent words in their context, long a weakness of previous attempts to automate psychological assessment from language. While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.

The Cantril Ladder elicits thoughts about power and wealth

The Cantril Ladder is among the most widely administered subjective well-being measures; every year, it is collected in 140+ countries in the Gallup World Poll and reported in the World Happiness Report. The measure asks respondents to evaluate their lives on a ladder from worst (bottom) to best (top). Prior work found Cantril Ladder scores sensitive to social comparison and to reflect one's relative position in the income distribution. To understand this, we explored how respondents interpret the Cantril Ladder. We analyzed word responses from 1581 UK adults and tested the impact of the (a) ladder imagery, (b) scale anchors of worst to best possible life, and c) bottom to top. Using three language analysis techniques (dictionary, topic, and word embeddings), we found that the Cantril Ladder framing emphasizes power and wealth over broader well-being and relationship concepts in comparison to the other study conditions. Further, altering the framings increased preferred scale levels from 8.4 to 8.9 (Cohen's d = 0.36). Introducing harmony as an anchor yielded the strongest divergence from the Cantril Ladder, reducing mentions of power and wealth topics the most (Cohen's d = -0.76). Our findings refine the understanding of historical Cantril Ladder data and may help guide the future evolution of well-being metrics and guidelines.

LEIA: Linguistic Embeddings for the Identification of Affect

The wealth of text data generated by social media has enabled new kinds of analysis of emotions with language models. These models are often trained on small and costly datasets of text annotations produced by readers who guess the emotions expressed by others in social media posts. This affects the quality of emotion identification methods due to training data size limitations and noise in the production of labels used in model development. We present LEIA, a model for emotion identification in text that has been trained on a dataset of more than 6 million posts with self-annotated emotion labels for happiness, affection, sadness, anger, and fear. LEIA is based on a word masking method that enhances the learning of emotion words during model pre-training. LEIA achieves macro-F1 values of approximately 73 on three in-domain test datasets, outperforming other supervised and unsupervised methods in a strong benchmark that shows that LEIA generalizes across posts, users, and time periods. We further perform an out-of-domain evaluation on five different datasets of social media and other sources, showing LEIA's robust performance across media, data collection methods, and annotation schemes. Our results show that LEIA generalizes its classification of anger, happiness, and sadness beyond the domain it was trained on. LEIA can be applied in future research to provide better identification of emotions in text from the perspective of the writer.

Online reading habits can reveal personality traits: towards detecting psychological microtargeting

Building on big data from Reddit, we generated two computational text models: (i) Predicting the personality of users from the text they have written and (ii) predicting the personality of users based on the text they have consumed. The second model is novel and without precedent in the literature. We recruited active Reddit users (N = 1 , 105 ) of fiction-writing communities. The participants completed a Big Five personality questionnaire and consented for their Reddit activity to be scraped and used to create a machine learning model. We trained an natural language processing model [Bidirectional Encoder Representations from Transformers (BERT)], predicting personality from produced text (average performance: r = 0.33 ). We then applied this model to a new set of Reddit users (N = 10 , 050 ), predicted their personality based on their produced text, and trained a second BERT model to predict their predicted-personality scores based on consumed text (average performance: r = 0.13 ). By doing so, we provide the first glimpse into the linguistic markers of personality-congruent consumed content.

Modern views of machine learning for precision psychiatry

In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.

Three families of automated text analysis

Since the beginning of this millennium, data in the form of human-generated text in a machine-readable format has become increasingly available to social scientists, presenting a unique window into social life. However, harnessing vast quantities of this highly unstructured data in a systematic way presents a unique combination of analytical and methodological challenges. Luckily, our understanding of how to overcome these challenges has also developed greatly over this same period. In this article, I present a novel typology of the methods social scientists have used to analyze text data at scale in the interest of testing and developing social theory. I describe three "families" of methods: analyses of (1) term frequency, (2) document structure, and (3) semantic similarity. For each family of methods, I discuss their logical and statistical foundations, analytical strengths and weaknesses, as well as prominent variants and applications.