Psychometric and Machine Learning Approaches to Reduce the Length of Scales

Artificial Neural Networks for Short-Form Development of Psychometric Tests: A Study on Synthetic Populations Using Autoencoders

Short-form development is an important topic in psychometric research, which requires researchers to face methodological choices at different steps. The statistical techniques traditionally used for shortening tests, which belong to the so-called exploratory model, make assumptions not always verified in psychological data. This article proposes a machine learning-based autonomous procedure for short-form development that combines explanatory and predictive techniques in an integrative approach. The study investigates the item-selection performance of two autoencoders: a particular type of artificial neural network that is comparable to principal component analysis. The procedure is tested on artificial data simulated from a factor-based population and is compared with existent computational approaches to develop short forms. Autoencoders require mild assumptions on data characteristics and provide a method to predict long-form items' responses from the short form. Indeed, results show that they can help the researcher to develop a short form by automatically selecting a subset of items that better reconstruct the original item's responses and that preserve the internal structure of the long-form.

The Perceived Stress Scale 2&2: a two-factorial German short version of the Perceived Stress Scale

Background

Stress is among the leading causes for diseases. The assessment of subjectively perceived stress is essential for resilience research. While the Perceived Stress Scale (PSS) is a widely used questionnaire, a German short version of the scale is not yet available. In the current study, we developed such a short version using a machine learning approach for item reduction to facilitate the simultaneous optimization of multiple psychometric criteria.

Method

We recruited 1,437 participants from an online panel, who completed the German long version of the PSS along with measures of mental health and resilience. An ant-colony-optimization algorithm was used to select items, taking reliability, and construct validity into account. Findings on validity were visualized by psychological network models.

Results

We replicated a bifactor structure for the long version of the PSS and derived a two-factor German short version of the PSS with four items, the PSS-2&2. Its factors helplessness and self-efficacy showed differential associations with mental health indicators and resilience-related factors, with helplessness being mainly linked to mental distress.

Conclusion

The valid and economic short version of the PSS lends itself to be used in future resilience research. Our findings highlight the importance of the two-factor structure of the PSS short versions and challenge the validity of commonly used one-factor models. In cases where the general stress factor is of interest, researchers should use the longer versions of the PSS that allow for the interpretation of total scores, while the PSS-2&2 allows of an economic assessment of the PSS factors helplessness and self-efficacy.

Machine learning-decision tree classifiers in psychiatric assessment: An application to the diagnosis of major depressive disorder

This work illustrates the advantages of using machine learning classifiers in psychiatric assessment. Machine learning-decision trees (ML-DTs) represent a new approach to scoring and interpreting psychodiagnostic test data that allows for increasing assessment accuracy and efficiency. The approach is outlined in an easy yet detailed way, and its application is illustrated on real psychodiagnostic test data. Specifically, cross-sectional data concerning nonclinical and clinical Japanese populations were taken from a panel registered with an internet survey company. Responses to the Patient Health Questionnaire-9 (PHQ-9) underwent receiver operating characteristic (ROC) curve, DSM algorithm, and ML-DT analyses. The results showed greater diagnostic accuracy for ML-DT (0.71-0.75) compared with the DSM algorithm (0.69) and ROC curves (0.70-0.71). Moreover, ML-DT enabled classifying participants as having or not having a diagnosis of depression using, on average, the information from 2.99 out of 9 items (SD = 1.35). The application showed that ML-DTs can provide information of high clinical value to integrate traditional psychometric methods. The resulting assessments are informative, accurate, and efficient.

Machine learning-based brief version of the Caregiver-Teacher Report Form for preschoolers

BACKGROUND

The Caregiver-Teacher Report Form of the Child Behavior Checklist for Ages 1½-5 (C-TRF) is a widely used checklist to identify emotional and behavioral problems in preschoolers. However, the 100-item C-TRF restricts its utility.

AIMS

This study aimed to develop a machine learning-based short-form of the C-TRF (C-TRF-ML).

METHODS AND PROCEDURES

Three steps were executed. First, we split the data into three datasets in a ratio of 3:1:1 for training, validation, and cross-validation, respectively. Second, we selected a shortened item set and trained a scoring algorithm using joint learning for classification and regression using the training dataset. Then, we evaluated the similarity of scores between the C-TRF-ML and the C-TRF by r-squared and weighted kappa values using the validation dataset. Third, we cross-validated the C-TRF-ML by calculating the r-squared and weighted kappa values using the cross-validation dataset.

OUTCOMES AND RESULTS

Data of 363 children were analyzed. Thirty-six items of the C-TRF were retained. The r-squared values of C-TRF-ML scores were 0.86-0.96 in the cross-validation dataset. Weighted kappa values of the syndrome/problem grading were 0.72-0.94 in the cross-validation dataset.

CONCLUSIONS AND IMPLICATIONS

The C-TRF-ML had about 60 % fewer items than the C-TRF but yielded comparable scores with the C-TRF.

Development of a 13-item Short Form for Fugl-Meyer Assessment of Upper Extremity Scale Using a Machine Learning Approach

OBJECTIVE

To develop and validate a short form of the Fugl-Meyer Assessment of Upper Extremity Scale (FMA-UE) using a machine learning approach (FMA-UE-ML). In addition, scores of items not included in the FMA-UE-ML were predicted.

DESIGN

Secondary data from a previous study, which assessed individuals post-stroke using the FMA-UE at 4 time points: 5-30 days post-stroke screen, 2-month post-stroke baseline assessment, 6-month post-stroke assessment, and 12-month post-stroke assessment.

SETTING

Rehabilitation units in hospitals.

PARTICIPANTS

A total of 408 individuals post-stroke (N=408).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

The 30-item FMA-UE.

RESULTS

We established 29 candidate versions of the FMA-UE-ML with different numbers of items, from 1 to 29, and examined their concurrent validity and responsiveness. We found that the responsiveness of the candidate versions obviously declined when the number of items was less than 13. Thus, the 13-item version was selected as the FMA-UE-ML. The concurrent validity was good (intra-class correlation coefficients ≥0.99). The standardized response means of the FMA-UE-ML and FMA-UE were 0.54-0.88 and 0.52-0.91, respectively. The Pearson's rs between the change scores of the FMA-UE-ML and those of the FMA-UE were 0.96-0.98. The predicted item scores had acceptable to good accuracy (Kappa=0.50-0.92).

CONCLUSIONS

The FMA-UE-ML seems a promising short form to improve administrative efficiency while retaining good concurrent validity and responsiveness. In addition, the FAM-UE-ML can provide all item scores of the FMA-UE for users.

A Machine Learning Approach to Assess Differential Item Functioning of the KINDL Quality of Life Questionnaire Across Children with and Without ADHD

This study aimed to investigate differential item functioning (DIF) of the child and parent reports of the KINDL measure across children with and without Attention-deficit/hyperactivity disorder (ADHD). The sample included 122 children with ADHD and 1086 healthy peers, alongside 127 and 1061 of their parents, respectively. The generalized partial credit model with lasso penalization, as a machine learning method, was used to assess DIF of the KINDL across the two groups. The findings showed that three out of 24 items of the child reports and seven out of 24 items of the parent reports of the KINDL exhibited DIF between children with and without ADHD. Accordingly, Iranian children with and without ADHD along with their parents perceive almost all items in the KINDL similarly. Hence, the observed difference in quality of life scores between children with and without ADHD is a real difference and not a reflection of measurement bias.

Automatic Decision-Making Style Recognition Method Using Kinect Technology

In recent years, somatosensory interaction technology, represented by Microsoft's Kinect hardware platform, has been widely used in various fields, such as entertainment, education, and medicine. Kinect technology can easily capture and record behavioral data, which provides new opportunities for behavioral and psychological correlation analysis research. In this paper, an automatic decision-style recognition method is proposed. Experiments involving 240 subjects were conducted to obtain face data and individual decision-making style score. The face data was obtained using the Kinect camera, and the decision-style score were obtained via a questionnaire. To realize automatic recognition of an individual decision-making style, machine learning was employed to establish the mapping relationship between the face data and a scaled evaluation of the decision-making style score. This study adopts a variety of classical machine learning algorithms, including Linear regression, Support vector machine regression, Ridge regression, and Bayesian ridge regression. The experimental results show that the linear regression model returns the best results. The correlation coefficient between the linear regression model evaluation results and the scale evaluation results was 0.6, which represents a medium and higher correlation. The results verify the feasibility of automatic decision-making style recognition method based on facial analysis.

What Is Slowing Us Down? Six Challenges to Accelerating Advances in Health Behavior Change

BACKGROUND

Accelerating advances in health behavior change requires releasing the brake, as well as applying the throttle. This paper discusses six challenges or "brakes" that have slowed progress.

PURPOSE/METHODS/RESULTS

We engage with six issues that limit investigators' ability to delineate and test the strategy-target and target-behavior relations that underlie effective interventions according to the experimental medicine approach. We discuss the need for guidance on how to identify the relevant mechanism of action (target) in an intervention and whether a periodic table of health behavior constructs might aid investigators. Experimental and correlational analyses (prospective surveys and behavior change techniques) have been used to test the validity of targets, and we present evidence that there is little agreement among the findings from different research designs. Whereas target engagement is typically analyzed in terms of increasing scores on constructs that impel behavior change, we discuss the role of impeding targets and the benefits of adopting a broader construal of potential targets and approaches to engagement. There is presently a paucity of competitive tests regarding which strategies best engage targets and we discuss empirical criteria and conceptual developments that could enhance the evidence base. Finally, we highlight the need to take "context" or conditional intervention effects more seriously by leveraging the interplay between questions about why interventions work and questions about when and for whom they work.

CONCLUSION

Candid appraisal of the challenges facing research on health behavior change can generate new opportunities for theoretical development and offer direction and cumulative impetus for empirical work.

Examining the conceptual and empirical distinctiveness of Agreeableness and "dark" personality items

OBJECTIVE

A growing research literature has focused on what have been termed "dark" personality traits/constructs. More recently, the "dark factor" of personality has been proposed as a unifying framework for this research. To date, little work has rigorously investigated whether the traits/constructs investigated in the context of the dark factor can be captured by existing models of normative personality, namely Agreeableness from the Five-factor Model. Thus, the "dark factor" may be an instance of the "jangle" fallacy, where two constructs with different names are in fact the same construct.

METHOD

We used a preregistered approach that made use of bass-ackward factor analysis, structural equation modeling, and nomological network analysis to investigate the distinction between the D factor and Agreeableness.

RESULTS

Agreeableness and the D factor were similar in their coverage of antagonistic personality content, strongly negatively related (latent r = -.90), and showed near perfect profile dissimilarity (r_ICC  = -.99).

CONCLUSIONS

The results suggested that the D factor can be understood as the opposite pole of Agreeableness (i.e., antagonism) and not as a distinct construct. We discuss the implications for researchers interested in continuing to advance the study of antagonistic personality traits.