Random intercept EFA of personality scales

Psychometric benefits of self-chosen rating scales over given rating scales

Rating scales are susceptible to response styles that undermine the scale quality. Optimizing a rating scale can tailor it to individuals' cognitive abilities, thereby preventing the occurrence of response styles related to a suboptimal response format. However, the discrimination ability of individuals in a sample may vary, suggesting that different rating scales may be appropriate for different individuals. This study aims to examine (1) whether response styles can be avoided when individuals are allowed to choose a rating scale and (2) whether the psychometric properties of self-chosen rating scales improve compared to given rating scales. To address these objectives, data from the flourishing scale were used as an illustrative example. MTurk workers from Amazon's Mechanical Turk platform (N = 7042) completed an eight-item flourishing scale twice: (1) using a randomly assigned four-, six-, or 11-point rating scale, and (2) using a self-chosen rating scale. Applying the restrictive mixed generalized partial credit model (rmGPCM) allowed examination of category use across the conditions. Correlations with external variables were calculated to assess the effects of the rating scales on criterion validity. The results revealed consistent use of self-chosen rating scales, with approximately equal proportions of the three response styles. Ordinary response behavior was observed in 55-58% of individuals, which was an increase of 12-15% compared to assigned rating scales. The self-chosen rating scales also exhibited superior psychometric properties. The implications of these findings are discussed.

Dimensionality assessment in the presence of wording effects: A network psychometric and factorial approach

This study proposes a procedure for substantive dimensionality estimation in the presence of wording effects, the inconsistent response to regular and reversed self-report items. The procedure developed consists of subtracting an approximate estimate of the wording effects variance from the sample correlation matrix and then estimating the substantive dimensionality on the residual correlation matrix. This is achieved by estimating a random intercept factor with unit loadings for all the regular and unrecoded reversed items. The accuracy of the procedure was evaluated through an extensive simulation study that manipulated nine relevant variables and employed the exploratory graph analysis (EGA) and parallel analysis (PA) retention methods. The results indicated that combining the proposed procedure with EGA or PA achieved high accuracy in estimating the substantive latent dimensionality, but that EGA was superior. Additionally, the present findings shed light on the complex ways that wording effects impact the dimensionality estimates when the response bias in the data is ignored. A tutorial on substantive dimensionality estimation with the R package EGAnet is offered, as well as practical guidelines for applied researchers.

Distributional effects of parental time investments on children's socioemotional skills and nutritional health

Parental behavior is paramount to child health and skill formation, explaining a significant portion of differences in developmental outcomes. However, little is known regarding the distributional effects of parental time allocation at different levels of children's outcomes. I use a national administrative dataset of Chilean pre-school students to the estimate production functions for socioemotional development and body mass index z-scores at every decile of the distribution at baseline. Modest average effects conceal significant heterogeneity on the returns to parental time investments. Children in the bottom of the socioemotional development distribution could gain up to 0.4 standard deviations for a one standard deviation increase in time investments. A similar increase can lead to a reduction of 0.8 standard deviations in body mass index among severely obese students. Evidence reveals that children with high developmental scores are unlikely to benefit from additional parenting time.

From the BFI-44 to BFI-20: Psychometric Properties of the Short Form of the Big Five Inventory

The Big Five Inventory (BFI-44) has been a useful tool for researchers for over three decades. However, the contemporary way of life has created the need for abbreviated versions of psychological instruments. We derived the number of items from the BFI-44 questionnaire in order to create a short form of the questionnaire (BFI-20). In the first study (N = 1350, 82.4% females, aged 18-60), using a range of criteria, we identified 20 items (four for each of the Big Five traits) that most optimally represent each dimension. The five-factor structure was mostly replicated in the second (N = 215, 65.1% females, aged 18-65) and third study (N = 263, 83.7% females, aged 18-42). The BFI-20 demonstrated satisfactory reliability, representativeness, homogeneity, and part-whole convergence. Despite mild attenuation, most BFI-20 correlations with schizotypy, satisfaction with life, and positive orientation remained in the same range compared to the BFI-44. The Agreeableness domain was shown to be the most challenging for capturing with four items. We discuss the advantages of our BFI-20 compared to the other two 20-item versions. In sum, we can recommend the use of this BFI-20 version as a time-efficient, satisfactory reliable, and representative questionnaire.

Inventory of Interpersonal Problems-Personality Disorders: adaptation to Brazil and test of a bifactor model

OBJECTIVE

The Inventory of Interpersonal Problems-Personality Disorders (IIP-PD-47) has a controversial factor structure, as some studies have provided support for 5 correlated factors, and others have suggested the existence of a general second-order dimension. One approach of data modelling that reconciles multidimensionality and the existence of a general factor is the bifactor analysis. We used unrestricted exploratory-confirmatory bifactor modelling to validate the Brazilian version of the IIP-PD-47.

METHODS

The sample consisted of 1,091 subjects aged 18-64 years who answered the IIP-PD-47 and a collateral measure of pathological traits, the Dimensional Clinical Personality Inventory 2 (IDCP-2).

RESULTS

After testing many candidate models, our data were best represented by a bifactor model with one general factor and five specific uncorrelated factors. Nevertheless, a closer inspection of the discriminant validity of each IIP-PD-47 factor revealed strong support for the general factor and a factor capturing aggressive behaviours, but less support for the additional four specific factors.

CONCLUSIONS

The theoretical and practical implications of these findings are discussed, and some recommendations are offered about the need for controlling response styles when assessing PD traits via self-report inventories. Our findings indicate that the Brazilian version of IIP-PD has promising psychometric properties.

You Are What You Eat and So Is Our Planet: Identifying Dietary Groups Based on Personality and Environmentalism

Behavioral change interventions promoting the reduction of animal product consumption are valuable tools to improve ecological sustainability as well as public health and help the mitigation of climate change. Recent findings revealed improved efficacy of interventions targeted at barriers (e.g., self-efficacy) of three different types of meat consumers over non-targeted interventions (e.g., completion of unrelated surveys). However, such interventions have yet to factor in the role of individual differences in personality. Therefore, in a first step, we performed segmentation analysis on barriers and benefits of reducing animal product consumption (e.g., meat attachment, environmentalism) with the inclusion of personality. In an online sample of N=1135 participants, latent profile analysis revealed five distinct dietary groups: “plant-based eaters”, “meat-reducers”, “medium-hindrance meat eaters”, “medium strong-hindrance meat eaters, and “strong-hindrance meat eaters”, based on inhibitors and facilitators of meat reduction. Groups differed in terms of consumption of different animal products (η2=0.08 to η2=0.80) as well as the Big Five (η2=0.08 to η2=0.80) and Dark Triad (η2=0.08 to η2=0.80). Strong-hindrance meat eaters were characterized by low Conscientiousness, Agreeableness, and Openness as well as high dark trait expression, implying new targets for future intervention design.

How a Few Inconsistent Respondents Can Confound the Structure of Personality Survey Data

Abstract. In survey data, inconsistent responses due to careless/insufficient effort (C/IE) can lead to problems of replicability and validity. However, data cleaning prior to the main analyses is not yet a standard practice. We investigated the effect of C/IE responses on the structure of personality survey data. For this purpose, we analyzed the structure of the Core-Self Evaluations scale (CSE-S), including the detection of aberrant responses in the study design. While the original theoretical model of the CSE-S assumes that the construct is unidimensional ( Judge et al., 2003 ), recent studies have argued for a multidimensional solution (positive CSE and negative CSE). We hypothesized that this multidimensionality is not substantive but a result of the tendency of C/IE data to generate spurious dimensions. We estimated the confirmatory models before and after removing highly inconsistent response vectors in two independent samples (6% and 4.7%). The analysis of the raw samples clearly favored retaining the two-dimensional model. In contrast, the analysis of the clean datasets suggested the retention of a single factor. A mere 6% C/IE response rate showed enough power to confound the results of the factor analysis. This result suggests that the factor structure of positive and negative CSE factors is spurious, resulting from uncontrolled wording variance produced by a limited proportion of highly inconsistent response vectors. We encourage researchers to include screening for inconsistent responses in their research designs.

The Norwegian Adaptation of the Big Five Inventory-2

Two studies were conducted to assess the psychometric properties of scores from the Norwegian adaptation of the Big Five Inventory-2 (BFI-2). In Study 1, the BFI-2 was translated to Norwegian and the scores from a convenience sample (N = 606) demonstrated good psychometric properties. BFI-2 scores from subsamples correlated in expected ways with self- and other ratings of the Big Five, and with self-ratings of empathic concern and perspective taking. In Study 2, after some minor improvements in translation, the psychometric properties of BFI-2 scores were assessed in a new sample (N = 409). Results from random intercept EFA of scores supported the proposed model. The psychometric properties of two shorter versions of the inventory, the BFI-2-S and BFI-2-XS, were also examined. Overall, the results suggest that the Norwegian adaptation of the BFI-2 provide reliable and valid scores.

Escala Brasileira de Burnout (EBB): Estrutura Interna e Controle de Aquiescência

Resumo Escalas de autorrelato no contexto clínico são suscetíveis a vieses de resposta. A aquiescência pode superestimar uma dimensão geral em estudos fatoriais com esses instrumentos e enviesar os escores dos participantes. A maior parte dos instrumentos clínicos de autorrelato não dispõe de ferramentas para controle de vieses. O objetivo desta pesquisa foi avaliar a estrutura interna com o controle da aquiescência, da Escala Brasileira de Burnout (EBB). Fizeram parte da amostra 854 trabalhadores. Após o controle da aquiescência, a melhor solução foi um modelo bifactor com dois fatores teóricos (Exaustão Emocional e Frustração Profissional, e Despersonalização/Distanciamento) e um fator geral de viés. A EBB pode ser útil em pesquisas e no rastreio dos sintomas de burnout.

Refining the Short Social Dominance Orientation scale (SSDO): A validation in seven European countries

People and societies differ in their tendency to justify inequalities and group hierarchies, a motivation that has been labelled social dominance orientation (SDO). In order to efficiently measure this motivational tendency, Pratto and colleagues (2013, https://doi.org/10.1177/1948550612473663) proposed the four-item Short Social Dominance Orientation (SSDO) scale. The present study comprehensively assesses the SSDO scale’s psychometric properties in seven European countries (Austria, Czech Republic, Germany, France, Hungary, Italy, and Poland). Using large and diverse samples from these countries, we propose a measurement model to assess the scale’s structural validity and we assess measurement invariance (MI), reliability, and convergent validity. Results suggest that the scale is sufficiently reliable, shows theoretically predictable and consistent correlations with external criteria across countries, it exhibits at least partial scalar and partial uniqueness MI across the seven countries and full MI across gender. These findings offer support for the psychometric quality of the SSDO scale and its usefulness for cross-national and multi-topic social surveys.

Modeling Wording Effects Does Not Help in Recovering Uncontaminated Person Scores: A Systematic Evaluation With Random Intercept Item Factor Analysis

The item wording (or keying) effect consists of logically inconsistent answers to positively and negatively worded items that tap into similar (but polarly opposite) content. Previous research has shown that this effect can be successfully modeled through the random intercept item factor analysis (RIIFA) model, as evidenced by the improvements in the model fit in comparison to models that only contain substantive factors. However, little is known regarding the capability of this model in recovering the uncontaminated person scores. To address this issue, the study analyzes the performance of the RIIFA approach across three types of wording effects proposed in the literature: carelessness, item verification difficulty, and acquiescence. In the context of unidimensional substantive models, four independent variables were manipulated, using Monte Carlo methods: type of wording effect, amount of wording effect, sample size, and test length. The results corroborated previous findings by showing that the RIIFA models were consistently able to account for the variance in the data, attaining an excellent fit regardless of the amount of bias. Conversely, the models without the RIIFA factor produced increasingly a poorer fit with greater amounts of wording effects. Surprisingly, however, the RIIFA models were not able to better estimate the uncontaminated person scores for any type of wording effect in comparison to the substantive unidimensional models. The simulation results were then corroborated with an empirical dataset, examining the relationship between learning strategies and personality with grade point average in undergraduate studies. The apparently paradoxical findings regarding the model fit and the recovery of the person scores are explained, considering the properties of the factor models examined.

Inventory of Interpersonal Problems-Personality Disorders: adaptation to Brazil and test of a bifactor model

OBJECTIVE

The Inventory of Interpersonal Problems-Personality Disorders (IIP-PD-47) has a controversial factor structure, as some studies have provided support for 5 correlated factors, and others have suggested the existence of a general second-order dimension. One approach of data modelling that reconciles multidimensionality and the existence of a general factor is the bifactor analysis. We used unrestricted exploratory-confirmatory bifactor modelling to validate the Brazilian version of the IIP-PD-47.

METHODS

The sample consisted of 1,091 subjects aged 18-64 years who answered the IIP-PD-47 and a collateral measure of pathological traits, the Dimensional Clinical Personality Inventory 2 (IDCP-2).

RESULTS

After testing many candidate models, our data were best represented by a bifactor model with one general factor and five specific uncorrelated factors. Nevertheless, a closer inspection of the discriminant validity of each IIP-PD-47 factor revealed strong support for the general factor and a factor capturing aggressive behaviours, but less support for the additional four specific factors.

CONCLUSIONS

The theoretical and practical implications of these findings are discussed, and some recommendations are offered about the need for controlling response styles when assessing PD traits via self-report inventories. Our findings indicate that the Brazilian version of IIP-PD has promising psychometric properties.

The Big Five Inventory-2 in China: A Comprehensive Psychometric Evaluation in Four Diverse Samples

The Big Five Inventory-2 (BFI-2) has received wide recognition since its publication because it strikes a good balance between content coverage and brevity. The current study translated the BFI-2 into Chinese, evaluated its psychometric properties in four diverse Chinese samples (college students, adult employees, adults treated for substance use, and adolescents), and compared its factor structure with those obtained from two U.S. samples. Across two studies, the Chinese BFI-2 demonstrated good reliability (Cronbach's α and test-retest reliability), structural validity, convergent/discriminant validity, and criterion-related validity at the domain level. At lower levels of analyses, some facets and negatively worded items functioned better among participants with higher than those with lower education levels. Implications, limitations, and future directions are discussed.

Unraveling the Relationship Between Teacher-Assigned Grades, Student Personality, and Standardized Test Scores

The Big Five personality traits play a major role in student achievement. As such, there is consistent evidence that students that are more conscientious receive better teacher-assigned grades in secondary school. However, research often does not support the claim that students that are more conscientious similarly achieve higher scores in domain-specific standardized achievement tests. Based on the Invest-and-Accrue Model, we argue that conscientiousness explains to some extent why certain students receive better grades despite similar academic accomplishments (i.e., achieving similar scores in domain-specific standardized achievement tests). Therefore, the present study examines to what extent the relationship between student personality and teacher-assigned grades consists of direct as opposed to indirect associations (via subject-specific standardized test scores). We used a representative sample of 14,710 ninth-grade students to estimate these direct and indirect pathways in mathematics and German. Structural equation models showed that test scores explained between 8 and 11% of the variance in teacher-assigned grades in mathematics and German. The Big Five personality traits in students additionally explained between 8 and 10% of the variance in grades. Finally, the personality-grade relationship consisted of direct (0.02 | β| ≤ 0.27) and indirect associations via test scores (0.01 | β| ≤ 0.07). Conscientiousness explained discrepancies between teacher-assigned grades and students' scores in domain-specific standardized tests to a greater extent than any of the other Big Five personality traits. Our findings suggest that students that are more conscientious may invest more effort to accomplish classroom goals, but fall short of mastery.

Are some students graded more appropriately than others? Student characteristics as moderators of the relationships between teacher-assigned grades and test scores in mathematics

BACKGROUND

Building on the Realistic Accuracy Model, this paper explores whether it is easier for teachers to assess the achievement of some students than others. Accordingly, we suggest that certain individual characteristics of students, such as extraversion, academic self-efficacy, and conscientiousness, may guide teachers' evaluations of student achievement, resulting in more appropriate judgements and a stronger alignment of assigned grades with students' actual achievement level (as measured using standardized tests).

AIMS

We examine whether extraversion, academic self-efficacy, and conscientiousness moderate the relations between teacher-assigned grades and students' standardized test scores in mathematics.

SAMPLE

This study uses a representative sample of N = 5,919 seventh-grade students in Germany (48.8% girls; mean age: M = 12.5, SD = 0.62) who participated in a national, large-scale assessment focusing on students' academic development.

METHODS

We specified structural equation models to examine the inter-relations of teacher-assigned grades with students' standardized test scores in mathematics, Big Five personality traits, and academic self-efficacy, while controlling for students' socioeconomic status, gender, and age.

RESULTS

The correlation between teacher-assigned grades and standardized test scores in mathematics was r = .40. Teacher-assigned grades more closely related to standardized test scores when students reported higher levels of conscientiousness (β = .05, p = .002). Students' extraversion and academic self-efficacy did not moderate the relationship between teacher-assigned grades and standardized test scores.

CONCLUSIONS

Our findings indicate that students' conscientiousness is a personality trait that seems to be important when it comes to how closely mathematics teachers align their grades to standardized test scores.

Modelling the incremental value of personality facets: the domains-incremental facets-acquiescence bifactor showmodel

Personality can be described at different levels of abstraction. Whereas the Big Five domains are the dominant level of analysis, several researchers have called for more fine-grained approaches, such as facet-level analysis. Personality facets allow more comprehensive descriptions, more accurate predictions of outcomes, and a better understanding of the mechanisms underlying trait–outcome relationships. However, several methodological issues plague existing evidence on the added value of facet-level descriptions: Manifest facet scale scores differ with respect to their reliability, domain-level variance (variance that is due to the domain factor) and incremental facet-level variance (variance that is specific to a facet and not shared with the other facets). Moreover, manifest scale scores overlap substantially, which affects associations with criterion variables. We suggest a structural equation modelling approach that allows domain-level variance to be separated from incremental facet-level variance. We analysed data from a heterogeneous sample of adults in the USA (N = 1193) who completed the 60-item Big Five Inventory-2. The results illustrate how the variance of manifest personality items and scale scores can be decomposed into domain-level and incremental facet-level variance. The association with criterion variables (educational attainment, income, health, and life satisfaction) further demonstrates the incremental predictive power of personality facets.

O viés da aquiescência e a estrutura fatorial do Dirty Dozen e do Short Dark Triad

A tríade sombria—maquiavelismo, narcisismo e psicopatia—tem sido tradicionalmente avaliada via inventários de autorrelato. Entretanto, instrumentos de autorrelato podem estar suscetíveis à aquiescência ou tendência a concordar com itens mesmo quando possuem conteúdo antagônico. O presente estudo teve por objetivo investigar a dimensionalidade de dois populares instrumentos de autorrelato da tríade, o Dirty Dozen e o Short Dark Triad, verificando a interferência do viés da aquiescência na estrutura fatorial de cada medida. Participaram do estudo 449 universitários (média de idades = 23,47; DP = 6,76; 64% mulheres). O controle da aquiescência produziu soluções fatoriais mais interpretáveis do que a tradicional modelagem bi-factor, amplamente utilizada na área. Os achados também revelam fragilidades na composição teórica de ambos os instrumentos. Argumenta-se que a economia das medidas breves da “tríade sombria” não deve vir desencontrada de uma estrutura fatorial que reflita o conhecimento atual sobre a multidimensionalidade desses traços. 

Development of the Inventory of Supporting for Socio-Emotional Skills, evidence of internal structure controlling for acquiescence

Abstract This study aims to develop the Inventory of Supporting for Socio-Emotional Skills, which is an instrument for assessing the support for socio-emotional development provided by teachers, rated by students. It also aims to gather validity evidence of the internal structure of Inventory of Supporting for Socio-Emotional Skills and to control acquiescence. We elaborated the items, which content were evaluated by six experts. For the internal structure, a sample composed of elementary and high-school students (N = 443) was used to answer the instrument. The instrument presented good psychometric parameters after controlling for acquiescence using the Random Intercept Model. The final structure was set as unidimensional. We suggest controlling acquiescence to avoid biased scores. In addition, we make an inventory available that can be helpful for socio-emotional skills development.

Validation of the Short and Extra-Short Forms of the Big Five Inventory-2 (BFI-2) and Their German Adaptations

Abstract. The present study investigates the validity and utility of the German adaptations of the two short forms of the Big Five Inventory-2 (BFI-2), the 30-item BFI-2-S, and the 15-item BFI-2-XS, developed by Soto and John (2017b) . Both scales assess the Big Five domains. The BFI-2-S allows, in addition, the brief assessment of three facets per domain. Based on a large and heterogeneous sample, we show that the psychometric properties of these adapted short scales are consistent with those of the Anglo-American source versions, and we demonstrate substantial convergence between the adaptations and the source versions. Extending the original scale development study, we demonstrate high retest stability of the scales and their facets. Our results clearly indicate the construct and criterion validity of the two scales: Both show substantial convergence with the NEO-PI-R domain scales. Moreover, the distinctive correlation pattern found between the facets of the BFI-2 and the NEO-PI-R could be replicated for the facets of the BFI-2-S. Furthermore, we show that the domain scales of both instruments are associated in the hypothesized directions with important life outcomes, such as life satisfaction and intelligence, and that the facets of the BFI-2-S have incremental validity for predicting these outcomes.

Individual, situational, and cultural correlates of acquiescent responding: Towards a unified conceptual framework

Acquiescence ('yea-saying') can seriously harm the validity of self-report questionnaire data. Towards a better understanding of why some individuals and groups acquiesce more strongly than others do, we developed a unified conceptual framework of acquiescent responding. Our framework posits that acquiescent responding is a joint function of respondent characteristics (e.g. age, education, values), situational/survey characteristics (e.g., interview privacy, respondents' interest), and cultural characteristics (e.g., social norms, economic development). The framework posits two putative mechanisms through which these characteristics may relate to acquiescence: cognitive processing capacities and deferential communication styles. Multilevel analyses using data from 60 heterogeneous countries from the World Values Survey (N = 90,347) support our framework's proposition that acquiescence is a joint function of respondent, situational, and cultural characteristics. Acquiescence was higher among respondents who were older (over 55 years old), less educated, who valued deference (i.e., conformity and tradition), and, unexpectedly, were male. Interview privacy corresponded to lower acquiescence, but this association was small and vanished after including respondent characteristics. Unexpectedly, acquiescence was higher in interviewees who showed a stronger interest in the interview. Finally, acquiescence was considerably higher in countries with stronger social norms of deference. We discuss implications of these findings for the validity of research based on self-report data and delineate how our framework can guide future inquiries into acquiescent responding.

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies.

Measurement Invariance and Sex and Age Differences of the Big Five Inventory–2: Evidence From the Russian Version

The Big Five Inventory–2 (BFI-2) is a recently published 60-item questionnaire that measures personality traits within the five-factor model framework. An important aspect of the BFI-2 is that it measures the traits at both the domain and facet levels and also controls acquiescence bias via the balanced number of true- and false-keyed items across the domains and facets. The current research evaluates factorial measurement invariance of a Russian version of the BFI-2 across sex and age within samples of 1,024 university students (Study 1) and 1,029 Internet users (Study 2). Across these samples, men scored lower on the domains of negative emotionality and agreeableness and slightly higher on extraversion. Sex differences were also obtained on various facets. In the Internet sample, age correlated modestly with several Big Five domains in accordance with the well-documented maturity principle. The newly developed Russian version of BFI-2 showed good reliability and validity across both samples. Moreover, random intercept exploratory factor analyses showed that the BFI-2 displayed a hierarchical five-domain-15-facet structure that demonstrated strict measurement invariance across sex and age.

Balanced and positively worded personality short-forms: Mini-IPIP validity and cross-cultural invariance

Background

The Mini-IPIP scales (Donellan et al., 2006) are possibly one of the most commonly used short inventories for measuring the Big Five Factors of personality. In this study, we aimed to investigate the psychometric properties of two Mini-IPIP Spanish short forms, one balanced and one positively wording (PW).

Method

Two samples, one from native Spanish speakers and another from native English speakers, made up a total of 940 participants in this study. The short forms were translated and adapted based on international guidelines. Reliability (internal and composite) and validity analyses (construct ESEM, concurrent, predictive and cross-cultural invariance through multi-group factorial models) were performed.

Results

For both the balanced scale and the PW one, modeling a method factor was not relevant. The reliability and validity indices of both forms were according to theory and prior studies’ findings: (a) personality factors were medium-high related to affective factors; (b) personality factors were less related to life satisfaction than affective factors; (c) life satisfaction was medium-high related to affective factors; (d) neuroticism appeared mainly related to all criteria variables; and (e) an acceptable level of invariance was achieved with regard to the English version.

Discussion

This study contributes to research on personality assessment by providing the first evidence regarding the psychometric properties of a PW short measure. These results suggest that PW short scales of personality used after data screening techniques may be appropriate for future studies (e.g., cross-cultural, content validity).

The Structure of the Rosenberg Self-Esteem Scale

Abstract. The Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965 ) intends to measure a single dominant factor representing global self-esteem. However, several studies have identified some form of multidimensionality for the RSES. Therefore, we examined the factor structure of the RSES with a fixed-effects meta-analytic structural equation modeling approach including 113 independent samples (N = 140,671). A confirmatory bifactor model with specific factors for positively and negatively worded items and a general self-esteem factor fitted best. However, the general factor captured most of the explained common variance in the RSES, whereas the specific factors accounted for less than 15%. The general factor loadings were invariant across samples from the United States and other highly individualistic countries, but lower for less individualistic countries. Thus, although the RSES essentially represents a unidimensional scale, cross-cultural comparisons might not be justified because the cultural background of the respondents affects the interpretation of the items.

Measurement Equivalence of the Patient Reported Outcomes Measurement Information System® (PROMIS®) Applied Cognition - General Concerns, Short Forms in Ethnically Diverse Groups

AIMS

The goals of these analyses were to examine the psychometric properties and measurement equivalence of a self-reported cognition measure, the Patient Reported Outcome Measurement Information System^® (PROMIS^®) Applied Cognition - General Concerns short form. These items are also found in the PROMIS Cognitive Function (version 2) item bank. This scale consists of eight items related to subjective cognitive concerns. Differential item functioning (DIF) analyses of gender, education, race, age, and (Spanish) language were performed using an ethnically diverse sample (n = 5,477) of individuals with cancer. This is the first analysis examining DIF in this item set across ethnic and racial groups.

METHODS

DIF hypotheses were derived by asking content experts to indicate whether they posited DIF for each item and to specify the direction. The principal DIF analytic model was item response theory (IRT) using the graded response model for polytomous data, with accompanying Wald tests and measures of magnitude. Sensitivity analyses were conducted using ordinal logistic regression (OLR) with a latent conditioning variable. IRT-based reliability, precision and information indices were estimated.

RESULTS

DIF was identified consistently only for the item, brain not working as well as usual. After correction for multiple comparisons, this item showed significant DIF for both the primary and sensitivity analyses. Black respondents and Hispanics in comparison to White non-Hispanic respondents evidenced a lower conditional probability of endorsing the item, brain not working as well as usual. The same pattern was observed for the education grouping variable: as compared to those with a graduate degree, conditioning on overall level of subjective cognitive concerns, those with less than high school education also had a lower probability of endorsing this item. DIF was also observed for age for two items after correction for multiple comparisons for both the IRT and OLR-based models: "I have had to work really hard to pay attention or I would make a mistake" and "I have had trouble shifting back and forth between different activities that require thinking". For both items, conditional on cognitive complaints, older respondents had a higher likelihood than younger respondents of endorsing the item in the cognitive complaints direction. The magnitude and impact of DIF was minimal. The scale showed high precision along much of the subjective cognitive concerns continuum; the overall IRT-based reliability estimate for the total sample was 0.88 and the estimates for subgroups ranged from 0.87 to 0.92.

CONCLUSION

Little DIF of high magnitude or impact was observed in the PROMIS Applied Cognition - General Concerns short form item set. One item, "It has seemed like my brain was not working as well as usual" might be singled out for further study. However, in general the short form item set was highly reliable, informative, and invariant across differing race/ethnic, educational, age, gender, and language groups.

Measurement Equivalence of the Patient Reported Outcomes Measurement Information System® (PROMIS®) Pain Interference Short Form Items: Application to Ethnically Diverse Cancer and Palliative Care Populations

Reducing the response burden of standardized pain measures is desirable, particularly for individuals who are frail or live with chronic illness, e.g., those suffering from cancer and those in palliative care. The Patient Reported Outcome Measurement Information System^® (PROMIS^®) project addressed this issue with the provision of computerized adaptive tests (CAT) and short form measures that can be used clinically and in research. Although there has been substantial evaluation of PROMIS item banks, little is known about the performance of PROMIS short forms, particularly in ethnically diverse groups. Reviewed in this article are findings related to the differential item functioning (DIF) and reliability of the PROMIS pain interference short forms across diverse sociodemographic groups.

METHODS

DIF hypotheses were generated for the PROMIS short form pain interference items. Initial analyses tested item response theory (IRT) model assumptions of unidimensionality and local independence. Dimensionality was evaluated using factor analytic methods; local dependence (LD) was tested using IRT-based LD indices. Wald tests were used to examine group differences in IRT parameters, and to test DIF hypotheses. A second DIF-detection method used in sensitivity analyses was based on ordinal logistic regression with a latent IRT-derived conditioning variable. Magnitude and impact of DIF were investigated, and reliability and item and scale information statistics were estimated.

RESULTS

The reliability of the short form item set was excellent. However, there were a few items with high local dependency, which affected the estimation of the final discrimination parameters. As a result, the item, "How much did pain interfere with enjoyment of social activities?" was excluded in the DIF analyses for all subgroup comparisons. No items were hypothesized to show DIF for race and ethnicity; however, five items showed DIF after adjustment for multiple comparisons in both primary and sensitivity analyses: ability to concentrate, enjoyment of recreational activities, tasks away from home, participation in social activities, and socializing with others. The magnitude of DIF was small and the impact negligible. Three items were consistently identified with DIF for education: enjoyment of life, ability to concentrate, and enjoyment of recreational activities. No item showed DIF above the magnitude threshold and the impact of DIF on the overall measure was minimal. No item showed gender DIF after correction for multiple comparisons in the primary analyses. Four items showed consistent age DIF: enjoyment of life, ability to concentrate, day to day activities, and enjoyment of recreational activities, none with primary magnitude values above threshold. Conditional on the pain state, Spanish speakers were hypothesized to report less pain interference on one item, enjoyment of life. The DIF findings confirmed the hypothesis; however, the magnitude was small. Using an arbitrary cutoff point of theta (θ) ≥ 1.0 to classify respondents with acute pain interference, the highest number of changes were for the education groups analyses. There were 231 respondents (4% of the total sample) who changed from the designation of no acute pain interference to acute interference after the DIF adjustment. There was no change in the designations for race/ethnic subgroups, and a small number of changes for respondents aged 65 to 84.

CONCLUSIONS

Although significant DIF was observed after correction for multiple comparisons, all DIF was of low magnitude and impact. However, some individual-level impact was observed for low education groups. Reliability estimates were high. Thus, the PROMIS short form pain items examined in this ethnically diverse sample performed relatively well; although one item was problematic and removed from the analyses. It is concluded that the majority of the PROMIS pain interference short form items can be recommended for use among ethnically diverse groups, including those in palliative care and with cancer and chronic illness.

Does Acquiescence Affect Individual Items Consistently?

Previous research has found the effects of acquiescence to be generally consistent across item aggregates within a single survey (i.e., essential tau-equivalence), but it is unknown whether this phenomenon is consistent at the individual item level. This article evaluated the often assumed but inadequately tested proposition that individual items are affected by acquiescence to the same degree. We modeled an external acquiescence criterion to assess (a) whether it affected scale items consistently and (b) whether it would be strongly correlated with an acquiescence factor based on an assumption of tau-equivalence. The results did not support this assumption. As further evidence, we identified a situation in which this tau-equivalence assumption could potentially be violated. We propose that the response style be best understood within a framework of an acquiescence × item interaction.