Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling

An examination of model fit and measurement invariance of general mental ability and personality measures used in the multilingual context of the Swiss Armed Forces: A Bayesian structural equation modeling approach

Measurement invariance of psychological test batteries is an essential quality criterion when the test batteries are administered in different cultural and language contexts. The purpose of this study was to examine to what extent measurement model fit and measurement invariance across the two largest language groups in Switzerland (i.e., German and French speakers) can be assumed for selected general mental ability and personality tests used in the Swiss Armed Forces' cadre selection process. For the model fit and invariance testing, we used Bayesian structural equation modeling (BSEM). Because the sizes of the language group samples were unbalanced, we reran the invariance testing with the subsampling procedure as a robustness check. The results showed that at least partial approximate scalar invariance can be assumed for the constructs. However, comparisons in the full sample and subsamples also showed that certain test items function differently across the language groups. The results are discussed regarding the three following issues: First, we critically discuss the applied criterion and alternative effect size measures for assessing the practical importance of non-invariances. Second, we highlight potential remedies and further testing options, that can be applied, once certain items have been detected to function differently. Third, we discuss alternative modeling and invariance testing approaches to BSEM and outline future research avenues.

Examination of the validity of the Injury-Psychological Readiness to Return to Sport (I-PRRS) scale in male professional football players: A worldwide study of 29 professional teams

Perceived confidence is an important dimension of an athlete's psychological readiness to return-to-play. However, there is no established and validated tool to evaluate confidence in professional football. This study aimed to provide preliminary evaluation of the internal structure of the Injury-Psychological Readiness to Return-to-Sport scale (I-PRRS) in a cohort of injured male professional footballers. Over an 18-month period, 29 teams from 17 leagues participated. Players sustaining injuries eliciting ≥ 3 weeks' time-loss were recruited. Cross culturally adapted to 4 further languages, the I-PRRS was administered on two occasions: 1) day before returning-to-training and 2) day before returning-to-match-play. In total, 113 injuries were recorded with 96 completed I-PRRS data sets collected. Confirmatory factor analysis indicated the I-PRRS was a unidimensional scale, with all items measuring the same construct. The scale demonstrated good internal consistency (ω = .88). When examining longitudinal invariance of the I-PRRS across administration time-points, indices of model fit supported scalar invariance. There was preliminary evidence of good internal structure for the I-PRRS in professional male footballers. However, before further research involving the I-PRRS can be endorsed, efforts to confirm or refute empirical developments pertaining to psychological readiness are necessary.

Psychometric properties of the Farsi version of the Eating Pathology Symptoms Inventory (F-EPSI) among Iranian University men and women

BACKGROUND

Limited research has validated eating pathology assessments in Iranian men and women. The purpose of the current study was to translate and validate a Farsi version of the Eating Pathology Symptoms Inventory (F-EPSI) in Iranian university students.

METHODS

Men (n = 279) and women (n = 486) completed questionnaires including the F-EPSI.

RESULTS

Confirmatory factor analysis (CFA) indicated that the F-EPSI had an acceptable fit to the data and supported the eight-factor model. The scale was partially invariant across genders. Men reported higher scores on Excessive Exercise and Muscle Building subscales, and women reported higher scores on Body Dissatisfaction and Restricting subscales. The F-EPSI subscales had good 5- to 6-month test-retest reliability. The F-EPSI demonstrated convergent validity with clinical impairment, eating pathology, and body mass index (BMI). Finally, individuals scoring above the Clinical Impairment Assessment (CIA) cutoffs reported higher scores on the F-EPSI subscales, further supporting convergent validity of the scale.

CONCLUSION

Findings suggest that the F-EPSI will enable researchers to examine eating pathology symptoms among men and women in Iran.

Adjusting for partial invariance in latent parameter estimation: Comparing forward specification search and approximate invariance methods

Measurement invariance is the condition that an instrument measures a target construct in the same way across subgroups, settings, and time. In psychological measurement, usually only partial, but not full, invariance is achieved, which potentially biases subsequent parameter estimations and statistical inferences. Although existing literature shows that a correctly specified partial invariance model can remove such biases, it ignores the model uncertainty in the specification search step: flagging the wrong items may lead to additional bias and variability in subsequent inferences. On the other hand, several new approaches, including Bayesian approximate invariance and alignment optimization methods, have been proposed; these methods use an approximate invariance model to adjust for partial measurement invariance without the need to directly identify noninvariant items. However, there has been limited research on these methods in situations with a small number of groups. In this paper, we conducted three systematic simulation studies to compare five methods for adjusting partial invariance. While specification search performed reasonably well when the proportion of noninvariant parameters was no more than one-third, alignment optimization overall performed best across conditions in terms of efficiency of parameter estimates, confidence interval coverage, and type I error rates. In addition, the Bayesian version of alignment optimization performed best for estimating latent means and variances in small-sample and low-reliability conditions. We thus recommend the use of the alignment optimization methods for adjusting partial invariance when comparing latent constructs across a few groups.

The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation

Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the ubiquity of correlated residuals and imperfect model specification. Our research focuses on a scale evaluation context and the performance of four standard model fit indices: root mean square error of approximate (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI), and two equivalence test-based model fit indices: RMSEAt and CFIt. We use Monte Carlo simulation to generate and analyze data based on a substantive example using the positive and negative affective schedule (N = 1,000). We systematically vary the number and magnitude of correlated residuals as well as nonspecific misspecification, to evaluate the impact on model fit indices in fitting a two-factor exploratory factor analysis. Our results show that all fit indices, except SRMR, are overly sensitive to correlated residuals and nonspecific error, resulting in solutions that are overfactored. SRMR performed well, consistently selecting the correct number of factors; however, previous research suggests it does not perform well with categorical data. In general, we do not recommend using model fit indices to select number of factors in a scale evaluation framework.

It Matters: Reference Indicator Selection in Measurement Invariance Tests

Conventional approaches for selecting a reference indicator (RI) could lead to misleading results in testing for measurement invariance (MI). Several newer quantitative methods have been available for more rigorous RI selection. However, it is still unknown how well these methods perform in terms of correctly identifying a truly invariant item to be an RI. Thus, Study 1 was designed to address this issue in various conditions using simulated data. As a follow-up, Study 2 further investigated the advantages/disadvantages of using RI-based approaches for MI testing in comparison with non-RI-based approaches. Altogether, the two studies provided a solid examination on how RI matters in MI tests. In addition, a large sample of real-world data was used to empirically compare the uses of the RI selection methods as well as the RI-based and non-RI-based approaches for MI testing. In the end, we offered a discussion on all these methods, followed by suggestions and recommendations for applied researchers.

Measurement invariance of the SATAQ-4 in Black and White undergraduate women

The Sociocultural Attitudes Towards Appearance Questionnaire-4 (SATAQ-4) is a widely used measure of sociocultural factors affecting body image. Despite evidence sociocultural appearance ideals and pressures might differ between White and Black women, the SATAQ-4's invariance has not been formally evaluated. The current study examined its measurement invariance across White and Black college women. Women (N = 730; 68.5 % White, 31.5 % Black), recruited from a large, public university in the United States completed measures online. Confirmatory factor analysis (CFA) showed the original five-factor structure was adequate in each group, and it appeared equivalent across groups. However, full metric invariance was not supported. Factor loadings on one item differed, suggesting differences in how this item contributed to thin-ideal internalization between groups. Similarly, one threshold differed, suggesting Black women who mostly agreed they wanted a "very thin" body had more thin-ideal internalization than White women with the same rating. These results suggest comparisons between these groups on the thin-ideal internalization subscale should be interpreted cautiously, as group differences could reflect some measurement bias. Given appearance pressures and ideals extend beyond thinness, particularly for Black women, future research should consider refining or developing culturally sensitive measures, to enhance relevance between and within racial/ethnic groups.

An illustration of Bayesian approximate measurement invariance with longitudinal data and a small sample size

This article illustrates the Bayesian approximate measurement invariance (MI) approach in M plus with longitudinal data and small sample size. Approximate MI incorporates zero-mean small variance prior distributions on the differences between parameter estimates over time. Contrary to traditional invariance testing methods, where exact invariance is tested, this method allows for some “wiggle room” in the parameter estimates over time. The procedure is illustrated using longitudinal data on college students’ academic stress as it changes in the period leading up to and right after an important midterm. Results show that traditional invariance testing methods come to a standstill due to the small sample size. Bayesian approximate MI testing was able to identify non-invariant parameters, after which a partially invariant model could be estimated.