A Simulation Study on the Performance of Different Reliability Estimation Methods

Beyond alpha and omega: The accuracy of single-test reliability estimators in unidimensional continuous data

Coefficient alpha is commonly used as a reliability estimator. However, several estimators are believed to be more accurate than alpha, with factor analysis (FA) estimators being the most commonly recommended. Furthermore, unstandardized estimators are considered more accurate than standardized estimators. In other words, the existing literature suggests that unstandardized FA estimators are the most accurate regardless of data characteristics. To test whether this conventional knowledge is appropriate, this study examines the accuracy of 12 estimators using a Monte Carlo simulation. The results show that several estimators are more accurate than alpha, including both FA and non-FA estimators. The most accurate on average is a standardized FA estimator. Unstandardized estimators (e.g., alpha) are less accurate on average than the corresponding standardized estimators (e.g., standardized alpha). However, the accuracy of estimators is affected to varying degrees by data characteristics (e.g., sample size, number of items, outliers). For example, standardized estimators are more accurate than unstandardized estimators with a small sample size and many outliers, and vice versa. The greatest lower bound is the most accurate when the number of items is 3 but severely overestimates reliability when the number of items is more than 3. In conclusion, estimators have their advantageous data characteristics, and no estimator is the most accurate for all data characteristics.

Task-dependent Alteration in Delta Band Corticomuscular Coherence during Standing in Chronic Stroke Survivors

Balance control is an important indicator of mobility and independence in activities of daily living. How the changes in functional integrity of corticospinal tract due to stroke affects the maintenance of upright stance remains to be known. We investigated the changes in functional coupling between the cortex and lower limb muscles during a challenging balance task over multiple frequency bands in chronic stroke survivors. Eleven stroke patients and nine healthy controls performed a challenging balance task. They stood on a computerized platform with/without somatosensory input distortion created by sway-referencing the support surface, thereby varying the difficulty levels of the task. We computed corticomuscular coherence between Cz (electroencephalography) and leg muscles and assessed balance performance using Berg Balance scale (BBS), Timed-up and go (TUG) and center of pressure (COP) measures. We found lower delta frequency band coherence in stroke patients when compared with healthy controls under medium difficulty condition for distal but not proximal leg muscles. For both groups, we found similar coherence at other frequency bands. On BBS and TUG, stroke patients showed poor balance. However, similar group differences were not consistently observed across COP measures. The presence of distal versus proximal effect suggests differences in the (re)organization of the corticospinal connections across the two muscles groups for balance control. We argue that the observed group difference in the delta coherence might be due to altered mechanisms for the detection of somatosensory modulation resulting from sway-referencing of the support platform for balance control.

Need satisfaction and frustration scale (NSFS): adaptation and validation for Brazilian gig work context

Purpose

This study aims to adapt and validate the need satisfaction and frustration scale (NSFS) for the Brazilian gig work context and investigate the distinctiveness of the constructs of basic needs satisfaction and frustration among Brazilian gig workers.

Design/methodology/approach

Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were conducted on a sample of 351 Brazilian gig workers.

Findings

Results from EFA supported a three-factor structure, while results from CFA psychometrically supported a six-factor correlated model. The items that constitute the NSFS presented good discriminant validity (heterotrait-monotrait ratio) and internal consistency (Cronbach’s alpha and McDonald’s omega coefficients). Results regarding criterion validity partially support previous empirical evidence that considers needs satisfaction and frustration independent in relation to well-being and ill-being while reinforcing the need for further investigations. The Brazilian version of the NSFS is shown to be an instrument with robust psychometric qualities to assess workers’ perception of basic needs satisfaction and frustration in Brazilian gig work context.

Originality/value

This study broadens the scope of research on basic psychological needs by introducing a valid and reliable instrument to assess workers’ perceptions of needs satisfaction and frustration in the Brazilian gig work context, a population that has been neglected in self-determination theory research.

Performance of Coefficient Alpha and Its Alternatives: Effects of Different Types of Non-Normality

We examined the performance of coefficient alpha and its potential competitors (ordinal alpha, omega total, Revelle's omega total [omega RT], omega hierarchical [omega h], greatest lower bound [GLB], and coefficient H) with continuous and discrete data having different types of non-normality. Results showed the estimation bias was acceptable for continuous data with varying degrees of non-normality when the scales were strong (high loadings). This bias, however, became quite large with moderate strength scales and increased with increasing non-normality. For Likert-type scales, other than omega h, most indices were acceptable with non-normal data having at least four points, and more points were better. For different exponential distributed data, omega RT and GLB were robust, whereas the bias of other indices for binomial-beta distribution was generally large. An examination of an authentic large-scale international survey suggested that its items were at worst moderately non-normal; hence, non-normality was not a big concern. We recommend (a) the demand for continuous and normally distributed data for alpha may not be necessary for less severely non-normal data; (b) for severely non-normal data, we should have at least four scale points, and more points are better; and (c) there is no single golden standard for all data types, other issues such as scale loading, model structure, or scale length are also important.

Typology of Deflation-Corrected Estimators of Reliability

The reliability of a test score is discussed from the viewpoint of underestimation of and, specifically, deflation in estimates or reliability. Many widely used estimators are known to underestimate reliability. Empirical cases have shown that estimates by widely used estimators such as alpha, theta, omega, and rho may be deflated by up to 0.60 units of reliability or even more, with certain types of datasets. The reason for this radical deflation lies in the item-score correlation (Rit) embedded in the estimators: because the estimates by Rit are deflated when the number of categories in scales are far from each other, as is always the case with item and score, the estimates of reliability are deflated as well. A short-cut method to reach estimates closer to the true magnitude, new types of estimators, and deflation-corrected estimators of reliability (DCERs), are studied in the article. The empirical section is a study on the characteristics of combinations of DCERs formed by different bases for estimators (alpha, theta, omega, and rho), different alternative estimators of correlation as the linking factor between item and the score variable, and different conditions. Based on the simulation, an initial typology of the families of DCERs is presented: some estimators are better with binary items and some with polytomous items; some are better with small sample sizes and some with larger ones.