Misstatements, misperceptions, and mistakes in controlling for covariates in observational research

We discuss 12 misperceptions, misstatements, or mistakes concerning the use of covariates in observational or nonrandomized research. Additionally, we offer advice to help investigators, editors, reviewers, and readers make more informed decisions about conducting and interpreting research where the influence of covariates may be at issue. We primarily address misperceptions in the context of statistical management of the covariates through various forms of modeling, although we also emphasize design and model or variable selection. Other approaches to addressing the effects of covariates, including matching, have logical extensions from what we discuss here but are not dwelled upon heavily. The misperceptions, misstatements, or mistakes we discuss include accurate representation of covariates, effects of measurement error, overreliance on covariate categorization, underestimation of power loss when controlling for covariates, misinterpretation of significance in statistical models, and misconceptions about confounding variables, selecting on a collider, and p value interpretations in covariate-inclusive analyses. This condensed overview serves to correct common errors and improve research quality in general and in nutrition research specifically.

How to Select 2D and 3D Roughness Parameters at Their Relevant Scales by the Analysis of Covariance

In this paper, a multi-scale methodology is proposed to model and characterize the effect of two lubricants on changes in surface morphology during a running-in test. The test concerns two steels samples, mounted on a twin-disc tribometer to test each of lubricants A and B for a period of 42 h. The changes are characterized by the standardized roughness parameters given in ISO 25178. A technique involving replication is used to monitor wear during the test. Using all these replication measurements, a multi-scale methodology is applied. These selected models highlighted the relevant parameters for quantifying wear during lifespan, and also showed that lubricant A was better able to preserve surface integrity during wear than lubricant B.

The relationship between expected engagement and talking therapy outcome

The aim of the study was to investigate whether client‐reported expected engagement with therapy predicted therapy outcome. It was hypothesized that higher expected engagement with cognitive behavioural therapy (CBT) or person‐centred experiential therapy (PCET) would predict more symptomatic improvement following therapy and higher likelihood of therapy completion. The Sheffield Expected Engagement with Therapy Scale was administered to 96 clients at pre‐therapy assessment with all meeting a diagnosis of moderate or severe depression with 53 receiving CBT and 43 receiving PCET. Higher expected engagement predicted more symptomatic improvement in CBT but not PCET. Expected engagement only predicted improvement in CBT when clients rated the credibility of CBT as low or moderate. Expected engagement did not predict therapy completion in either therapy. Assessment of expected engagement could be a useful tool in prediction of symptomatic improvement in CBT.

Comparing models of change to estimate the mediated effect in the pretest-posttest control group design

Models to assess mediation in the pretest-posttest control group design are understudied in the behavioral sciences even though it is the design of choice for evaluating experimental manipulations. The paper provides analytical comparisons of the four most commonly used models used to estimate the mediated effect in this design: Analysis of Covariance (ANCOVA), difference score, residualized change score, and cross-sectional model. Each of these models are fitted using a Latent Change Score specification and a simulation study assessed bias, Type I error, power, and confidence interval coverage of the four models. All but the ANCOVA model make stringent assumptions about the stability and cross-lagged relations of the mediator and outcome that may not be plausible in real-world applications. When these assumptions do not hold, Type I error and statistical power results suggest that only the ANCOVA model has good performance. The four models are applied to an empirical example.

Action and perception in social contexts: intentional binding for social action effects

The subjective experience of controlling events in the environment alters the perception of these events. For instance, the interval between one's own actions and their consequences is subjectively compressed—a phenomenon known as intentional binding. In two experiments, we studied intentional binding in a social setting in which actions of one agent prompted a second agent to perform another action. Participants worked in pairs and were assigned to a “leader” and a “follower” role, respectively. The leader's key presses triggered (after a variable interval) a tone and this tone served as go signal for the follower to perform a keypress as well. Leaders and followers estimated the interval between the leader's keypress and the following tone, or the interval between the tone and the follower's keypress. The leader showed reliable intentional binding for both intervals relative to the follower's estimates. These results indicate that human agents experience a pre-reflective sense of agency for genuinely social consequences of their actions.

A Monte Carlo Comparison Study of the Power of the Analysis of Covariance, Simple Difference, and Residual Change Scores in Testing Two-Wave Data

This study compares the analysis of covariance (ANCOVA), difference score, and residual change score methods in testing the group effect for pretest–posttest data in terms of statistical power and Type I error rates using a Monte Carlo simulation. Previous research has mathematically shown the effect of stability of individual scores from pretest to posttest, reliability, and nonrandomization (i.e., pretest imbalance) on the performance of the ANCOVA, difference score, and residual change score methods. However, related power issues have not been adequately addressed. The authors examined the impact of stability of measurement over time, reliability of covariate and criterion, nonrandomization, sample size, and treatment effect size on statistical power of the three methods. Across conditions, ANCOVA and residual change score methods had similar power rates. When reliability was less than perfect, ANCOVA had more power than the difference score method when there was an increase from pretest to posttest and a positive baseline imbalance (i.e., treatment group had higher pretest scores than the control group), or when there was a decrease from pretest to posttest and a negative baseline imbalance, and vice versa. In case of perfect reliability, the statistical power of ANCOVA did not differ from the difference score method. For the difference score method, when reliability was low, there was no effect of stability on power, whereas when reliability was high or perfect, power increased as stability increased for medium and large effect sizes. Difference scores may be preferred over ANCOVA under certain circumstances.

Gender effects in gaming research: a case for regression residuals?

Numerous recent studies have examined the impact of video gaming on various dependent variables, including the players' affective reactions, positive as well as detrimental cognitive effects, and real-world aggression. These target variables are typically analyzed as a function of game characteristics and player attributes-especially gender. However, findings on the uneven distribution of gaming experience between males and females, on the one hand, and the effect of gaming experience on several target variables, on the other hand, point at a possible confound when gaming experiments are analyzed with a standard analysis of variance. This study uses simulated data to exemplify analysis of regression residuals as a potentially beneficial data analysis strategy for such datasets. As the actual impact of gaming experience on each of the various dependent variables differs, the ultimate benefits of analysis of regression residuals entirely depend on the research question, but it offers a powerful statistical approach to video game research whenever gaming experience is a confounding factor.