Inappropriate statistical method in a parallel-group randomized controlled trial results in unsubstantiated conclusions

Comment on Marsigliante et al. Effects on Children's Physical and Mental Well-Being of a Physical-Activity-Based School Intervention Program: A Randomized Study. Int. J. Environ. Res. Public Health 2023, 20, 1927

We conducted a critical review of the article "Effects on Children's Physical and Mental Well-Being of a Physical-Activity-Based School Intervention Program: A Randomized Study", published in the International Journal of Environmental Research and Public Health in 2023 as part of the Special Issue "Psychomotricity and Physical Education in School Health". We identified multiple mistakes in the statistical analyses applied. First, the authors claim to have found a statistically significant association between the proposed intervention and change in body composition (body mass index (BMI) percentiles, relative fat mass, and BMI classes) by way of exhibiting differences in nominal significance between the pre- and post-intervention changes within the control and intervention groups, instead of exhibiting a significant difference between groups. Furthermore, the analysis described fails to account for clustering and nesting in the data. The reporting of the statistical methods and results include multiple elements that are variously incorrect, incoherent, or impossible. Revised statistical analyses are proposed which can render the study's methods valid and its results substantiated, whereas the current methods and results are invalid and unsubstantiated, respectively.

Errors in the implementation, analysis, and reporting of randomization within obesity and nutrition research: a guide to their avoidance

Randomization is an important tool used to establish causal inferences in studies designed to further our understanding of questions related to obesity and nutrition. To take advantage of the inferences afforded by randomization, scientific standards must be upheld during the planning, execution, analysis, and reporting of such studies. We discuss ten errors in randomized experiments from real-world examples from the literature and outline best practices for their avoidance. These ten errors include: representing nonrandom allocation as random, failing to adequately conceal allocation, not accounting for changing allocation ratios, replacing subjects in nonrandom ways, failing to account for non-independence, drawing inferences by comparing statistical significance from within-group comparisons instead of between-groups, pooling data and breaking the randomized design, failing to account for missing data, failing to report sufficient information to understand study methods, and failing to frame the causal question as testing the randomized assignment per se. We hope that these examples will aid researchers, reviewers, journal editors, and other readers to endeavor to a high standard of scientific rigor in randomized experiments within obesity and nutrition research.

Contrary to the Conclusions Stated in the Paper, Only Dry Fat-Free Mass Was Different between Groups upon Reanalysis. Comment on: "Intermittent Energy Restriction Attenuates the Loss of Fat-Free Mass in Resistance Trained Individuals. A Randomized Controlled Trial"

Campbell and colleagues recently published a randomised controlled trial investigating the effects of diets involving intermittent energy restriction versus continuous energy restriction on changes in body composition and resting metabolic rate (RMR) in resistance-trained adults[...].

Common scientific and statistical errors in obesity research

This review identifies 10 common errors and problems in the statistical analysis, design, interpretation, and reporting of obesity research and discuss how they can be avoided. The 10 topics are: 1) misinterpretation of statistical significance, 2) inappropriate testing against baseline values, 3) excessive and undisclosed multiple testing and "P-value hacking," 4) mishandling of clustering in cluster randomized trials, 5) misconceptions about nonparametric tests, 6) mishandling of missing data, 7) miscalculation of effect sizes, 8) ignoring regression to the mean, 9) ignoring confirmation bias, and 10) insufficient statistical reporting. It is hoped that discussion of these errors can improve the quality of obesity research by helping researchers to implement proper statistical practice and to know when to seek the help of a statistician.