Systematic heterogenization revisited: Increasing variation in animal experiments to improve reproducibility?

Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor

The reproducibility of studies involving insect species is an underexplored area in the broader discussion about poor reproducibility in science. Our study addresses this gap by conducting a systematic multi-laboratory investigation into the reproducibility of ecological studies on insect behavior. We implemented a 3 × 3 experimental design, incorporating three study sites, and three independent experiments on three insect species from different orders: the turnip sawfly (Athalia rosae, Hymenoptera), the meadow grasshopper (Pseudochorthippus parallelus, Orthoptera), and the red flour beetle (Tribolium castaneum, Coleoptera). Using random-effect meta-analysis, we compared the consistency and accuracy of treatment effects on insect behavioral traits across replicate experiments. We successfully reproduced the overall statistical treatment effect in 83% of the replicate experiments, but overall effect size replication was achieved in only 66% of the replicates. Thus, though demonstrating sufficient reproducibility in some measures, this study also provides the first experimental evidence for cases of poor reproducibility in insect experiments. Our findings further show that reasons causing poor reproducibility established in rodent research also hold for other study organisms and research questions. We believe that a rethinking of current best practices is required to face reproducibility issues in insect studies but also across disciplines. Specifically, we advocate for adopting open research practices and the implementation of methodological strategies that reduce bias and problems arising from over-standardization. With respect to the latter, the introduction of systematic variation through multi-laboratory or heterogenized designs may contribute to improved reproducibility in studies involving any living organisms.

The 6Rs of EFDT Studies

Given the increased concerns over reproducibility, replicability, and reliability of scientific studies, the embryo-fetal developmental toxicity (EFDT) study was examined from this perspective together with the classic 3Rs. These 6Rs contribute to external validity (i.e., applicability to human safety). Species differences and experimental conditions lower the validity of all preclinical studies to variable extents. The apparent lack/paucity of demonstrated replicate EFDT studies lowers reliability and thus validity. I make suggestions intended to add reliability for the continued use of these studies, which are a cornerstone for assessing safety during human pregnancies. Experienced readers will recognize that the suggestions would also apply to other guideline-mandated developmental and reproductive toxicity and toxicity studies.

The Effect of Micronutrients on Obese Phenotype of Adult Mice Is Dependent on the Experimental Environment

The environment of the test laboratory affects the reproducibility of treatment effects on physiological phenotypes of rodents and may be attributed to the plasticity of the epigenome due to nutrient-gene-environment interactions. Here, we explored the reproducibility of adding a multi-vitamin-mineral (MVM) mix to a nutrient-balanced high-fat (HF) diet on obesity, insulin resistance (IR), and gene expression in the tissues of adult male mice. Experiments of the same design were conducted in three independent animal facilities. Adult C57BL/6J male mice were fed an HF diet for 6 weeks (diet induced-obesity model) and then continued for 9-12 weeks on the HF diet with or without 5-fold additions of vitamins A, B1, B6, B12, Zn, and 2-fold Se. The addition of the MVM affected body weight, fat mass, gene expression, and markers of IR in all three locations (p < 0.05). However, the direction of the main effects was influenced by the interaction with the experimental location and its associated environmental conditions known to affect the epigenome. In conclusion, MVM supplementation influenced phenotypes and expression of genes related to adipose function in obese adult male mice, but the experimental location and its associated conditions were significant interacting factors. Preclinical studies investigating the relationship between diet and metabolic outcomes should acknowledge the plasticity of the epigenome and implement measures to reproduce studies in different locations.