Reproducibility of Experiments with Laboratory Animals: What Should We Do Now?

Questionable Research Practices, Low Statistical Power, and Other Obstacles to Replicability: Why Preclinical Neuroscience Research Would Benefit from Registered Reports

Replicability, the degree to which a previous scientific finding can be repeated in a distinct set of data, has been considered an integral component of institutionalized scientific practice since its inception several hundred years ago. In the past decade, large-scale replication studies have demonstrated that replicability is far from favorable, across multiple scientific fields. Here, I evaluate this literature and describe contributing factors including the prevalence of questionable research practices (QRPs), misunderstanding of p-values, and low statistical power. I subsequently discuss how these issues manifest specifically in preclinical neuroscience research. I conclude that these problems are multifaceted and difficult to solve, relying on the actions of early and late career researchers, funding sources, academic publishers, and others. I assert that any viable solution to the problem of substandard replicability must include changing academic incentives, with adoption of registered reports being the most immediately impactful and pragmatic strategy. For animal research in particular, comprehensive reporting guidelines that document potential sources of sensitivity for experimental outcomes is an essential addition.

Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis

To those involved in discussions about rigor, reproducibility, and replication in science, conversation about the "reproducibility crisis" appear ill-structured. Seemingly very different issues concerning the purity of reagents, accessibility of computational code, or misaligned incentives in academic research writ large are all collected up under this label. Prior work has attempted to address this problem by creating analytical definitions of reproducibility. We take a novel empirical, mixed methods approach to understanding variation in reproducibility discussions, using a combination of grounded theory and correspondence analysis to examine how a variety of authors narrate the story of the reproducibility crisis. Contrary to expectations, this analysis demonstrates that there is a clear thematic core to reproducibility discussions, centered on the incentive structure of science, the transparency of methods and data, and the need to reform academic publishing. However, we also identify three clusters of discussion that are distinct from the main body of articles: one focused on reagents, another on statistical methods, and a final cluster focused on the heterogeneity of the natural world. Although there are discursive differences between scientific and popular articles, we find no strong differences in how scientists and journalists write about the reproducibility crisis. Our findings demonstrate the value of using qualitative methods to identify the bounds and features of reproducibility discourse, and identify distinct vocabularies and constituencies that reformers should engage with to promote change.

Three Pillars of Automated Home-Cage Phenotyping of Mice: Novel Findings, Refinement, and Reproducibility Based on Literature and Experience

Animal models of neurodegenerative and neuropsychiatric disorders require extensive behavioral phenotyping. Currently, this presents several caveats and the most important are: (i) rodents are nocturnal animals, but mostly tested during the light period; (ii) the conventional behavioral experiments take into consideration only a snapshot of a rich behavioral repertoire; and (iii) environmental factors, as well as experimenter influence, are often underestimated. Consequently, serious concerns have been expressed regarding the reproducibility of research findings on the one hand, and appropriate welfare of the animals (based on the principle of 3Rs-reduce, refine and replace) on the other hand. To address these problems and improve behavioral phenotyping in general, several solutions have been proposed and developed. Undisturbed, 24/7 home-cage monitoring (HCM) is gaining increased attention and popularity as demonstrating the potential to substitute or complement the conventional phenotyping methods by providing valuable data for identifying the behavioral patterns that may have been missed otherwise. In this review, we will briefly describe the different technologies used for HCM systems. Thereafter, based on our experience, we will focus on two systems, IntelliCage (NewBehavior AG and TSE-systems) and Digital Ventilated Cage (DVC®, Tecniplast)-how they have been developed and applied during recent years. Additionally, we will touch upon the importance of the environmental/experimenter artifacts and propose alternative suggestions for performing phenotyping experiments based on the published evidence. We will discuss how the integration of telemetry systems for deriving certain physiological parameters can help to complement the description of the animal model to offer better translation to human studies. Ultimately, we will discuss how such HCM data can be statistically interpreted and analyzed.

An alcohol withdrawal test battery measuring multiple behavioral symptoms in mice

Despite acceptance that risk for alcohol-use disorder (AUD) has a large genetic component, the identification of genes underlying various components of risk for AUD has been hampered in humans, in part by the heterogeneity of expression of the phenotype. One aspect of AUD is physical dependence. Alcohol withdrawal is a serious consequence of alcohol dependence with multiple symptoms, many of which are seen in multiple species, and can be experienced over a wide-ranging time course. In the present three studies, we developed a battery of withdrawal tests in mice, examining behavioral symptoms from multiple domains that could be measured over time. To permit eventual use of the battery in different strains of mice, we used male and female mice of a genetically heterogeneous stock developed from intercrossing eight inbred strains. Withdrawal symptoms were assessed using commonly used tests after administration of ethanol in vapor for 72 continuous hours. We found significant effects of ethanol withdrawal versus air-breathing controls on nearly all symptoms, spanning 4 days following ethanol vapor inhalation. Withdrawal produced hypothermia, greater neurohyperexcitability (seizures and tremor), anxiety-like behaviors using an apparatus (such as reduced transitions between light and dark compartments), anhedonia (reduced sucrose preference), Straub tail, backward walking, and reductions in activity; however, there were no changes in thermal pain sensitivity, hyper-reactivity to handling, or anxiety-like emergence behaviors in other apparatus. Using these data, we constructed a refined battery of withdrawal tests. Individual differences in severity of withdrawal among different tests were weakly correlated at best. This battery should be useful for identifying genetic influences on particular withdrawal behaviors, which should reflect the influences of different constellations of genes.

Reproducibility and replicability of rodent phenotyping in preclinical studies

The scientific community is increasingly concerned with the proportion of published “discoveries” that are not replicated in subsequent studies. The field of rodent behavioral phenotyping was one of the first to raise this concern, and to relate it to other methodological issues: the complex interaction between genotype and environment; the definitions of behavioral constructs; and the use of laboratory mice and rats as model species for investigating human health and disease mechanisms. In January 2015, researchers from various disciplines gathered at Tel Aviv University to discuss these issues. The general consensus was that the issue is prevalent and of concern, and should be addressed at the statistical, methodological and policy levels, but is not so severe as to call into question the validity and the usefulness of model organisms as a whole. Well-organized community efforts, coupled with improved data and metadata sharing, have a key role in identifying specific problems and promoting effective solutions. Replicability is closely related to validity, may affect generalizability and translation of findings, and has important ethical implications.

Of Mice, Dirty Mice, and Men: Using Mice To Understand Human Immunology

Mouse models have enabled breakthroughs in our understanding of the immune system, but it has become increasingly popular to emphasize their shortcomings when translating observations to humans. This review provides a brief summary of mouse natural history, husbandry, and the pros and cons of pursuing basic research in mice versus humans. Opportunities are discussed for extending the predictive translational value of mouse research, with an emphasis on exploitation of a "dirty" mouse model that better mimics the diverse infectious history that is typical of most humans.