A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus)

A systematic review and meta-analysis of environmental contaminant exposure impacts on weight loss and glucose regulation during calorie-restricted diets in preclinical studies: Persistent organic pollutants may impede glycemic control

Epidemiological evidence links chemical exposure with type 2 diabetes (T2DM) risk and prevalence. Chemical exposure may therefore also limit success of weight loss or restoration of glycemic control during calorie restricted diets. Few human studies examine this hypothesis. This systematic review and clustered meta-analysis examines preclinical evidence that exposure to anthropogenic environmental contaminants impedes weight loss and resumption of glycemic control during calorie restriction. Of five eligible papers from 212 unique citations, four used C57BL/6 mice and one used Sprague Dawley rats. In four the animals received high fat diets to induce obesity and impaired glycemic control. All examined persistent organic pollutants (POPs). Polychlorinated biphenyl (PCB) 77 exposure did not affect final mass (standardised mean difference (SMD) = -0.35 [-1.09, 0.39]; n = 5 (experiments); n = 3 (papers)), or response to insulin in insulin tolerance tests (SMD = -1.54 [-3.25, 0.16] n = 3 (experiments); n = 2 (papers)), but impaired glucose control in glucose tolerance tests (SMD = -1.30 [-1.96, -0.63]; n = 6 (experiments); n = 3 (papers)). The impaired glycemic control following perfluoro-octane sulphonic acid (PFOS) exposure and enhanced mass loss following dichlorodiphenyltrichloroethane (DDT) exposure have not been replicated. Animal studies thus suggest some chemical groups, especially PCB and PFOS, could impair glucose control management during calorie restriction, similar to conclusions from limited existing clinical studies. We discuss the research that is urgently required to inform weight management services that are now the mainstay prevention initiative for T2DM.

Enhancing reporting through structure: a before and after study on the effectiveness of SPIRIT-based templates to improve the completeness of reporting of randomized controlled trial protocols

BACKGROUND

Despite the improvements in the completeness of reporting of randomized trial protocols after the publication of the Standard Protocol Items: Recommendations for Interventional Trial (SPIRIT) guidelines, many items remain poorly reported. This study aimed to assess the effectiveness of using SPIRIT-tailored templates for trial protocols to improve the completeness of reporting of the protocols that master's students write as part of their master's theses.

METHODS

Before and after experimental study performed at the University Master's Degree in Orthopaedic Manual Physiotherapy of the Universitat Internacional de Catalunya (Barcelona, Spain). While students in the post-intervention period were instructed to use a trial protocol template that was tailored to SPIRIT, students in the pre-intervention period did not use the template.

PRIMARY OUTCOME

Difference between the pre- and post-intervention periods in the mean number of adequately reported items (0-10 scale). The outcomes were evaluated independently and in duplicate by two blinded assessors. Students and their supervisors were not aware that they were part of a research project. For the statistical analysis, we used a generalized linear regression model (dependent variable: number of adequately reported items in the protocol; independent variables: intervention period, call, language).

RESULTS

Thirty-four trial protocols were included (17, pre-intervention; 17, post-intervention). Protocols produced during the post-intervention period (mean: 8.24; SD: 1.52) were more completely reported than those produced during the pre-intervention period (mean: 6.35; SD: 1.80); adjusted difference: 1.79 (95% CI: 0.58 to 3.00).

CONCLUSIONS

SPIRIT-based templates could be used to improve the completeness of reporting of randomized trial protocols.

Twelve years after the ARRIVE guidelines: Animal research has not yet arrived at high standards

The reproducibility crisis across animal studies jeopardizes the credibility of the main findings derived from animal research, even though these findings are critical for informing human studies. To clarify and improve transparency among animal studies, the ARRIVE reporting guidelines were first announced in 2010 and upgraded to version 2.0 in 2020. However, compliance with and awareness of those reporting guidelines has remained suboptimal. Journal editors should encourage the authors to adhere to those guidelines. Authors, editors, referees, and reviewers should be aware of the ARRIVE guideline 2.0 when assessing and evaluating the methodology and findings of animal studies. However, we should also question whether reporting guidelines alone can change a research culture and improve the reproducibility of animal investigations. Reported research may not reflect actual research. Large segments of animal research efforts are wasted because of poor design choices and because of non-publication rather than suboptimal reporting. Better training of the scientific workforce, interventions at improving animal research at the design stage, registration practices, and alignment of the reward system with the publication of rigorous animal research may achieve more than reporting guidelines alone.

A minimal metadata set (MNMS) to repurpose nonclinical in vivo data for biomedical research

Although biomedical research is experiencing a data explosion, the accumulation of vast quantities of data alone does not guarantee a primary objective for science: building upon existing knowledge. Data collected that lack appropriate metadata cannot be fully interrogated or integrated into new research projects, leading to wasted resources and missed opportunities for data repurposing. This issue is particularly acute for research using animals, where concerns regarding data reproducibility and ensuring animal welfare are paramount. Here, to address this problem, we propose a minimal metadata set (MNMS) designed to enable the repurposing of in vivo data. MNMS aligns with an existing validated guideline for reporting in vivo data (ARRIVE 2.0) and contributes to making in vivo data FAIR-compliant. Scenarios where MNMS should be implemented in diverse research environments are presented, highlighting opportunities and challenges for data repurposing at different scales. We conclude with a 'call for action' to key stakeholders in biomedical research to adopt and apply MNMS to accelerate both the advancement of knowledge and the betterment of animal welfare.

Practical approaches to improving translatability and reproducibility in preclinical pain research

Pain research continues to face the challenge of poor translatability of pre-clinical studies. In this short primer, we are summarizing the possible causes, with an emphasis on practical and constructive solutions. In particular, we stress the importance of increased heterogeneity in animal studies; formal or informal pre-registration to combat publication bias; and increased statistical training in order to help pre-clinical scientists appreciate the usefulness of available experimental design and reporting guidelines.

Open Science 2.0: Towards a truly collaborative research ecosystem

Conversations about open science have reached the mainstream, yet many open science practices such as data sharing remain uncommon. Our efforts towards openness therefore need to increase in scale and aim for a more ambitious target. We need an ecosystem not only where research outputs are openly shared but also in which transparency permeates the research process from the start and lends itself to more rigorous and collaborative research. To support this vision, this Essay provides an overview of a selection of open science initiatives from the past 2 decades, focusing on methods transparency, scholarly communication, team science, and research culture, and speculates about what the future of open science could look like. It then draws on these examples to provide recommendations for how funders, institutions, journals, regulators, and other stakeholders can create an environment that is ripe for improvement.

Addressing the elephant in the T-maze: Developing experimental conduct guidelines for neuroscience

Personal view - no abstract.

European Society of Sexual Medicine consensus statement on the use of animal models for studying Peyronie's disease

Introduction

Animal models are frequently used for translational research in Peyronie's disease (PD). However, due to lack of availability of guidelines, there is some heterogeneity in study design, data reporting, and outcome measures.

Aim

This European Society for Sexual Medicine consensus statement aims to provide guidance in utilization of animal models in PD research in a standardized and uniform fashion.

Methods

PubMed was searched for studies using animal models for PD. The following search terms were used: ("Peyronie's disease" OR "penile fibrosis" OR "penile curvature" OR "induration penis plastica" OR "erectile dysfunction") AND ("rodent" OR "mouse" OR "mice" OR "rat" OR "rabbit").

Outcomes

This European Society for Sexual Medicine statement describes best practice guidelines for utilization of animals in PD research: power calculation, details of available models, surgical procedures, and measurement techniques, while highlighting possible pitfalls and translational limitations of the models.

Results

In total, 2490 studies were retrieved and 2446 articles were excluded. A total of 44 studies were included, of which 40 studies used rats, 1 study used both rats and mice, 1 study used a genetic mouse model, and 2 studies used rabbits. A significant number of the studies (70.5%) used transforming growth factor β 1 for induction of fibrosis. Oxford 2011 Levels of Evidence criteria could not be applied due to the nature of the studies.

Conclusion

Despite certain limitations of PD animal models presented, we aimed to provide guidance for their appropriate use in translational research, with the purpose of improving study quality and reproducibility as well as facilitating interpretation of reported results and conclusions.

Mapping strategies towards improved external validity in preclinical translational research

INTRODUCTION

Translation is about successfully bringing findings from preclinical contexts into the clinic. This transfer is challenging as clinical trials frequently fail despite positive preclinical results. Limited robustness of preclinical research has been marked as one of the drivers of such failures. One suggested solution is to improve the external validity of in vitro and in vivo experiments via a suite of complementary strategies.

AREAS COVERED

In this review, the authors summarize the literature available on different strategies to improve external validity in in vivo, in vitro, or ex vivo experiments; systematic heterogenization; generalizability tests; and multi-batch and multicenter experiments. Articles that tested or discussed sources of variability in systematically heterogenized experiments were identified, and the most prevalent sources of variability are reviewed further. Special considerations in sample size planning, analysis options, and practical feasibility associated with each strategy are also reviewed.

EXPERT OPINION

The strategies reviewed differentially influence variation in experiments. Different research projects, with their unique goals, can leverage the strengths and limitations of each strategy. Applying a combination of these approaches in confirmatory stages of preclinical research putatively increases the chances of success in clinical studies.

Improved Wound Healing and Skin Regeneration Ability of 3,2'-Dihydroxyflavone-Treated Mesenchymal Stem Cell-Derived Extracellular Vesicles

Flavonoids enhance the self-renewal and differentiation potential of mesenchymal stem cells (MSCs) and have therapeutic activities, including regenerative, anti-oxidative, and anti-inflammatory effects. Recent studies have revealed that MSC-derived extracellular vesicles (MSC-EVs) have therapeutic effects on tissue regeneration and inflammation. To facilitate further research on the therapeutic potential of MSC-EVs derived from flavonoid-treated MSCs, we surveyed the production of EVs and their therapeutic applications in wound regeneration. MSCs treated with flavonoids enhanced EV production twofold compared with naïve MSCs. EVs produced by MSCs treated with flavonoids (Fla-EVs) displayed significant anti-inflammatory and wound-healing effects in vitro. The wound-healing capacity of EVs was mediated by the upregulation of mitogen-activated protein kinase kinase (MEK)/extracellular signal-regulated kinase (ERK) signaling. Interestingly, the protein level of p-ERK under inhibition of MEK signals was maintained in Fla-EV-treated fibroblasts, suggesting that Fla-EVs have a higher therapeutic potential than naïve MSC-EVs (Cont-EVs) in wound healing. Moreover, the in vivo wound closure effect of the Fla-EVs showed significant improvement compared with that of the flavonoid-only treatment group and the Cont-EVs. This study provides a strategy for the efficient production of EVs with superior therapeutic potential using flavonoids.

Statistical review of animal trials-A guideline

Any experiment involving living organisms requires justification of the need and moral defensibleness of the study. Statistical planning, design, and sample size calculation of the experiment are no less important review criteria than general medical and ethical points to consider. Errors made in the statistical planning and data evaluation phase can have severe consequences on both results and conclusions. They might proliferate and thus impact future trials-an unintended outcome of fundamental research with profound ethical consequences. Unified statistical standards are currently missing for animal review boards in Germany. In order to accompany, we developed a biometric form to be filled and handed in with the proposal at the concerned local authority on animal welfare. It addresses relevant points to consider for biostatistical planning of animal experiments and can help both the applicants and the reviewers in overseeing the entire experiment(s) planned. Furthermore, the form might also aid in meeting the current standards set by the 3+3R's principle of animal experimentation: Replacement, Reduction, Refinement as well as Robustness, Registration, and Reporting. The form has already been in use by the concerned local authority of animal welfare in Berlin, Germany. In addition, we provide reference to our user guide giving more detailed explanation and examples for each section of the biometric form. Unifying the set of biostatistical aspects will help both the applicants and the reviewers to equal standards and increase quality of preclinical research projects, also for translational, multicenter, or international studies.

Recommendations for robust and reproducible preclinical research in personalised medicine

BACKGROUND

Personalised medicine is a medical model that aims to provide tailor-made prevention and treatment strategies for defined groups of individuals. The concept brings new challenges to the translational step, both in clinical relevance and validity of models. We have developed a set of recommendations aimed at improving the robustness of preclinical methods in translational research for personalised medicine.

METHODS

These recommendations have been developed following four main steps: (1) a scoping review of the literature with a gap analysis, (2) working sessions with a wide range of experts in the field, (3) a consensus workshop, and (4) preparation of the final set of recommendations.

RESULTS

Despite the progress in developing innovative and complex preclinical model systems, to date there are fundamental deficits in translational methods that prevent the further development of personalised medicine. The literature review highlighted five main gaps, relating to the relevance of experimental models, quality assessment practices, reporting, regulation, and a gap between preclinical and clinical research. We identified five points of focus for the recommendations, based on the consensus reached during the consultation meetings: (1) clinically relevant translational research, (2) robust model development, (3) transparency and education, (4) revised regulation, and (5) interaction with clinical research and patient engagement. Here, we present a set of 15 recommendations aimed at improving the robustness of preclinical methods in translational research for personalised medicine.

CONCLUSIONS

Appropriate preclinical models should be an integral contributor to interventional clinical trial success rates, and predictive translational models are a fundamental requirement to realise the dream of personalised medicine. The implementation of these guidelines is ambitious, and it is only through the active involvement of all relevant stakeholders in this field that we will be able to make an impact and effectuate a change which will facilitate improved translation of personalised medicine in the future.

Identifying barriers and enablers to rigorous conduct and reporting of preclinical laboratory studies

Use of rigorous study design methods and transparent reporting in publications are 2 key strategies proposed to improve the reproducibility of preclinical research. Despite promotion of these practices by funders and journals, assessments suggest uptake is low in preclinical research. Thirty preclinical scientists were interviewed to better understand barriers and enablers to rigorous design and reporting. The interview guide was informed by the Theoretical Domains Framework, which is a framework used to understand determinants of current and desired behavior. Four global themes were identified; 2 reflecting enablers and 2 reflecting barriers. We found that basic scientists are highly motivated to apply the methods of rigorous design and reporting and perceive a number of benefits to their adoption (e.g., improved quality and reliability). However, there was varied awareness of the guidelines and in implementation of these practices. Researchers also noted that these guidelines can result in disadvantages, such as increased sample sizes, expenses, time, and can require several personnel to operationalize. Most researchers expressed additional resources such as personnel and education/training would better enable the application of some methods. Using existing guidance (Behaviour Change Wheel (BCW); Expert Recommendations for Implementing Change (ERIC) project implementation strategies), we mapped and coded our interview findings to identify potential interventions, policies, and implementation strategies to improve routine use of the guidelines by preclinical scientists. These findings will help inform specific strategies that may guide the development of programs and resources to improve experimental design and transparent reporting in preclinical research.

The potential of gene therapies for spinal cord injury repair: a systematic review and meta-analysis of pre-clinical studies

Currently, there is no cure for traumatic spinal cord injury but one therapeutic approach showing promise is gene therapy. In this systematic review and meta-analysis, we aim to assess the efficacy of gene therapies in pre-clinical models of spinal cord injury and the risk of bias. In this meta-analysis, registered at PROSPERO (Registration ID: CRD42020185008), we identified relevant controlled in vivo studies published in English by searching the PubMed, Web of Science, and Embase databases. No restrictions of the year of publication were applied and the last literature search was conducted on August 3, 2020. We then conducted a random-effects meta-analysis using the restricted maximum likelihood estimator. A total of 71 studies met our inclusion criteria and were included in the systematic review. Our results showed that overall, gene therapies were associated with improvements in locomotor score (standardized mean difference [SMD]: 2.07, 95% confidence interval [CI]:1.68–2.47, Tau² = 2.13, I² = 83.6%) and axonal regrowth (SMD: 2.78, 95%CI: 1.92–3.65, Tau² = 4.13, I² = 85.5%). There was significant asymmetry in the funnel plots of both outcome measures indicating the presence of publication bias. We used a modified CAMARADES (Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies) checklist to assess the risk of bias, finding that the median score was 4 (IQR:3–5). In particular, reports of allocation concealment and sample size calculations were lacking. In conclusion, gene therapies are showing promise as therapies for spinal cord injury repair, but there is no consensus on which gene or genes should be targeted.

[The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsDeclaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas]

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

Current knowledge, challenges and innovations in developmental pharmacology: A combined conect4children Expert Group and European Society for Developmental, Perinatal and Paediatric Pharmacology White Paper

Developmental pharmacology describes the impact of maturation on drug disposition (pharmacokinetics, PK) and drug effects (pharmacodynamics, PD) throughout the paediatric age range. This paper, written by a multidisciplinary group of experts, summarizes current knowledge, and provides suggestions to pharmaceutical companies, regulatory agencies and academicians on how to incorporate the latest knowledge regarding developmental pharmacology and innovative techniques into neonatal and paediatric drug development. Biological aspects of drug absorption, distribution, metabolism and excretion throughout development are summarized. Although this area made enormous progress during the last two decades, remaining knowledge gaps were identified. Minimal risk and burden designs allow for optimally informative but minimally invasive PK sampling, while concomitant profiling of drug metabolites may provide additional insight in the unique PK behaviour in children. Furthermore, developmental PD needs to be considered during drug development, which is illustrated by disease- and/or target organ-specific examples. Identifying and testing PD targets and effects in special populations, and application of age- and/or population-specific assessment tools are discussed. Drug development plans also need to incorporate innovative techniques such as preclinical models to study therapeutic strategies, and shift from sequential enrolment of subgroups, to more rational designs. To stimulate appropriate research plans, illustrations of specific PK/PD-related as well as drug safety-related challenges during drug development are provided. The suggestions made in this joint paper of the Innovative Medicines Initiative conect4children Expert group on Developmental Pharmacology and the European Society for Developmental, Perinatal and Paediatric Pharmacology, should facilitate all those involved in drug development.

A methodological review with meta-epidemiological analysis of preclinical systematic reviews with meta-analyses

Systematic reviews and meta-analyses have been proposed as an approach to synthesize the literature and counteract the lack of power of small preclinical studies. We aimed to evaluate (1) the methodology of these reviews, (2) the methodological quality of the studies they included and (3) whether study methodological characteristics affect effect size. We searched MEDLINE to retrieve 212 systematic reviews with meta-analyses of preclinical studies published from January, 2018 to March, 2020. Less than 15% explored the grey literature. Selection, data extraction and risk of bias assessment were performed in duplicate in less than two thirds of reviews. Most of them assessed the methodological quality of included studies and reported the meta-analysis model. The risk of bias of included studies was mostly rated unclear. In meta-epidemiological analysis, none of the study methodological characteristics was associated with effect size. The methodological characteristics of systematic reviews with meta-analyses of recently published preclinical studies seem to have improved as compared with previous assessments, but the methodological quality of included studies remains poor, thus limiting the validity of their results. Our meta-epidemiological analysis did not show any evidence of a potential association between methodological characteristics of included studies and effect size.

Reporting quality in preclinical animal experimental research in 2009 and 2018: A nationwide systematic investigation

Lack of translation and irreproducibility challenge preclinical animal research. Insufficient reporting methodologies to safeguard study quality is part of the reason. This nationwide study investigates the reporting prevalence of these methodologies and scrutinizes the reported information’s level of detail. Publications were from two time periods to convey any reporting progress and had at least one author affiliated to a Danish University. We retrieved all relevant animal experimental studies using a predefined research protocol and a systematic search. A random sampling of 250 studies from 2009 and 2018 led to 500 publications in total. Reporting of measures known to impact study results estimates were assessed. Part I discloses a simplified two-level scoring “yes/no” to identify the presence of reporting. Part II demonstrates an additional three-level scoring to analyze the reported information’s level of detail. Overall reporting prevalence is low, although minor improvements are noted. Reporting of randomization increased from 24.0% in 2009 to 40.8% in 2018, blinded experiment conduct from 2.4% to 4.4%, blinded outcome assessment from 23.6% to 38.0%, and sample size calculation from 3.2% to 14.0%. Poor reporting of details is striking with reporting of the random allocation method to groups being only 1.2% in 2009 and 6.0% in 2018. Reporting of sample size calculation method was 2.4% in 2009 and 7.6% in 2018. Only conflict-of-interest statements reporting increased from 37.6% in 2009 to 90.4%. Measures safeguarding study quality are poorly reported in publications affiliated with Danish research institutions. Only a modest improvement was noted during the period 2009–2018, and the lack of details urgently prompts institutional strategies to accelerate this. We suggest thorough teaching in designing, conducting and reporting animal studies. Education in systematic review methodology should be implemented in this training and will increase motivation and behavior working towards quality improvements in science.

'Invisible actors'-How poor methodology reporting compromises mouse models of oncology: A cross-sectional survey

The laboratory mouse is a key player in preclinical oncology research. However, emphasis of techniques reporting at the expense of critical animal-related detail compromises research integrity, animal welfare, and, ultimately, the translation potential of mouse-based oncology models. To evaluate current reporting practices, we performed a cross-sectional survey of 400 preclinical oncology studies using mouse solid-tumour models. Articles published in 2020 were selected from 20 journals that specifically endorsed the ARRIVE (Animal Research: Reporting of In Vivo Experiments) preclinical reporting guidelines. We assessed reporting compliance for 22 items in five domains: ethical oversight assurance, animal signalment, husbandry, welfare, and euthanasia. Data were analysed using hierarchical generalised random-intercept models, clustered on journal. Overall, reporting of animal-related items was poor. Median compliance over all categories was 23%. There was little or no association between extent of reporting compliance and journal or journal impact factor. Age, sex, and source were reported most frequently, but verifiable strain information was reported for <10% of studies. Animal husbandry, housing environment, and welfare items were reported by <5% of studies. Fewer than one in four studies reported analgesia use, humane endpoints, or an identifiable method of euthanasia. Of concern was the poor documentation of ethical oversight information. Fewer than one in four provided verifiable approval information, and almost one in ten reported no information, or information that was demonstrably false. Mice are the "invisible actors" in preclinical oncology research. In spite of widespread endorsement of reporting guidelines, adherence to reporting guidelines on the part of authors is poor and journals fail to enforce guideline reporting standards. In particular, the inadequate reporting of key animal-related items severely restricts the utility and translation potential of mouse models, and results in research waste. Both investigators and journals have the ethical responsibility to ensure animals are not wasted in uninformative research.

A guide to open science practices for animal research

Translational biomedical research relies on animal experiments and provides the underlying proof of practice for clinical trials, which places an increased duty of care on translational researchers to derive the maximum possible output from every experiment performed. The implementation of open science practices has the potential to initiate a change in research culture that could improve the transparency and quality of translational research in general, as well as increasing the audience and scientific reach of published research. However, open science has become a buzzword in the scientific community that can often miss mark when it comes to practical implementation. In this Essay, we provide a guide to open science practices that can be applied throughout the research process, from study design, through data collection and analysis, to publication and dissemination, to help scientists improve the transparency and quality of their work. As open science practices continue to evolve, we also provide an online toolbox of resources that we will update continually.

Open science has become a buzzword in the scientific community that too often misses the practical application for individual researchers. This Essay, provides a guide to choosing the most appropriate tools to make animal research more transparent.

Insufficient reporting of experimental variables as a cause for nonreproducibility in animal physiology? A case study

Nonreproducibility in scientific investigations has been explained by inadequately reporting methodology, honest error, and even misconduct. We hypothesized that, within the field of animal physiology, the most parsimonious explanation for nonreproducibility is inadequate reporting of key methodological details. We further hypothesized that implementation of relatively recently released reporting guidelines has positively impacted journal article quality, as measured by completeness of the methodology descriptions. We analyzed 84 research articles published in five primarily organismal animal physiology journals in 2008-2010 (generally before current guidelines) and 2018-2020. Compliance for reporting 34 variables referring to biology, experiments, and data collection was assessed. Reporting compliance was just ∼61% in 2008-2010, rising only slightly to 67.5% for 2018-2020. Only 21% of the reported variables showed significant differences across the period from 2008-2020. We conclude that, despite attempts by societies and journals to promote greater reporting compliance, such efforts have so far been relatively unsuccessful in the field of animal physiology.

Planning experiments: Updated guidance on experimental design and analysis and their reporting III

Scientists who plan to publish in British Journal of Pharmacology (BJP) must read this article before undertaking a study. This editorial provides guidance for the design of experiments. We have published previously two guidance documents on experimental design and analysis (Curtis et al., 2015; Curtis et al., 2018). This update clarifies and simplifies the requirements on design and analysis for BJP manuscripts. This editorial also details updated requirements following an audit and discussion on best practice by the BJP editorial board. Explanations for the requirements are provided in the previous articles. Here, we address new issues that have arisen in the course of handling manuscripts and emphasise three aspects of design that continue to present the greatest challenge to authors: randomisation, blinded analysis and balance of group sizes.

Replacing bar graphs of continuous data with more informative graphics: Are we making progress?

Recent work has raised awareness about the need to replace bar graphs of continuous data with informative graphs showing the data distribution. The impact of these efforts is not known. The present observational meta-research study examined how often scientists in different fields use various graph types, and assessed whether visualization practices have changed between 2010 and 2020. We developed and validated an automated screening tool, designed to identify bar graphs of counts or proportions, bar graphs of continuous data, bar graphs with dot plots, dot plots, box plots, violin plots, histograms, pie charts, and flow charts. Papers from 23 fields (approximately 1000 papers/field per year) were randomly selected from PubMed Central and screened (n=227998). F1 scores for different graphs ranged between 0.83 and 0.95 in the internal validation set. While the tool also performed well in external validation sets, F1 scores were lower for uncommon graphs. Bar graphs are more often used incorrectly to display continuous data than they are used correctly to display counts or proportions. The proportion of papers that use bar graphs of continuous data varies markedly across fields (range in 2020: 4–58%), with high rates in biochemistry and cell biology, complementary and alternative medicine, physiology, genetics, oncology and carcinogenesis, pharmacology, microbiology and immunology. Visualization practices have improved in some fields in recent years. Fewer than 25% of papers use flow charts, which provide information about attrition and the risk of bias. The present study highlights the need for continued interventions to improve visualization and identifies fields that would benefit most.

Animal welfare requirements in publishing guidelines

Descriptions of measures taken to optimize animal welfare are often absent from scientific reports of animal experiments. One reason may be that journal guidelines inadequately compel authors to provide such information. In this study, online English language versions of the 'Guidelines to authors' (GTAs) from 54 national biomedical journals were examined for neutral (unrelated to welfare) and non-neutral keywords referring to: animal welfare; the '3Rs'; the ARRIVE (2010) guidelines, and regulations pertaining to animal experimentation. Journals were selected from nine countries (UK, US, China, Canada, India, Brazil, Germany, Japan and Australia) and seven biomedical specialties (oncology, rheumatology, surgery, pharmacology, medicine, anaesthesia and veterinary medicine). Total GTA word counts varied from 1137 to 31,609. The keyword count identified per category were expressed per myriad (10,000) of total word count. One-way analyses of variance followed by post hoc Tukey pairwise comparisons revealed greater non-neutral per myriad word counts for (a) veterinary GTAs compared with medicine, oncology, rheumatology or surgery; (b) British, compared with Australian, Canadian, German and Japanese GTAs; and (c) no differences between non-neutral categories. The English language versions of GTAs of British and veterinary medical journals contain more words associated with animal welfare, the 3Rs and the ARRIVE guidelines than those from eight other countries and six other medical specialities. The exclusion of 'national' language versions from analysis precludes attempts to identify national differences in attitudes to laboratory animal welfare.

Is the future of peer review automated?

The rising rate of preprints and publications, combined with persistent inadequate reporting practices and problems with study design and execution, have strained the traditional peer review system. Automated screening tools could potentially enhance peer review by helping authors, journal editors, and reviewers to identify beneficial practices and common problems in preprints or submitted manuscripts. Tools can screen many papers quickly, and may be particularly helpful in assessing compliance with journal policies and with straightforward items in reporting guidelines. However, existing tools cannot understand or interpret the paper in the context of the scientific literature. Tools cannot yet determine whether the methods used are suitable to answer the research question, or whether the data support the authors' conclusions. Editors and peer reviewers are essential for assessing journal fit and the overall quality of a paper, including the experimental design, the soundness of the study's conclusions, potential impact and innovation. Automated screening tools cannot replace peer review, but may aid authors, reviewers, and editors in improving scientific papers. Strategies for responsible use of automated tools in peer review may include setting performance criteria for tools, transparently reporting tool performance and use, and training users to interpret reports.

Improving the reproducibility and integrity of research: what can different stakeholders contribute?

Increasing awareness of problems with the reproducibility and integrity of research led the UK Parliament Science and Technology Committee to launch, in July 2021, an inquiry into reproducibility and research integrity. We recognise at least four potential reasons why attempts to replicate a research finding may be unsuccessful: false positive statistical analyses, low generalisability of findings, suboptimal study designs (research integrity), and deliberate malfeasance (researcher integrity). It is important to make a distinction between the contributions of research integrity and of researcher integrity to the reproducibility crisis. While the impact of an individual instance of compromised researcher integrity is substantial, the aggregate impact of more prevalent problems with research integrity is likely much greater. The research community will be most efficient when failed replication efforts are never due to issues of research integrity or of researcher integrity, as this would allow focus on the scientific reasons for why two apparently similar experiments should reach different conclusions. We discuss the role of funders, institutions and government in addressing the “reproducibility crisis” before considering which interventions might have a positive impact on academia’s approach to reproducible research, and a possible role for a committee on research integrity.

Weaknesses in Experimental Design and Reporting Decrease the Likelihood of Reproducibility and Generalization of Recent Cardiovascular Research

Recent evidence indicates that many clinical and preclinical studies are not reproducible. Prominent causes include design and implementation issues, low statistical power, unintentional bias, and incomplete reporting in the published literature. The primary goal of this study was to assess the quality of published research in three prominent cardiovascular research journals by examining statistical power and assessing the adherence to augmented ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments). For unpaired t-tests, the average median power for a 20% and 50% change was 0.27 ± 0.06 and 0.88 ± 0.08, respectively. For analysis of guidelines, 40 categories were assessed with a 0-2 scale. Although many strengths were observed, several key elements that were needed for reproducibility were inadequate, including differentiation of primary and secondary outcomes, power calculations for group size, allocation methods, use of randomization and blinding, checks for normality, reports of attrition, and adverse events of subjects, and assessment of bias. A secondary goal was to examine whether a required checklist improved the quality of reporting; those results indicated that a checklist improved compliance and quality of reporting, but adequacy levels in key categories were still too low. Overall, the findings of this study indicated that the probability for reproducibility of many clinical and preclinical cardiovascular research studies was low because of incomplete reporting, low statistical power, and lack of research practices that decrease experimental bias. Expansion of group sizes to increase power, use of detailed checklists, and closer monitoring for checklist adherence by editors and journals should remediate many of these deficits and increase the likelihood of reproducibility.

Misidentified biomedical resources: Journal guidelines are not a quick fix

Biomedical researchers routinely use a variety of biological models and resources, such as cultured cell lines, antibodies and laboratory animals. Unfortunately, these resources are not flawless: cell lines can be misidentified; for antibodies, problems with specificity, lot‐to‐lot consistency and sensitivity are common; and the reliability of animal models is questioned due to poor translation of animal studies to human clinical trials. In some cases, these problems can render the results of a study meaningless. As a response, some journals have implemented guidelines regarding the use and reporting of cell lines, antibodies and laboratory animals. In our study we use a portfolio of existing and newly created datasets to investigate identification and authentication information of cell lines, antibodies and organisms before and after guideline introduction, compared to journals without guidelines. We observed a general improvement of reporting quality over time, which the implementation of guidelines accelerated only in some cases. We therefore conclude that the effectiveness of journal guidelines is likely to be context dependent, affected by factors such as implementation conditions, research community support and monitoring and resource availability. Hence, journal reporting guidelines in themselves are not a quick fix to repair shortcomings in biomedical resource documentation, even though they can be part of the solution.

Risk of bias assessment in preclinical literature using natural language processing

We sought to apply natural language processing to the task of automatic risk of bias assessment in preclinical literature, which could speed the process of systematic review, provide information to guide research improvement activity, and support translation from preclinical to clinical research. We use 7840 full‐text publications describing animal experiments with yes/no annotations for five risk of bias items. We implement a series of models including baselines (support vector machine, logistic regression, random forest), neural models (convolutional neural network, recurrent neural network with attention, hierarchical neural network) and models using BERT with two strategies (document chunk pooling and sentence extraction). We tune hyperparameters to obtain the highest F1 scores for each risk of bias item on the validation set and compare evaluation results on the test set to our previous regular expression approach. The F1 scores of best models on test set are 82.0% for random allocation, 81.6% for blinded assessment of outcome, 82.6% for conflict of interests, 91.4% for compliance with animal welfare regulations and 46.6% for reporting animals excluded from analysis. Our models significantly outperform regular expressions for four risk of bias items. For random allocation, blinded assessment of outcome, conflict of interests and animal exclusions, neural models achieve good performance; for animal welfare regulations, BERT model with a sentence extraction strategy works better. Convolutional neural networks are the overall best models. The tool is publicly available which may contribute to the future monitoring of risk of bias reporting for research improvement activities.

GoodReports: developing a website to help health researchers find and use reporting guidelines

BACKGROUND

Th EQUATOR Network improves the quality and transparency in health research, primarily by promoting awareness and use of reporting guidelines. In 2018, the UK EQUATOR Centre launched GoodReports.org , a website that helps authors find and use reporting guidelines. This paper describes the tool's development so far. We describe user experience and behaviour of using GoodReports.org both inside and outside a journal manuscript submission process. We intend to use our findings to inform future development and testing of the tool.

METHODS

We conducted a survey to collect data on user experience of the GoodReports website. We cross-checked a random sample of 100 manuscripts submitted to a partner journal to describe the level of agreement between the tool's checklist recommendation and what we would have recommended. We compared the proportion of authors submitting a completed reporting checklist alongside their manuscripts between groups exposed or not exposed to the GoodReports tool. We also conducted a study comparing completeness of reporting of manuscript text before an author received a reporting guideline recommendation from GoodReports.org with the completeness of the text subsequently submitted to a partner journal.

RESULTS

Seventy percent (423/599) of survey respondents rated GoodReports 8 or more out of 10 for usefulness, and 74% (198/267) said they had made changes to their manuscript after using the website. We agreed with the GoodReports reporting guideline recommendation in 84% (72/86) of cases. Of authors who completed the guideline finder questionnaire, 14% (10/69) failed to submit a completed checklist compared to 30% (41/136) who did not use the tool. Of the 69 authors who received a GoodReports reporting guideline recommendation, 20 manuscript pairs could be reviewed before and after use of GoodReports. Five included more information in their methods section after exposure to GoodReports. On average, authors reported 57% of necessary reporting items before completing a checklist on GoodReports.org and 60% after.

CONCLUSION

The data suggest that reporting guidance is needed early in the writing process, not at submission stage. We are developing GoodReports by adding more reporting guidelines and by creating editable article templates. We will test whether GoodReports users write more complete study reports in a randomised trial targeting researchers starting to write health research articles.

Improving rigor and reproducibility in nonhuman primate research

Nonhuman primates (NHPs) are a critical component of translational/preclinical biomedical research due to the strong similarities between NHP and human physiology and disease pathology. In some cases, NHPs represent the most appropriate, or even the only, animal model for complex metabolic, neurological, and infectious diseases. The increased demand for and limited availability of these valuable research subjects requires that rigor and reproducibility be a prime consideration to ensure the maximal utility of this scarce resource. Here, we discuss a number of approaches that collectively can contribute to enhanced rigor and reproducibility in NHP research.

Nonhuman primates (NHP) are a crucial component of biomedical research due to their similarities with human physiology and pathobiology.

Increased rigor and reproducibility in NHP research are critical to maximizing the information obtained from this scarce and valuable resource.

More widespread adoption of approaches such as normative protocols, preregistration, and data sharing, as well as more extensive training in biostatistics, can enhance rigor and reproducibility in NHP studies.

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews. Full English text available from:www.revespcardiol.org/en.

AK, Pennisi CP. Quality of Reporting in Preclinical Urethral Tissue Engineering Studies: A Systematic Review to Assess Adherence to the ARRIVE Guidelines

Preclinical research within the area of urethral tissue engineering has not yet been successfully translated into an efficient therapeutic option for patients. This gap could be attributed, in part, to inadequate design and reporting of the studies employing laboratory animals. In this study, a systematic review was conducted to investigate the quality of reporting in preclinical studies utilizing tissue engineering approaches for urethral repair. The scope was on studies performed in rabbits, published between January 2014 and March 2020. Quality assessment of the data was conducted according to the Animal Research: Reporting of in Vivo Experiments (ARRIVE) guidelines by the scoring of a 38-item checklist in different categories. A total of 28 articles that fulfilled the eligibility criteria were included in the study. The range of ARRIVE score was from 0 to 100, taking into consideration having reported the item in question or not. The mean checklist score was 53%. The items that attained the highest scores included the number of animals utilized, the size of control and experimental groups, and the definition of experimental outcomes. The least frequently reported items included the data regarding the experimental procedure, housing and husbandry, determination and justification of the number of animals, and reporting of adverse events. Surprisingly, full disclosure about ethical guidelines and animal protocol approval was missing in 54% of the studies. No paper stated the sample size estimation. Overall, our study found that a large number of studies display inadequate reporting of fundamental information and that the quality of reporting improved marginally over the study period. We encourage a comprehensive implementation of the ARRIVE guidelines in animal studies exploring tissue engineering for urethral repair, not only to facilitate effective translation of preclinical research findings into clinical therapies, but also to ensure compliance with ethical principles and to minimize unnecessary animal studies.

Internal consistency and compatibility of the 3Rs and 3Vs principles for project evaluation of animal research

Using animals for research raises ethical concerns that are addressed in project evaluation by weighing expected harm to animals against expected benefit to society. A harm-benefit analysis (HBA) relies on two preconditions: (a) the study protocol is scientifically suitable and (b) the use of (sentient) animals and harm imposed on them are necessary for achieving the study's aims. The 3Rs (Replace, Reduce and Refine) provide a guiding principle for evaluating whether the use of animals, their number and the harm imposed on them are necessary. A similar guiding principle for evaluating whether a study protocol is scientifically suitable has recently been proposed: the 3Vs principle referring to the three main aspects of scientific validity in animal research (construct, internal and external validity). Here, we analyse the internal consistency and compatibility of these two principles, address conflicts within and between the 3Rs and 3Vs principles and discuss their implications for project evaluation. We show that a few conflicts and trade-offs exist, but that these can be resolved either by appropriate study designs or by ethical deliberation in the HBA. In combination, the 3Vs, 3Rs and the HBA thus offer a coherent framework for a logically structured evaluation procedure to decide about the legitimacy of animal research projects.

Translational Block in Stroke: A Constructive and "Out-of-the-Box" Reappraisal

Why can we still not translate preclinical research to clinical treatments for acute strokes? Despite > 1000 successful preclinical studies, drugs, and concepts for acute stroke, only two have reached clinical translation. This is the translational block. Yet, we continue to routinely model strokes using almost the same concepts we have used for over 30 years. Methodological improvements and criteria from the last decade have shed some light but have not solved the problem. In this conceptual analysis, we review the current status and reappraise it by thinking "out-of-the-box" and over the edges. As such, we query why other scientific fields have also faced the same translational failures, to find common denominators. In parallel, we query how migraine, multiple sclerosis, and hypothermia in hypoxic encephalopathy have achieved significant translation successes. Should we view ischemic stroke as a "chronic, relapsing, vascular" disease, then secondary prevention strategies are also a successful translation. Finally, based on the lessons learned, we propose how stroke should be modeled, and how preclinical and clinical scientists, editors, grant reviewers, and industry should reconsider their routine way of conducting research. Translational success for stroke treatments may eventually require a bold change with solutions that are outside of the box.

A descriptive analysis of the data availability statements accompanying medRxiv preprints and a comparison with their published counterparts

OBJECTIVE

To determine whether medRxiv data availability statements describe open or closed data-that is, whether the data used in the study is openly available without restriction-and to examine if this changes on publication based on journal data-sharing policy. Additionally, to examine whether data availability statements are sufficient to capture code availability declarations.

DESIGN

Observational study, following a pre-registered protocol, of preprints posted on the medRxiv repository between 25th June 2019 and 1st May 2020 and their published counterparts.

MAIN OUTCOME MEASURES

Distribution of preprinted data availability statements across nine categories, determined by a prespecified classification system. Change in the percentage of data availability statements describing open data between the preprinted and published versions of the same record, stratified by journal sharing policy. Number of code availability declarations reported in the full-text preprint which were not captured in the corresponding data availability statement.

RESULTS

3938 medRxiv preprints with an applicable data availability statement were included in our sample, of which 911 (23.1%) were categorized as describing open data. 379 (9.6%) preprints were subsequently published, and of these published articles, only 155 contained an applicable data availability statement. Similar to the preprint stage, a minority (59 (38.1%)) of these published data availability statements described open data. Of the 151 records eligible for the comparison between preprinted and published stages, 57 (37.7%) were published in journals which mandated open data sharing. Data availability statements more frequently described open data on publication when the journal mandated data sharing (open at preprint: 33.3%, open at publication: 61.4%) compared to when the journal did not mandate data sharing (open at preprint: 20.2%, open at publication: 22.3%).

CONCLUSION

Requiring that authors submit a data availability statement is a good first step, but is insufficient to ensure data availability. Strict editorial policies that mandate data sharing (where appropriate) as a condition of publication appear to be effective in making research data available. We would strongly encourage all journal editors to examine whether their data availability policies are sufficiently stringent and consistently enforced.

Preclinical Research Reporting in Shock: Room for Improvement

ABSTRACT

The ARRIVE (Animals in Research: Reporting In Vivo Experiments) guidelines were endorsed by the Shock Society in 2012, but to date there has been no systematic evaluation of research reporting quality for Shock. We systematically assessed 100 randomly selected animal-based research articles published between 2014 and 2018 for reporting quality and statistical practice, compared with 40 pre-ARRIVE studies. More than half of surveyed papers omitted verifiable ethical oversight information and basic animal descriptive information. Few papers reported best-practice methods, such as sample size justification (10%), randomization (43%), randomization method (7%), blinding (23%). Only one paper reported effect sizes to interpret study results. Most troubling was inadequate reporting of welfare-related information (anesthesia, analgesia, humane endpoints, euthanasia). Almost a decade after ARRIVE endorsement, our findings show that reporting deficiencies have persisted with little sign of correction. There is a clear need for investigators to increase transparency of research methods reporting, and drastically improve skills in experimental design. Improvement in standards and greater attention paid to reporting will lead to improvement in reproducibility, replicability, and research quality. It is incumbent upon the research community to improve reporting practices; accurate and transparent reporting is integral to producing rigorous and ethical science.