A systematic scoping review of the neurological effects of COVID-19

BACKGROUND

The global coronavirus 2019 (COVID-19) pandemic began in early 2020, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). In mid-2020 the CIAO (Modelling the Pathogenesis of COVID-19 Using the Adverse Outcome Pathway Framework) project was established, bringing together over 75 interdisciplinary scientists worldwide to collaboratively investigate the underlying biological mechanisms of COVID-19 and consolidate the data using the Adverse Outcome Pathway (AOP) Framework. Neurological symptoms such as anosmia and encephalitis have been frequently reported to be associated with infection with SARS-CoV-2.

OBJECTIVE

Within CIAO, a working group was formed to conduct a systematic scoping review of COVID-19 and its related neurological symptoms to determine which key events and modulating factors are most commonly reported and to identify knowledge gaps.

DESIGN

LitCOVID was used to retrieve 86,075 papers of which 10,244 contained relevant keywords. After title and abstract screening, 2,328 remained and their full texts were reviewed based on predefined inclusion and exclusion criteria. 991 studies fulfilled the inclusion criteria and were retrieved to conduct knowledge synthesis.

RESULTS

The majority of publications reported human observational studies. Early key events were less likely to be reported compared to middle and late key events/adverse outcomes. The majority of modulating factors described related to age or sex. Less recognised COVID-19 associated AO or neurological effects of COVID-19 were also identified including multiple sclerosis/demyelination, neurodegeneration/cognitive effects and peripheral neuronal effects.

CONCLUSION

There were many methodological and reporting issues noted in the reviewed studies. In particular, publication abstracts would benefit from clearer reporting of the methods and endpoints used and the key findings, to ensure relevant papers are included when systematic reviews are conducted. The information extracted from the scoping review may be useful in understanding the mechanisms of neurological effects of COVID-19 and to further develop or support existing AOPs linking COVID-19 and its neurological key events and adverse outcomes. Further evaluation of the less recognised COVID-19 effects is needed.

60 Years of the 3Rs symposium: Lessons learned and the road ahead

When The Principles of Humane Experimental Technique was published in 1959, authors William Russell and Rex Burch had a modest goal: to make researchers think about what they were doing in the laboratory - and to do it more humanely. Sixty years later, their groundbreaking book was celebrated for inspiring a revolution in science and launching a new field: The 3Rs of alternatives to animal experimentation. On November 22, 2019, some pioneering and leading scientists and researchers in the field gathered at the Johns Hopkins Bloomberg School of Public Health in Bal-timore for the 60 Years of the 3Rs Symposium: Lessons Learned and the Road Ahead. The event was sponsored by the Johns Hopkins Center for Alternatives to Animal Testing (CAAT), the Foundation for Chemistry Research and Initiatives, the Alternative Research & Development Foundation (ARDF), the American Cleaning Institute (ACI), the International Fragrance Association (IFRA), the Institute for In Vitro Sciences (IIVS), John "Jack" R. Fowle III, and the Society of Toxicology (SoT). Fourteen pres-entations shared the history behind the groundbreaking publication, international efforts to achieve its aims, stumbling blocks to progress, as well as remarkable achievements. The day was a tribute to Russell and Burch, and a testament to what is possible when people from many walks of life - science, government, and industry - work toward a common goal.

Guidance on protocol development for EFSA generic scientific assessments

EFSA Strategy 2027 outlines the need for fit-for-purpose protocols for EFSA generic scientific assessments to aid in delivering trustworthy scientific advice. This EFSA Scientific Committee guidance document helps address this need by providing a harmonised and flexible framework for developing protocols for EFSA generic assessments. The guidance replaces the 'Draft framework for protocol development for EFSA's scientific assessments' published in 2020. The two main steps in protocol development are described. The first is problem formulation, which illustrates the objectives of the assessment. Here a new approach to translating the mandated Terms of Reference into scientifically answerable assessment questions and sub-questions is proposed: the 'APRIO' paradigm (Agent, Pathway, Receptor, Intervention and Output). Owing to its cross-cutting nature, this paradigm is considered adaptable and broadly applicable within and across the various EFSA domains and, if applied using the definitions given in this guidance, is expected to help harmonise the problem formulation process and outputs and foster consistency in protocol development. APRIO may also overcome the difficulty of implementing some existing frameworks across the multiple EFSA disciplines, e.g. the PICO/PECO approach (Population, Intervention/Exposure, Comparator, Outcome). Therefore, although not mandatory, APRIO is recommended. The second step in protocol development is the specification of the evidence needs and the methods that will be applied for answering the assessment questions and sub-questions, including uncertainty analysis. Five possible approaches to answering individual (sub-)questions are outlined: using evidence from scientific literature and study reports; using data from databases other than bibliographic; using expert judgement informally collected or elicited via semi-formal or formal expert knowledge elicitation processes; using mathematical/statistical models; and - not covered in this guidance - generating empirical evidence ex novo. The guidance is complemented by a standalone 'template' for EFSA protocols that guides the users step by step through the process of planning an EFSA scientific assessment.

First Organoid Intelligence (OI) workshop to form an OI community

The brain is arguably the most powerful computation system known. It is extremely efficient in processing large amounts of information and can discern signals from noise, adapt, and filter faulty information all while running on only 20 watts of power. The human brain's processing efficiency, progressive learning, and plasticity are unmatched by any computer system. Recent advances in stem cell technology have elevated the field of cell culture to higher levels of complexity, such as the development of three-dimensional (3D) brain organoids that recapitulate human brain functionality better than traditional monolayer cell systems. Organoid Intelligence (OI) aims to harness the innate biological capabilities of brain organoids for biocomputing and synthetic intelligence by interfacing them with computer technology. With the latest strides in stem cell technology, bioengineering, and machine learning, we can explore the ability of brain organoids to compute, and store given information (input), execute a task (output), and study how this affects the structural and functional connections in the organoids themselves. Furthermore, understanding how learning generates and changes patterns of connectivity in organoids can shed light on the early stages of cognition in the human brain. Investigating and understanding these concepts is an enormous, multidisciplinary endeavor that necessitates the engagement of both the scientific community and the public. Thus, on Feb 22-24 of 2022, the Johns Hopkins University held the first Organoid Intelligence Workshop to form an OI Community and to lay out the groundwork for the establishment of OI as a new scientific discipline. The potential of OI to revolutionize computing, neurological research, and drug development was discussed, along with a vision and roadmap for its development over the coming decade.

Development of an Evidence-Based Risk Assessment Framework

Assessment of potential human health risks associated with environmental and other agents requires careful evaluation of all available and relevant evidence for the agent of interest, including both data-rich and data-poor agents. With the advent of new approach methodologies in toxicological risk assessment, guidance on integrating evidence from mul-tiple evidence streams is needed to ensure that all available data is given due consideration in both qualitative and quantitative risk assessment. The present report summarizes the discussions among academic, government, and private sector participants from North America and Europe in an international workshop convened to explore the development of an evidence-based risk assessment framework, taking into account all available evidence in an appropriate manner in order to arrive at the best possible characterization of potential human health risks and associated uncertainty. Although consensus among workshop participants was not a specific goal, there was general agreement on the key consider-ations involved in evidence-based risk assessment incorporating 21st century science into human health risk assessment. These considerations have been embodied into an overarching prototype framework for evidence integration that will be explored in more depth in a follow-up meeting.

Deploying elements of scoping review methods for Adverse Outcome Pathway development: A space travel case example

Purpose Health protection agencies require scientific information for evidence-based decision-making and guideline development. However, vetting and collating large quantities of published research to identify relevant high-quality studies is a challenge. One approach to address this issue is the use of Adverse Outcome Pathways (AOPs) that provide a framework to assemble toxicological knowledge into causally linked chains of key events across levels of biological organization to culminate in an adverse health outcome of significance. Traditionally, AOPs have been constructed using a narrative review approach where the collection of evidence that supports each pathway is based on prior knowledge of influential studies that can also be supplemented by individually selecting and reviewing relevant references. Objectives: We aimed to create a protocol for AOP weight of evidence gathering that harnesses elements of both scoping review methods and artificial intelligence tools to increase transparency while reducing bias and workload of human screeners. Methods: To develop this protocol, an existing space-health AOP in the workplan of the Organisation for Economic Co-operation and Development (OECD) AOP program was used as a case example. To balance the benefits of both scoping review tools and narrative approaches, a study protocol outlining a screening and search strategy was developed, and three reference collection workflows were tested to identify the most efficient method to inform weight of evidence. The workflows differed in their literature search strategies, and combinations of software tools used. Results: Across the three tested workflows, over 59 literature searches were completed, retrieving over 34000 references of which over 3300 were human reviewed. The most effective of the three methods used a search strategy with searches across each component of the AOP network, SWIFT Review as a pre-filtering software, and DistillerSR to create structured screening and data extraction forms. This methodology effectively retrieved relevant studies while balancing efficiency in data retrieval without compromising transparency, leading to a well-synthesized evidence base to support the AOP. Conclusions: The workflow is still exploratory in the context of AOP development, and we anticipate adaptations to the protocol with further experience. To further the systematicity, future iterations of the workflow could include structured quality assessment and risk of bias analysis. Overall, the workflow provides a transparent and unbiased approach to support AOP development, which in turn will support the need for rigorous methods to identify relevant scientific evidence while being practical to allow uptake by the broader community.

Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks

The workshop titled “Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks” was co-organized by the Evidence-based Toxicology Collaboration and the European Food Safety Authority (EFSA) and hosted by EFSA at its headquarters in Parma, Italy on October 2 and 3, 2019. The goal was to explore integration of systematic review with mechanistic evidence evaluation. Participants were invited to work on concrete products to advance the exploration of how evidence-based approaches can support the development and application of adverse outcome pathways (AOP) in chemical risk assessment. The workshop discussions were centered around three related themes: 1) assessing certainty in AOPs, 2) literature-based AOP development, and 3) integrating certainty in AOPs and non-animal evidence into decision frameworks. Several challenges, mostly related to methodology, were identified and largely determined the workshop recommendations. The workshop recommendations included the comparison and potential alignment of processes used to develop AOP and systematic review methodology, including the translation of vocabulary of evidence-based methods to AOP and vice versa, the development and improvement of evidence mapping and text mining methods and tools, as well as a call for a fundamental change in chemical risk and uncertainty assessment methodology if to be conducted based on AOPs and new approach methodologies (NAM). The usefulness of evidence-based approaches for mechanism-based chemical risk assessments was stressed, particularly the potential contribution of the rigor and transparency inherent to such approaches in building stakeholders’ trust for implementation of NAM evidence and AOPs into chemical risk assessment.

A new ChEMBL dataset for the similarity-based target fishing engine FastTargetPred: Annotation of an exhaustive list of linear tetrapeptides

Drug discovery often requires the identification of off-targets as the binding of a compound to targets other than the intended target(s) can be beneficial in some cases or detrimental in other situations (e.g., binding to anti-targets). Such investigations are also of importance during the early stage of a project, for example when the target is not known (e.g., phenotypic screening). Target identification can be performed in-vitro, but various in-silico methods have also been developed in recent years to facilitate target identification and help generate ideas. FastTargetPred is one such approach, it is a freely available Python/C program that attempts to predict putative macromolecular targets (i.e., target fishing) for a single input small molecule query or an entire compound collection using established chemical similarity search approaches. Indeed, the putative macromolecular target(s) of a small chemical compound can be predicted by identifying ligands that are known experimentally to bind to some targets and that are structurally similar to the input query chemical compound. Therefore, this type of target fishing approach relies on a large collection of experimentally validated macromolecule-chemical compound binding data. The small chemical compounds can be described as molecular fingerprints encoding their structural characteristics as a vector. The published version of FastTargetPred used ligand-target binding data extracted from the release 25 (2019) of the ChEMBL database. Here we provide a new dataset for FastTargetPred extracted from the last ChEMBL release, namely, at the time of writing, ChEMBL29 (2021). Four fingerprints were computed (ECFP4, ECFP6, MACCS and PL) for the extracted compound dataset (714,780 unique ChEMBL29 compounds while the entire ChEMBL29 database contained about 2.1 million compounds). However, it was not possible to compute fingerprints for 19 molecules because of their unusual chemistry (complex macrocycles). These data files were then prepared so as to be compatible with FastTargetPred requirements. The 714,761 ChEMBL chemical compounds with computed fingerprints hit 6,477 macromolecular targets based on the selected criteria. For these ChEMBL compounds a ChEMBL target ID is reported and these target IDs were matched with the corresponding UniProt IDs. Thus, when available, the UniProt ID is provided, the protein UniProt name, the gene name, the organism as well as annotated involvement in diseases, gene ontology data, and cross-references to the Reactome pathway database. As short peptides can be of interest for drug discovery and chemical biology endeavours, we were interested in attempting to predict putative macromolecular targets for a previously reported exhaustive combination of peptides containing four natural amino acids (i.e., 20 × 20 × 20 × 20 = 160,000 linear tetrapeptides) using FastTargetPred and the presently generated ChEMBL29 dataset. With the parameters used, putative targets are reported for 63,944 unique query peptides. These target predictions are provided in two different searchable files with hyperlinks to the ChEMBL, UniProt and Reactome databases.

Pluripotent Stem Cell Assays: Modalities and Applications For Predictive Developmental Toxicity

•

A systematic scoping review of the literature evaluated the embryonic stem cell test (EST).

•

1533 publications included 18 publications testing 10 or more compounds in human or mouse EST.

•

Selected case examples included 5-fluorouracil, thalidomide, and caffeine.

•

Applicability, limitations, and recommendations for further work are discussed.

This manuscript provides a review focused on embryonic stem cell-based models and their place within the landscape of alternative developmental toxicity assays. Against the background of the principles of developmental toxicology, the wide diversity of alternative methods using pluripotent stem cells developed in this area over the past half century is reviewed. In order to provide an overview of available models, a systematic scoping review was conducted following a published protocol with inclusion criteria, which were applied to select the assays. Critical aspects including biological domain, readout endpoint, availability of standardized protocols, chemical domain, reproducibility and predictive power of each assay are described in detail, in order to review the applicability and limitations of the platform in general and progress moving forward to implementation. The horizon of innovative routes of promoting regulatory implementation of alternative methods is scanned, and recommendations for further work are given.

Biological plausibility in environmental health systematic reviews: a GRADE concept paper

BACKGROUND

"Biological plausibility" is a concept frequently referred to in environmental and public health when researchers are evaluating how confident they are in the results and inferences of a study or evidence review. Biological plausibility is not, however, a domain of one of the most widely-used approaches for assessing the certainty of evidence (CoE) which underpins the findings of a systematic review, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) CoE Framework. Whether the omission of biological plausibility is a potential limitation of the GRADE CoE Framework is a topic that is regularly discussed, especially in the context of environmental health systematic reviews.

OBJECTIVES

We analyse how the concept of "biological plausibility", as applied in the context of assessing certainty of the evidence that supports the findings of a systematic review, is accommodated under the processes of systematic review and the existing GRADE domains.

RESULTS AND DISCUSSION

We argue that "biological plausibility" is a concept which primarily comes into play when direct evidence about the effects of an exposure on a population of concern (usually humans) is absent, at high risk of bias, is inconsistent, or limited in other ways. In such circumstances, researchers look toward evidence from other study designs in order to draw conclusions. In this respect, we can consider experimental animal and in vitro evidence as "surrogates" for the target populations, exposures, comparators and outcomes of actual interest. Through discussion of 10 examples of experimental surrogates, we propose that the concept of biological plausibility consists of two principal aspects: a "generalisability aspect" and a "mechanistic aspect". The "generalisability aspect" concerns the validity of inferences from experimental models to human scenarios, and asks the same question as does the assessment of external validity or indirectness in systematic reviews. The "mechanistic aspect" concerns certainty in knowledge of biological mechanisms and would inform judgements of indirectness under GRADE, and thus the overall CoE. While both aspects are accommodated under the indirectness domain of the GRADE CoE Framework, further research is needed to determine how to use knowledge of biological mechanisms in the assessment of indirectness of the evidence in systematic reviews.

How evidence-based methodologies can help identify and reduce uncertainty in chemical risk assessment

Evidence-based methodology, in particular systematic review, is increasingly being applied in environmental, public, and occupational health to increase the transparency, comprehensiveness, and objectivity of the processes by which existing evidence is gathered, assessed, and synthesized in answering research questions. This development is also changing risk assessment practices and will impact the assessment of uncertainties in the evidence for risks to human health that are posed by exposure to chemicals. The potential of evidence-based methodology for characterizing uncertainties in risk assessment has been widely recognized, while its contribution to uncertainty reduction is yet to be fully elucidated. We therefore present some key aspects of the evidence-based approach to risk assessment, showing how they can contribute to the identification and the assessment of uncertainties. We focus on the pre-specification of an assessment method­ology in a protocol, comprehensive search strategies, study selection using predefined eligibility criteria, critical appraisal of individual studies, and an evidence integration and uncertainty characterization process based on certainty of evi­dence frameworks that are well-established in health care research. We also provide examples of uncertainty in risk assessment and discuss how evidence-based methodology could address those. This perspective, which neither claims to be comprehensive nor complete, is intended to stimulate discussion of the topic and to motivate detailed exploration of how evidence-based methodology contributes to characterization of uncertainties, and how it will lead to uncertainty reduction in the conduct of health risk assessment.

Probabilistic risk assessment - the keystone for the future of toxicology

Safety sciences must cope with uncertainty of models and results as well as information gaps. Acknowledging this uncertainty necessitates embracing probabilities and accepting the remaining risk. Every toxicological tool delivers only probable results. Traditionally, this is taken into account by using uncertainty / assessment factors and worst-case / precautionary approaches and thresholds. Probabilistic methods and Bayesian approaches seek to characterize these uncertainties and promise to support better risk assessment and, thereby, improve risk management decisions. Actual assessments of uncertainty can be more realistic than worst-case scenarios and may allow less conservative safety margins. Most importantly, as soon as we agree on uncertainty, this defines room for improvement and allows a transition from traditional to new approach methods as an engineering exercise. The objective nature of these mathematical tools allows to assign each methodology its fair place in evidence integration, whether in the context of risk assessment, systematic reviews, or in the definition of an integrated testing strategy (ITS) / defined approach (DA) / integrated approach to testing and assessment (IATA). This article gives an overview of methods for probabilistic risk assessment and their application for exposure assessment, physiologically-based kinetic modelling, probability of hazard assessment (based on quantitative and read-across based structure-activity relationships, and mechanistic alerts from in vitro studies), individual susceptibility assessment, and evidence integration. Additional aspects are opportunities for uncertainty analysis of adverse outcome pathways and their relation to thresholds of toxicological concern. In conclusion, probabilistic risk assessment will be key for constructing a new toxicology paradigm – probably!

The effect of radiofrequency electromagnetic fields (RF-EMF) on biomarkers of oxidative stress in vivo and in vitro: A protocol for a systematic review

BACKGROUND

Oxidative stress is conjectured to be related to many diseases. Furthermore, it is hypothesized that radiofrequency fields may induce oxidative stress in various cell types and thereby compromise human and animal health. This systematic review (SR) aims to summarize and evaluate the literature related to this hypothesis.

OBJECTIVES

The main objective of this SR is to evaluate the associations between the exposure to radiofrequency electromagnetic fields and oxidative stress in experimental models (in vivo and in vitro).

METHODS

The SR framework has been developed following the guidelines established in the WHO Handbook for Guideline Development and the Handbook for Conducting a Literature-Based Health Assessment). We will include controlled in vivo and in vitro laboratory studies that assess the effects of an exposure to RF-EMF on valid markers for oxidative stress compared to no or sham exposure. The protocol is registered in PROSPERO. We will search the following databases: PubMed, Embase, Web of Science Core Collection, Scopus, and the EMF-Portal. The reference lists of included studies and retrieved review articles will also be manually searched.

STUDY APPRAISAL AND SYNTHESIS METHOD

Data will be extracted according to a pre-defined set of forms developed in the DistillerSR online software and synthesized in a meta-analysis when studies are judged sufficiently similar to be combined. If a meta-analysis is not possible, we will describe the effects of the exposure in a narrative way.

RISK OF BIAS

The risk of bias will be assessed with the NTP/OHAT risk of bias rating tool for human and animal studies. We will use GRADE to assess the certainty of the conclusions (high, moderate, low, or inadequate) regarding the association between radiofrequency electromagnetic fields and oxidative stress.

FUNDING

This work was funded by the World Health Organization (WHO).

REGISTRATION

The protocol was registered on the PROSPERO webpage on July 8, 2021.

COVID-19 through Adverse Outcome Pathways: Building networks to better understand the disease - 3rd CIAO AOP Design Workshop

On April 28-29, 2021, 50 scientists from different fields of expertise met for the 3rd online CIAO workshop. The CIAO project “Modelling the Pathogenesis of COVID-19 using the Adverse Outcome Pathway (AOP) framework” aims at building a holistic assembly of the available scientific knowledge on COVID-19 using the AOP framework. An individual AOP depicts the disease progression from the initial contact with the SARS-CoV-2 virus through biological key events (KE) toward an adverse outcome such as respiratory distress, anosmia or multiorgan failure. Assembling the individual AOPs into a network highlights shared KEs as central biological nodes involved in multiple outcomes observed in COVID-19 patients. During the workshop, the KEs and AOPs established so far by the CIAO members were presented and posi­tioned on a timeline of the disease course. Modulating factors influencing the progression and severity of the disease were also addressed as well as factors beyond purely biological phenomena. CIAO relies on an interdisciplinary crowd­sourcing effort, therefore, approaches to expand the CIAO network by widening the crowd and reaching stakeholders were also discussed. To conclude the workshop, it was decided that the AOPs/KEs will be further consolidated, inte­grating virus variants and long COVID when relevant, while an outreach campaign will be launched to broaden the CIAO scientific crowd.

Systematic review in evidence-based risk assessment

Systematic reviews provide a structured framework for summarizing the available evidence in a comprehensive, objective, and transparent manner. They inform evidence-based guidelines in medicine, public policy, and more recently, in environmental health and toxicology. Many regulatory agencies have extended and adapted the well-established systematic review methods, initially developed for clinical studies, for their assessment needs. The use of systematic reviews to summarize evidence from existing human, animal, and mechanistic studies can reduce reliance on animal test data in risk assessment and can help avoid unnecessary duplication of animal experiments that have already been conducted. As alternative test methods can be expected to play an increasing role in human health risk assessment in the future, systematic reviews can be particularly helpful in validating these alternatives. The field of evidence-based toxicology has undergone extensive development since its first meeting in 2007 as a result of collaborative efforts among international experts and public health agencies, particularly with respect to the use of mechanistic data and evidence integration. The continued development and wider adoption of systematic review methodology can lead to better 3R implementation. As undertaking a systematic review can be a complex and lengthy process, it is important to understand the main steps involved. Key steps, along with current best practices, are described with references to guidance from organizations with expertise in evidence synthesis. Applications of systematic reviews in clinical, observational, and experimental studies are presented. Finally, software tools available to facilitate and increase the efficiency of completing a systematic review are described.

A Systematic Review to Compare Chemical Hazard Predictions of the Zebrafish Embryotoxicity Test With Mammalian Prenatal Developmental Toxicity

Originally developed to inform the acute toxicity of chemicals on fish, the zebrafish embryotoxicity test (ZET) has also been proposed for assessing the prenatal developmental toxicity of chemicals, potentially replacing mammalian studies. Although extensively evaluated in primary studies, a comprehensive review summarizing the available evidence for the ZET's capacity is lacking. Therefore, we conducted a systematic review of how well the presence or absence of exposure-related findings in the ZET predicts prenatal development toxicity in studies with rats and rabbits. A two-tiered systematic review of the developmental toxicity literature was performed, a review of the ZET literature was followed by one of the mammalian literature. Data were extracted using DistillerSR, and study validity was assessed with an amended SYRCLE's risk-of-bias tool. Extracted data were analyzed for each species and substance, which provided the basis for comparing the 2 test methods. Although limited by the number of 24 included chemicals, our results suggest that the ZET has potential to identify chemicals that are mammalian prenatal developmental toxicants, with a tendency for overprediction. Furthermore, our analysis confirmed the need for further standardization of the ZET. In addition, we identified contextual and methodological challenges in the application of systematic review approaches to toxicological questions. One key to overcoming these challenges is a transition to more comprehensive and transparent planning, conduct and reporting of toxicological studies. The first step toward bringing about this change is to create broad awareness in the toxicological community of the need for and benefits of more evidence-based approaches.

Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

Applying evidence-based methods to the development and use of adverse outcome pathways

The workshop “Application of evidence-based methods to construct mechanistic frameworks for the development and use of non-animal toxicity tests” was organized by the Evidence-based Toxicology Collaboration and hosted by the Grading of Recommendations Assessment, Development and Evaluation Working Group on June 12, 2019. The purpose of the workshop was to bring together international regulatory bodies, risk assessors, academic scientists, and industry to explore how systematic review methods and the adverse outcome pathway framework could be combined to develop and use mechanistic test methods for predicting the toxicity of chemical substances in an evidence-based manner. The meeting covered the history of biological frameworks, the way adverse outcome pathways are currently developed, the basic principles of systematic methodology, including systematic reviews and evidence maps, and assessment of cer­tainty in models, and adverse outcome pathways in particular. Specific topics were discussed via case studies in small break-out groups. The group concluded that adverse outcome pathways provide an important framework to support mechanism-based assessment in environmental health. The process of their development has a few challenges that could be addressed with systematic methods and automation tools. Addressing these challenges will increase the transparency of the evidence behind adverse outcome pathways and the consistency with which they are defined; this in turn will increase their value for supporting public health decisions. It was suggested to explore the details of applying systematic methods to adverse outcome pathway development in a series of case studies and workshops.

Reviewing the animal literature: how to describe and choose between different types of literature reviews

Before starting any (animal) research project, review of the existing literature is good practice. From both the scientific and the ethical perspective, high-quality literature reviews are essential. Literature reviews have many potential advantages besides synthesising the evidence for a research question. First, they can show if a proposed study has already been performed, preventing redundant research. Second, when planning new experiments, reviews can inform the experimental design, thereby increasing the reliability, relevance and efficiency of the study. Third, reviews may even answer research questions using already available data. Multiple definitions of the term literature review co-exist. In this paper, we describe the different steps in the review process, and the risks and benefits of using various methodologies in each step. We then suggest common terminology for different review types: narrative reviews, mapping reviews, scoping reviews, rapid reviews, systematic reviews and umbrella reviews. We recommend which review to select, depending on the research question and available resources. We believe that improved understanding of review methods and terminology will prevent ambiguity and increase appropriate interpretation of the conclusions of reviews.

GRADE Guidelines 30: the GRADE approach to assessing the certainty of modeled evidence-An overview in the context of health decision-making

OBJECTIVES

The objective of the study is to present the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) conceptual approach to the assessment of certainty of evidence from modeling studies (i.e., certainty associated with model outputs).

STUDY DESIGN AND SETTING

Expert consultations and an international multidisciplinary workshop informed development of a conceptual approach to assessing the certainty of evidence from models within the context of systematic reviews, health technology assessments, and health care decisions. The discussions also clarified selected concepts and terminology used in the GRADE approach and by the modeling community. Feedback from experts in a broad range of modeling and health care disciplines addressed the content validity of the approach.

RESULTS

Workshop participants agreed that the domains determining the certainty of evidence previously identified in the GRADE approach (risk of bias, indirectness, inconsistency, imprecision, reporting bias, magnitude of an effect, dose-response relation, and the direction of residual confounding) also apply when assessing the certainty of evidence from models. The assessment depends on the nature of model inputs and the model itself and on whether one is evaluating evidence from a single model or multiple models. We propose a framework for selecting the best available evidence from models: 1) developing de novo, a model specific to the situation of interest, 2) identifying an existing model, the outputs of which provide the highest certainty evidence for the situation of interest, either "off-the-shelf" or after adaptation, and 3) using outputs from multiple models. We also present a summary of preferred terminology to facilitate communication among modeling and health care disciplines.

CONCLUSION

This conceptual GRADE approach provides a framework for using evidence from models in health decision-making and the assessment of certainty of evidence from a model or models. The GRADE Working Group and the modeling community are currently developing the detailed methods and related guidance for assessing specific domains determining the certainty of evidence from models across health care-related disciplines (e.g., therapeutic decision-making, toxicology, environmental health, and health economics).

Quantitative Systems Pharmacology for Neuroscience Drug Discovery and Development: Current Status, Opportunities, and Challenges

The substantial progress made in the basic sciences of the brain has yet to be adequately translated to successful clinical therapeutics to treat central nervous system (CNS) diseases. Possible explanations include the lack of quantitative and validated biomarkers, the subjective nature of many clinical endpoints, and complex pharmacokinetic/pharmacodynamic relationships, but also the possibility that highly selective drugs in the CNS do not reflect the complex interactions of different brain circuits. Although computational systems pharmacology modeling designed to capture essential components of complex biological systems has been increasingly accepted in pharmaceutical research and development for oncology, inflammation, and metabolic disorders, the uptake in the CNS field has been very modest. In this article, a cross-disciplinary group with representatives from academia, pharma, regulatory, and funding agencies make the case that the identification and exploitation of CNS therapeutic targets for drug discovery and development can benefit greatly from a system and network approach that can span the gap between molecular pathways and the neuronal circuits that ultimately regulate brain activity and behavior. The National Institute of Neurological Disorders and Stroke (NINDS), in collaboration with the National Institute on Aging (NIA), National Institute of Mental Health (NIMH), National Institute on Drug Abuse (NIDA), and National Center for Advancing Translational Sciences (NCATS), convened a workshop to explore and evaluate the potential of a quantitative systems pharmacology (QSP) approach to CNS drug discovery and development. The objective of the workshop was to identify the challenges and opportunities of QSP as an approach to accelerate drug discovery and development in the field of CNS disorders. In particular, the workshop examined the potential for computational neuroscience to perform QSP-based interrogation of the mechanism of action for CNS diseases, along with a more accurate and comprehensive method for evaluating drug effects and optimizing the design of clinical trials. Following up on an earlier white paper on the use of QSP in general disease mechanism of action and drug discovery, this report focuses on new applications, opportunities, and the accompanying limitations of QSP as an approach to drug development in the CNS therapeutic area based on the discussions in the workshop with various stakeholders.

Software tools for literature screening in systematic reviews in biomedical research

Systematic Reviews (SRs) hold promise for implementing the 3Rs in animal sciences: they can retrieve available alternative models, help refining experiments, and identify insufficiencies, or an excess of, scientific knowledge on a particular topic. Unfortunately, SRs can be labour- and time-intensive, especially the reference screening and data extraction phases. Fortunately, there are several software tools available that help make screening faster and easier. However, it is not always clear which features the tools offer. Therefore, a feature analysis was performed to compare different reference screening tools as objectively as possible. This analysis enables researchers to select the most appropriate tool for their needs. Fifteen different tools were compared: CADIMA, Covidence, DistillerSR, Endnote, Endnote using Bramer's method, EROS, HAWC, Microsoft Excel, Excel using VonVille's method, Microsoft Word, Rayyan, RevMan, SyRF, SysRev.com, and SWIFT Active Screener. Their support of 21 features was tested. Features were categorised as mandatory, desirable, and optional. DistillerSR, Covidence, and SWIFT Active Screener are the tools that support all mandatory features. These tools are preferred for screening references, but none of them are free. The best scoring free tool is Rayyan, which lacks one mandatory function: distinct title/abstract and full-text phases. The lowest scoring tools are those not specifically designed for SRs, like Microsoft Word and Endnote. Their use can only be advised for small and simple SRs. A well-informed selection of SR screening tools will benefit review quality and speed, which can contribute to the advancement of the 3Rs in animal studies.

Toward Good In Vitro Reporting Standards

A good experiment reported badly is worthless. Meaningful contributions to the body of science are made by sharing the full methodology and results so that they can be evaluated and reproduced by peers. Erroneous and incomplete reporting does not do justice to the resources spent on conducting the experiment and the time peers spend reading the article. In theory peer-review should ensure adequate reporting - in practice it does not. Many areas have developed reporting standards and checklists to support the adequate reporting of scientific efforts, but in vitro research still has no generally accepted criteria. It is characterized by a "Wild West" or "anything goes" attitude. Such a culture may undermine trust in the reproducibility of animal-free methods, and thus parallel the "reproducibility crisis" discussed for other life science fields. The increasing data retrieval needs of computational approaches (in extreme as "big data" and artificial intelligence) makes reporting quality even more important so that the scientific community can take full advantage of the results. The first priority of reporting standards is to ensure the completeness and transparency of information provided (data focus). The second tier is a quality of data display that makes information digestible and easy to grasp, compare and further analyze (information focus). This article summarizes a series of initiatives geared towards improving the quality of in vitro work and its reporting. This shall ultimately lead to Good In Vitro Reporting Standards (GIVReSt).

Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned with the Zebrafish Embryotoxicity Test

Systematic review methodology is a means of addressing specific questions through structured, consistent, and transparent examinations of the relevant scientific evidence. This methodology has been used to advantage in clinical medicine, and is being adapted for use in other disciplines. Although some applications to toxicology have been explored, especially for hazard identification, the present preparatory study is, to our knowledge, the first attempt to adapt it to the assessment of toxicological test methods. As our test case, we chose the zebrafish embryotoxicity test (ZET) for developmental toxicity and its mammalian counterpart, the standard mammalian prenatal development toxicity study, focusing the review on how well the ZET predicts the presence or absence of chemical-induced prenatal developmental toxicity observed in mammalian studies. An interdisciplinary team prepared a systematic review protocol and adjusted it throughout this piloting phase, where needed. The final protocol was registered and will guide the main study (systematic review), which will execute the protocol to comprehensively answer the review question. The goal of this preparatory study was to translate systematic review methodology to the assessment of toxicological test method performance. Consequently, it focused on the methodological issues encountered, whereas the main study will report substantive findings. These relate to numerous systematic review steps, but primarily to searching and selecting the evidence. Applying the lessons learned to these challenges can improve not only our main study, but may also be helpful to others seeking to use systematic review methodology to compare toxicological test methods. We conclude with a series of recommendations that, if adopted, would help improve the quality of the published literature, and make conducting systematic reviews of toxicological studies faster and easier over time.

Evidence-based absorption, distribution, metabolism, excretion (ADME) and its interplay with alternative toxicity methods

ADME (absorption, distribution, metabolism, elimination) has rapidly evolved over the past two decades, creating a unique interdisciplinary interface between medicinal chemists, biologists, formulators, toxicologists, clinicians, and regulators across industries, but has advanced most rapidly in the pharmaceutical industry. The implementation of ADME profiling of drug candidates, in conjunction with biological efficacy and safety optimization, has dramatically reduced pharmacokinetic drug failures in clinical trials and has become a lingua franca between disciplines that are involved in drug development. This article briefly reviews the basics and current state-of-the-art of ADME and the major lessons from the pharmaceutical industry on its efficient use, points out the importance of defining ADME properties leading to toxicity across industries for safety and toxicity prediction of chemicals, and raises the issues of quality, reliability, and reproducibility of tests and inclusion of ADME under the umbrella of evidence-based toxicology. Increasingly, in vitro results are used to inform ADME assessments and computer modeling. The aspects of kinetics of substances in cellular models themselves, however, are still too often neglected. ADME information will play a critical role in establishing quantitative in vitro to in vivo extrapolations (QIVIVE), integrated testing strategies, and systems toxicology approaches.

De-Risking Drug Discovery with ADDME — Avoiding Drug Development Mistakes Early

The advent of early Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) screening has increased the elimination rate of weak drug candidates early in the drug-discovery process, and decreased the proportion of compounds failing in clinical trials for ADMET reasons. This paper reviews the history of ADMET screening and why it has become so important in drug discovery and development. Assays that have been developed in response to specific needs, and improvements in technology that result in higher throughput and greater accuracy of prediction of human mechanisms of toxicity, are discussed. The paper concludes with the authors’ forecast of new models that will better predict human efficacy and toxicity.

Cardionomics: a new integrative approach for screening cardiotoxicity of drug candidates

BACKGROUND

Despite the FDA guidelines for studies to be performed to rule out potential cardiac toxicity, many drugs have nevertheless entered the market only to be later withdrawn from the market owing to cardiac toxicity. Cardiac toxicity may result from drugs causing impaired function or death of cardiomyocytes, valvular damage, myocardial ischemia and/or ventricular arrhythmias. Negative cardiovascular events have been implicated in 28% of drug withdrawals in the USA. The significance for patients, regulators and the pharmaceutical industry is immense.

OBJECTIVE

We address whether a more rigorous and integrative approach is needed for cardiovascular safety screening of all new drug candidates. Furthermore, we will present a cardionomics approach that looks at several in vitro and in vivo models that can be applied to all drugs independent of category, therapeutic area or class.

METHODS

We present examples of drugs demonstrating cardiac toxicity and provide an in-depth review of how calcium homeostasis may be a unifying theme in clinically observed cardiotoxic events. We introduce a cardionomics approach that detects clinical cardiac toxicity early in the drug discovery process, thus, preventing costly late attrition.

CONCLUSION

The consequences of a failure to detect potential cardiovascular safety issues before clinical launch can have an enormous cost for the pharmaceutical industry, when major drugs are withdrawn due to lawsuits as well as loss of time and resources. An integrated cardionomics approach may reduce the risk of drug withdrawals as a result of unexpected clinical cardiac safety issues.

ADDME--Avoiding Drug Development Mistakes Early: central nervous system drug discovery perspective

The advent of early absorption, distribution, metabolism, excretion, and toxicity (ADMET) screening has increased the attrition rate of weak drug candidates early in the drug-discovery process, and decreased the proportion of compounds failing in clinical trials for ADMET reasons. This paper reviews the history of ADMET screening and its place in pharmaceutical development, and central nervous system drug discovery in particular. Assays that have been developed in response to specific needs and improvements in technology that result in higher throughput and greater accuracy of prediction of human mechanisms of absorption and toxicity are discussed. The paper concludes with the authors' forecast of new models that will better predict human efficacy and toxicity.