The principles of weight of evidence validation of test methods and testing strategies. The report and recommendations of ECVAM workshop 58

A Systematic Review to Compare Chemical Hazard Predictions of the Zebrafish Embryotoxicity Test With Mammalian Prenatal Developmental Toxicity

Originally developed to inform the acute toxicity of chemicals on fish, the zebrafish embryotoxicity test (ZET) has also been proposed for assessing the prenatal developmental toxicity of chemicals, potentially replacing mammalian studies. Although extensively evaluated in primary studies, a comprehensive review summarizing the available evidence for the ZET's capacity is lacking. Therefore, we conducted a systematic review of how well the presence or absence of exposure-related findings in the ZET predicts prenatal development toxicity in studies with rats and rabbits. A two-tiered systematic review of the developmental toxicity literature was performed, a review of the ZET literature was followed by one of the mammalian literature. Data were extracted using DistillerSR, and study validity was assessed with an amended SYRCLE's risk-of-bias tool. Extracted data were analyzed for each species and substance, which provided the basis for comparing the 2 test methods. Although limited by the number of 24 included chemicals, our results suggest that the ZET has potential to identify chemicals that are mammalian prenatal developmental toxicants, with a tendency for overprediction. Furthermore, our analysis confirmed the need for further standardization of the ZET. In addition, we identified contextual and methodological challenges in the application of systematic review approaches to toxicological questions. One key to overcoming these challenges is a transition to more comprehensive and transparent planning, conduct and reporting of toxicological studies. The first step toward bringing about this change is to create broad awareness in the toxicological community of the need for and benefits of more evidence-based approaches.

Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned with the Zebrafish Embryotoxicity Test

Systematic review methodology is a means of addressing specific questions through structured, consistent, and transparent examinations of the relevant scientific evidence. This methodology has been used to advantage in clinical medicine, and is being adapted for use in other disciplines. Although some applications to toxicology have been explored, especially for hazard identification, the present preparatory study is, to our knowledge, the first attempt to adapt it to the assessment of toxicological test methods. As our test case, we chose the zebrafish embryotoxicity test (ZET) for developmental toxicity and its mammalian counterpart, the standard mammalian prenatal development toxicity study, focusing the review on how well the ZET predicts the presence or absence of chemical-induced prenatal developmental toxicity observed in mammalian studies. An interdisciplinary team prepared a systematic review protocol and adjusted it throughout this piloting phase, where needed. The final protocol was registered and will guide the main study (systematic review), which will execute the protocol to comprehensively answer the review question. The goal of this preparatory study was to translate systematic review methodology to the assessment of toxicological test method performance. Consequently, it focused on the methodological issues encountered, whereas the main study will report substantive findings. These relate to numerous systematic review steps, but primarily to searching and selecting the evidence. Applying the lessons learned to these challenges can improve not only our main study, but may also be helpful to others seeking to use systematic review methodology to compare toxicological test methods. We conclude with a series of recommendations that, if adopted, would help improve the quality of the published literature, and make conducting systematic reviews of toxicological studies faster and easier over time.

The OECD QSAR Toolbox Starts Its Second Decade

The OECD QSAR Toolbox is a computer software designed to make pragmatic qualitative and quantitative structure-activity relationship methods-based predictions of toxicity, including read-across, available to the user in a comprehensible and transparent manner. The Toolbox, provide information on chemicals in structure-searchable, standardized files that are associated with chemical and toxicity data to ensure that proper structural analogs can be identified. This chapter describes the advantages of the Toolbox, the aims, approach, and workflow of it, as well as reviews its history. Additionally, key functional elements of it use are explained and features new to Version 4.1 are reported. Lastly, the further development of the Toolbox, likely needed to transform it into a more comprehensive Chemical Management System, is considered.

Mind the Gap! A Journey towards Computational Toxicology

Computational methods have advanced toxicology towards the development of target-specific models based on a clear cause-effect rationale. However, the predictive potential of these models presents strengths and weaknesses. On the good side, in silico models are valuable cheap alternatives to in vitro and in vivo experiments. On the other, the unconscious use of in silico methods can mislead end-users with elusive results. The focus of this review is on the basic scientific and regulatory recommendations in the derivation and application of computational models. Attention is paid to examine the interplay between computational toxicology and drug discovery and development. Avoiding the easy temptation of an overoptimistic future, we report our view on what can, or cannot, realistically be done. Indeed, studies of safety/toxicity represent a key element of chemical prioritization programs carried out by chemical industries, and primarily by pharmaceutical companies.

From "weight of evidence" to quantitative data integration using multicriteria decision analysis and Bayesian methods

“Weighing” available evidence in the process of decision-making is unavoidable, yet it is one step that routinely raises suspicions: what evidence should be used, how much does it weigh, and whose thumb may be tipping the scales? This commentary aims to evaluate the current state and future roles of various types of evidence for hazard assessment as it applies to environmental health. In its recent evaluation of the US Environmental Protection Agency’s Integrated Risk Information System assessment process, the National Research Council committee singled out the term “weight of evidence” (WoE) for critique, deeming the process too vague and detractive to the practice of evaluating human health risks of chemicals. Moving the methodology away from qualitative, vague and controversial methods towards generalizable, quantitative and transparent methods for appropriately managing diverse lines of evidence is paramount for both regulatory and public acceptance of the hazard assessments. The choice of terminology notwithstanding, a number of recent Bayesian WoE-based methods, the emergence of multi criteria decision analysis for WoE applications, as well as the general principles behind the foundational concepts of WoE, show promise in how to move forward and regain trust in the data integration step of the assessments. We offer our thoughts on the current state of WoE as a whole and while we acknowledge that many WoE applications have been largely qualitative and subjective in nature, we see this as an opportunity to turn WoE towards a quantitative direction that includes Bayesian and multi criteria decision analysis.

Why endocrine disrupting chemicals (EDCs) challenge traditional risk assessment and how to respond

Endocrine disrupting compounds (EDCs) are a diverse group of "chemicals of emerging concern" which have attracted much interest from the research community since the 1990s. Today there is still no definitive risk assessment tool for EDCs. While some decision making organizations have attempted to design methodology guidelines to evaluate the potential risk from this broadly defined group of constituents, risk assessors still face many uncertainties and unknowns. Until a risk assessment paradigm is designed specifically for EDCs and is vetted by the field, traditional risk assessment tools may be used with caution to evaluate EDCs. In doing so, each issue of contention should be addressed with transparency in order to leverage available information and technology without sacrificing integrity or accuracy. The challenges that EDCs pose to traditional risk assessment are described in this article to assist in this process.

Animal use in pharmacology education and research: the changing scenario

The use of animals in research and education dates back to the period when humans started to look for ways to prevent and cure ailments. Most of present day's drug discoveries were possible because of the use of animals in research. The dilemma to continue animal experiments in education and research continues with varied and confusing guidelines. However, the animal use and their handling vary in each laboratory and educational institution. It has been reported that the animals are being subjected to painful procedures in education and training unnecessarily. The extensive use of animals in toxicity studies and testing dermatological preparations has raised concerns about the ways animals are sacrificed for these “irrelevant experiments”. On the other side of the coin are scientists who advocate the relevant and judicious use of animals in research so that new discoveries can continue. In this review, we discuss the evolution of the use of animals in education and research and how these have been affected in recent times owing to concerns from animal lovers and government regulations. A number of computer simulation and other models have been recommended for use as alternatives to use of animals for pharmacology education. In this review we also discuss some of these alternatives.

Safety assessment of cosmetic products, with emphasis on the ocular area: regulatory aspects and validation processes

Before marketing a cosmetic product, a series of biological assays, such as ocular irritation tests, must be conducted in order to prove that the product is safe. However, a few scientific articles mention the discussion and evolution of cosmetic products testing performed in the eyes area. The aim of this study was to review the available literature on the evolution of tests carried out with cosmetics, in the ocular area, as well as to describe the methodologies that have been used and that are currently accepted. In Brazil, tests performed on animals are still allowed. However, the international laws strongly recommend the use of alternative methods for evaluating the risk of cosmetic ingredients and products. Regulatory requirements involving the registration of these products also request safety support of them in human beings. To perform ocular tests in human beings, it is necessary to involve an ophthalmologist for conducting clinical protocols. These protocols signed by the expert physician are sent to the National Health Surveillance Agency in order to endorse the product manufacturer concerning its safety. The safety support of a cosmetic product is very important, taking into account that the consumer has free access to these products of widespread use in today's society.

Applying Adverse Outcome Pathways (AOPs) to support Integrated Approaches to Testing and Assessment (IATA)

Chemical regulation is challenged by the large number of chemicals requiring assessment for potential human health and environmental impacts. Current approaches are too resource intensive in terms of time, money and animal use to evaluate all chemicals under development or already on the market. The need for timely and robust decision making demands that regulatory toxicity testing becomes more cost-effective and efficient. One way to realize this goal is by being more strategic in directing testing resources; focusing on chemicals of highest concern, limiting testing to the most probable hazards, or targeting the most vulnerable species. Hypothesis driven Integrated Approaches to Testing and Assessment (IATA) have been proposed as practical solutions to such strategic testing. In parallel, the development of the Adverse Outcome Pathway (AOP) framework, which provides information on the causal links between a molecular initiating event (MIE), intermediate key events (KEs) and an adverse outcome (AO) of regulatory concern, offers the biological context to facilitate development of IATA for regulatory decision making. This manuscript summarizes discussions at the Workshop entitled "Advancing AOPs for Integrated Toxicology and Regulatory Applications" with particular focus on the role AOPs play in informing the development of IATA for different regulatory purposes.

Regulatory decisions on endocrine disrupting chemicals should be based on the principles of endocrinology

For years, scientists from various disciplines have studied the effects of endocrine disrupting chemicals (EDCs) on the health and wellbeing of humans and wildlife. Some studies have specifically focused on the effects of low doses, i.e. those in the range that are thought to be safe for humans and/or animals. Others have focused on the existence of non-monotonic dose-response curves. These concepts challenge the way that chemical risk assessment is performed for EDCs. Continued discussions have clarified exactly what controversies and challenges remain. We address several of these issues, including why the study and regulation of EDCs should incorporate endocrine principles; what level of consensus there is for low dose effects; challenges to our understanding of non-monotonicity; and whether EDCs have been demonstrated to produce adverse effects. This discussion should result in a better understanding of these issues, and allow for additional dialog on their impact on risk assessment.

Use and validation of HT/HC assays to support 21st century toxicity evaluations

Advances in high throughput and high content (HT/HC) methods such as those used in the fields of toxicogenomics, bioinformatics, and computational toxicology have the potential to improve both the efficiency and effectiveness of toxicity evaluations and risk assessments. However, prior to use, scientific confidence in these methods should be formally established. Traditional validation approaches that define relevance, reliability, sensitivity and specificity may not be readily applicable. HT/HC methods are not exact replacements for in vivo testing, and although run individually, these assays are likely to be used as a group or battery for decision making and use robotics, which may be unique in each laboratory setting. Building on the frameworks developed in the 2010 Institute of Medicine Report on Biomarkers and the OECD 2007 Report on (Q)SAR Validation, we present constructs that can be adapted to address the validation challenges of HT/HC methods. These are flexible, transparent, and require explicit specification of context and purpose of use such that scientific confidence (validation) can be defined to meet different regulatory applications. Using these constructs, we discuss how anchoring the assays and their prediction models to Adverse Outcome Pathways (AOPs) could facilitate the interpretation of results and support scientifically defensible fit-for-purpose applications.

Integrated testing strategies for safety assessments

Despite the fact that toxicology uses many stand-alone tests, a systematic combination of several information sources very often is required: Examples include: when not all possible outcomes of interest (e.g., modes of action), classes of test substances (applicability domains), or severity classes of effect are covered in a single test; when the positive test result is rare (low prevalence leading to excessive false-positive results); when the gold standard test is too costly or uses too many animals, creating a need for prioritization by screening. Similarly, tests are combined when the human predictivity of a single test is not satisfactory or when existing data and evidence from various tests will be integrated. Increasingly, kinetic information also will be integrated to make an in vivo extrapolation from in vitro data. Integrated Testing Strategies (ITS) offer the solution to these problems. ITS have been discussed for more than a decade, and some attempts have been made in test guidance for regulations. Despite their obvious potential for revamping regulatory toxicology, however, we still have little guidance on the composition, validation, and adaptation of ITS for different purposes. Similarly, Weight of Evidence and Evidence-based Toxicology approaches require different pieces of evidence and test data to be weighed and combined. ITS also represent the logical way of combining pathway-based tests, as suggested in Toxicology for the 21st Century. This paper describes the state of the art of ITS and makes suggestions as to the definition, systematic combination, and quality assurance of ITS.

Comparison of genotoxicant-modified transcriptomic responses in conventional and epigenetically stabilized primary rat hepatocytes with in vivo rat liver data

The concept of mechanistic toxicogenomics implies that compound-induced changes in gene expression profiles provide valuable information about their mode of action. A growing number of research groups have presented evidence that whole-genome gene expression profiling techniques might be used as tools for in vivo and in vitro generation of gene signatures and elucidation of molecular mechanisms after exposure to toxic compounds. An important issue to be investigated is the in vivo relevance of in vitro-obtained data. In the current study, we compare the gene expression profiles generated in vitro, after exposing conventional and epigenetically stabilized primary rat hepatocytes to well-known genotoxic hepatocarcinogens (aflatoxin B1, 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanone and 2-nitrofluorene) with those derived in vivo after oral exposure of rats to these compounds. Similar statistical tools were applied on both sets of data. The major molecular pathways affected in the in vivo setting were DNA damage, detoxification and cell survival response, as previously described. In the conventional hepatocyte cultures, two of the three genotoxicants showed quite similar responses as in vivo with respect to these pathways. The third compound (2-nitrofluorene) revealed in vitro response which was not observed in vivo. In the epigenetically stabilized hepatocytes, in contrast to what was expected, the responses were less relevant for the in vivo situation. This study highlights the importance of in vitro/in vivo comparison of data that are generated using in vitro models and shows that conventional primary rat hepatocyte cultures represent an appropriate in vitro model to retrieve mechanistic information on the exposure to genotoxicants.

Validation and quality control of replacement alternatives – current status and future challenges

Alternatives to animal testing have been developed mainly in the fields of toxicology and vaccine testing. Typical examples are the evaluation of phototoxicity, eye irritation or skin corrosion/irritation of cosmetics and industrial chemicals. However, examples can also be found in other biomedical areas, such the control of the quality of drug preparations for pyrogens or for the control of the production process of biologics, such as botulinum neurotoxin. For regulatory purposes, the quality, transferability and predictivity of an alternative method needs to be evaluated. This procedure is called the “validation process” of a new method. It follows defined rules, and several governmental institutions have been established to perform, supervise or advise on this process. As this often results in a delay of method implementation, different alternatives for the evaluation of a method's suitability and quality are under discussion. We describe here the principles of model development and quality control. We also give an overview on methods that have undergone validation. Strengths and shortcomings of traditional approaches are discussed, and new developments and challenges are outlined.

Basic data on the hematology, serum biochemistry, urology, and organ weights of beagle dogs

This study was conducted to provide basic data on physiological and hematological characteristics, and organ weights of beagle dogs. A total of 237 beagle dogs were used to determine differences in physiological and hematological parameters, and organ weights depending on sex and age. The respiratory rate of both sexes tended to increase as they grew older and the female heart rate was slightly higher than that of males. Male and female body weights increased rapidly to 33 weeks old followed by a gradual increase to 41-weeks-old. The relative weight of the brain was negatively correlated with body weight, whereas the weight of reproductive organs was positively correlated with body weight. The platelet count of female dogs was slightly higher than that of males. The red blood cell, hemoglobin, and hematocrit of both sexes increased non-significantly with age. In the leukocyte differential count, the neutrophils, and eosinophils of both sexes tended to increase as they grew older, whereas basophils, lymphocytes, and monocytes decreased. In the serum biochemical profiles, alkaline phosphatase was slightly higher in males than females, while the total cholesterol of female dogs at 9-months-old was higher than that of males at the same age. Other biochemical components, including alanine aminotransferase, blood urea nitrogen, creatinine, triglyceride, and total protein increased non-significantly with age in both sexes. To conclude, we observe no significant physiological or hematological differences with sex or age, although decreasing and increasing trends were detected with some parameters. These data provide valuable reference indices of the normal physiological and hematological characteristics of beagle dogs, which should prove useful in toxicological and pharmacological studies.

Hypothesis-driven weight of evidence framework for evaluating data within the US EPA's Endocrine Disruptor Screening Program

"Weight of Evidence" (WoE) approaches are often used to critically examine, prioritize, and integrate results from different types of studies to reach general conclusions. For assessing hormonally active agents, WoE evaluations are necessary to assess screening assays that identify potential interactions with components of the endocrine system, long-term reproductive and developmental toxicity tests that define adverse effects, mode of action studies aimed at identifying toxicological pathways underlying adverse effects, and toxicity, exposure and pharmacokinetic data to characterize potential risks. We describe a hypothesis-driven WoE approach for hormonally active agents and illustrate the approach by constructing hypotheses for testing the premise that a substance interacts as an agonist or antagonist with components of estrogen, androgen, or thyroid pathways or with components of the aromatase or steroidogenic enzyme systems for evaluating data within the US EPA's Endocrine Disruptor Screening Program. Published recommendations are used to evaluate data validity for testing each hypothesis and quantitative weightings are proposed to reflect two data parameters. Relevance weightings should be derived for each endpoint to reflect the degree to which it probes each specific hypothesis. Response weightings should be derived based on assay results from the test substance compared to the range of responses produced in the assay by the appropriate prototype hormone and positive and negative controls. Overall WoE scores should be derived based on response and relevance weightings and a WoE narrative developed to clearly describe the final determinations.

Alternative in vitro assays in nanomaterial toxicology

Nanomaterials are acclaimed for their novel properties, for which broad new uses are being discovered with increasing frequency. It is obvious that, as the properties change, unwanted properties (toxicity) are to be expected as well. Current toxicology, however, is already overwhelmed with the challenge of addressing new chemicals, not to mention the enormous number of old chemicals never properly assessed. Limitations of traditional approaches range from animal welfare issues, which were a strong driving force for alternative approaches (the 3Rs concept) over the last two decades, to aspects of throughput and accuracy of the predicted toxicities. The latter has prompted discussion about a revolutionary change in chemical safety assessment, now known as Toxicology for the 21st Century (Tox-21c). The multitude of possible formulations of nanomaterials to be assessed for novel toxic properties makes these alternative approaches especially attractive, given the well recognized limitations of traditional animal-based approaches--limitations that might be even more pronounced for nanomaterials, which have notably altered biokinetics.

Toxicity testing in the 21st century: a vision and a strategy

With the release of the landmark report Toxicity Testing in the 21st Century: A Vision and a Strategy, the U.S. National Academy of Sciences, in 2007, precipitated a major change in the way toxicity testing is conducted. It envisions increased efficiency in toxicity testing and decreased animal usage by transitioning from current expensive and lengthy in vivo testing with qualitative endpoints to in vitro toxicity pathway assays on human cells or cell lines using robotic high-throughput screening with mechanistic quantitative parameters. Risk assessment in the exposed human population would focus on avoiding significant perturbations in these toxicity pathways. Computational systems biology models would be implemented to determine the dose-response models of perturbations of pathway function. Extrapolation of in vitro results to in vivo human blood and tissue concentrations would be based on pharmacokinetic models for the given exposure condition. This practice would enhance human relevance of test results, and would cover several test agents, compared to traditional toxicological testing strategies. As all the tools that are necessary to implement the vision are currently available or in an advanced stage of development, the key prerequisites to achieving this paradigm shift are a commitment to change in the scientific community, which could be facilitated by a broad discussion of the vision, and obtaining necessary resources to enhance current knowledge of pathway perturbations and pathway assays in humans and to implement computational systems biology models. Implementation of these strategies would result in a new toxicity testing paradigm firmly based on human biology.

Toxicity testing in the 21st century: a vision and a strategy

With the release of the landmark report Toxicity Testing in the 21st Century: A Vision and a Strategy, the U.S. National Academy of Sciences, in 2007, precipitated a major change in the way toxicity testing is conducted. It envisions increased efficiency in toxicity testing and decreased animal usage by transitioning from current expensive and lengthy in vivo testing with qualitative endpoints to in vitro toxicity pathway assays on human cells or cell lines using robotic high-throughput screening with mechanistic quantitative parameters. Risk assessment in the exposed human population would focus on avoiding significant perturbations in these toxicity pathways. Computational systems biology models would be implemented to determine the dose-response models of perturbations of pathway function. Extrapolation of in vitro results to in vivo human blood and tissue concentrations would be based on pharmacokinetic models for the given exposure condition. This practice would enhance human relevance of test results, and would cover several test agents, compared to traditional toxicological testing strategies. As all the tools that are necessary to implement the vision are currently available or in an advanced stage of development, the key prerequisites to achieving this paradigm shift are a commitment to change in the scientific community, which could be facilitated by a broad discussion of the vision, and obtaining necessary resources to enhance current knowledge of pathway perturbations and pathway assays in humans and to implement computational systems biology models. Implementation of these strategies would result in a new toxicity testing paradigm firmly based on human biology.

Validating and troubleshooting ocular in vitro toxicology tests

In vitro organotypic models for testing ocular irritants have warranted sufficient interest as methods to replace in vivo ocular testing. The in vitro organotypic models claim to maintain short-term normal physiological and biochemical functions of the mammalian cornea in an isolated system. In these test methods, damage by the test substance is assessed by quantitative measurements of changes in corneal opacity and permeability using opacitometry and spectrophotometry, respectively. Both measurements are used quantitatively for irritancy classification for prediction of the in vivo ocular irritation potential of a test substance. Examples of organotypic models that incorporate these criteria include: the bovine corneal opacity and permeability (BCOP) assay, the isolated chicken eye (ICE) test method and the isolated rabbit eye (IRE) assay. A fourth method, the hen's egg test-chorioallantoic membrane (HET-CAM) assay, differs in the evaluation criteria but is also normally included among this class of in vitro protocols. Each of these protocols is discussed in detail as representative candidate in vitro methods for assessing ocular irritation and corrosion. The methodologies, protocol details, applications, and their validation status are discussed. A brief historical perspective of the development of original in vitro ocular testing models is also mentioned. More importantly, improving and troubleshooting the current techniques, in order to present the models as stand-alone in vitro tools for ocular toxicity assessment, is emphasized.

A toxicology for the 21st century--mapping the road ahead

The landmark publication by the National Research Council putting forward a vision of a toxicology for the 21st century in 2007 has created an atmosphere of departure in our field. The alliances formed, symposia and meetings held and the articles following are remarkable, indicating that this is an idea whose time has come. Most of the discussion centers on the technical opportunities to map pathways of toxicity and the financing of the program. Here, the other part of the work ahead shall be discussed, that is, the focus is on regulatory implementation once the technological challenges are managed, but we are well aware that the technical aspects of what the National Academy of Science report suggests still need to be addressed: A series of challenges are put forward which we will face in addition to finding a technical solution (and its funding) to set this vision into practice. This includes the standardization and quality assurance of novel methodologies, their formal validation, their integration into test strategies including threshold setting and finally a global acceptance and implementation. This will require intense conceptual steering to have all pieces of the puzzle come together.

Overview of alternative methodologies in toxicology

In vitro toxicology generally refers to the study of toxicological phenomena in non-whole-animal models. This broader connotation includes studies utilizing isolated organs, tissue slices, explants cultures, and isolated primary cells, as well as cell lines and subcellular fractions (e.g., microsomes). These model systems have made major contributions to toxicological sciences, particularly in our understanding of the mechanisms of toxicity, xenobiotic metabolism, and species differences in expressions of toxicity. This unit reviews the use of in vitro models and their value in toxicology testing.