How well can in vitro data predict in vivo effects of chemicals? Rodent carcinogenicity as a case study

Accurate clinical toxicity prediction using multi-task deep neural nets and contrastive molecular explanations

Explainable machine learning for molecular toxicity prediction is a promising approach for efficient drug development and chemical safety. A predictive ML model of toxicity can reduce experimental cost and time while mitigating ethical concerns by significantly reducing animal and clinical testing. Herein, we use a deep learning framework for simultaneously modeling in vitro, in vivo, and clinical toxicity data. Two different molecular input representations are used; Morgan fingerprints and pre-trained SMILES embeddings. A multi-task deep learning model accurately predicts toxicity for all endpoints, including clinical, as indicated by the area under the Receiver Operator Characteristic curve and balanced accuracy. In particular, pre-trained molecular SMILES embeddings as input to the multi-task model improved clinical toxicity predictions compared to existing models in MoleculeNet benchmark. Additionally, our multitask approach is comprehensive in the sense that it is comparable to state-of-the-art approaches for specific endpoints in in vitro, in vivo and clinical platforms. Through both the multi-task model and transfer learning, we were able to indicate the minimal need of in vivo data for clinical toxicity predictions. To provide confidence and explain the model's predictions, we adapt a post-hoc contrastive explanation method that returns pertinent positive and negative features, which correspond well to known mutagenic and reactive toxicophores, such as unsubstituted bonded heteroatoms, aromatic amines, and Michael receptors. Furthermore, toxicophore recovery by pertinent feature analysis captures more of the in vitro (53%) and in vivo (56%), rather than of the clinical (8%), endpoints, and indeed uncovers a preference in known toxicophore data towards in vitro and in vivo experimental data. To our knowledge, this is the first contrastive explanation, using both present and absent substructures, for predictions of clinical and in vivo molecular toxicity.

Towards the sustainable discovery and development of new antibiotics

An ever-increasing demand for novel antimicrobials to treat life-threatening infections caused by the global spread of multidrug-resistant bacterial pathogens stands in stark contrast to the current level of investment in their development, particularly in the fields of natural-product-derived and synthetic small molecules. New agents displaying innovative chemistry and modes of action are desperately needed worldwide to tackle the public health menace posed by antimicrobial resistance. Here, our consortium presents a strategic blueprint to substantially improve our ability to discover and develop new antibiotics. We propose both short-term and long-term solutions to overcome the most urgent limitations in the various sectors of research and funding, aiming to bridge the gap between academic, industrial and political stakeholders, and to unite interdisciplinary expertise in order to efficiently fuel the translational pipeline for the benefit of future generations.

Antimicrobial resistance is an increasing threat to public health and encouraging the development of new antimicrobials is one of the most important ways to address the problem. This Roadmap article aims to bring together industrial, academic and political partners, and proposes both short-term and long-term solutions to this challenge.

Predictive modeling of biological responses in the rat liver using in vitro Tox21 bioactivity: Benefits from high-throughput toxicokinetics

Computational methods are needed to more efficiently leverage data from in vitro cell-based models to predict what occurs within whole body systems after chemical insults. This study set out to test the hypothesis that in vitro high-throughput screening (HTS) data can more effectively predict in vivo biological responses when chemical disposition and toxicokinetic (TK) modeling are employed. In vitro HTS data from the Tox21 consortium were analyzed in concert with chemical disposition modeling to derive nominal, aqueous, and intracellular estimates of concentrations eliciting 50% maximal activity. In vivo biological responses were captured using rat liver transcriptomic data from the DrugMatrix and TG-Gates databases and evaluated for pathway enrichment. In vivo dosing data were translated to equivalent body concentrations using HTTK modeling. Random forest models were then trained and tested to predict in vivo pathway-level activity across 221 chemicals using in vitro bioactivity data and physicochemical properties as predictor variables, incorporating methods to address imbalanced training data resulting from high instances of inactivity. Model performance was quantified using the area under the receiver operator characteristic curve (AUC-ROC) and compared across pathways for different combinations of predictor variables. All models that included toxicokinetics were found to outperform those that excluded toxicokinetics. Biological interpretation of the model features revealed that rather than a direct mapping of in vitro assays to in vivo pathways, unexpected combinations of multiple in vitro assays predicted in vivo pathway-level activities. To demonstrate the utility of these findings, the highest-performing model was leveraged to make new predictions of in vivo biological responses across all biological pathways for remaining chemicals tested in Tox21 with adequate data coverage (n = 6617). These results demonstrate that, when chemical disposition and toxicokinetics are carefully considered, in vitro HT screening data can be used to effectively predict in vivo biological responses to chemicals.

Recommendations for further revisions to improve the International Agency for Research on Cancer (IARC) Monograph program

In 2019, the International Agency for Research on Cancer (IARC) "Preamble to the IARC Monographs" expanded guidance regarding the scientific approaches that should be employed in its monographs. These amendments to the monograph development process are an improvement but still fall short in several areas. While the revised Preamble lays out broad methods and approaches to evaluate scientific evidence, there is a lack of specificity with regard to how IARC Working Groups will conduct consistent evaluations in a standardized, objective, and transparent manner; document systematic review and evidence integration actions, and substantiate how these actions and decisions inform the ultimate classifications. Furthermore, no guidance is provided to ensure Working Groups consistently incorporate mechanistic evidence in a robust manner using a defined approach in the context of 21st century knowledge of modes of action. Nor are the conclusions of the working groups subjected to outside, independent scientific peer review. Continued improvements and modernization of the procedures for evaluating, presenting, and communicating study quality, and in the methods used to conduct and peer-review evidence-based decision making will benefit the Working Group members, the IARC Monographs Programme overall, and the international regulatory community and public who rely upon the monographs.

A Case Study with Triazole Fungicides to Explore Practical Application of Next-Generation Hazard Assessment Methods for Human Health

The ongoing developments in chemical risk assessment have led to new concepts building on integration of sophisticated nonanimal models for hazard characterization. Here we explore a pragmatic approach for implementing such concepts, using a case study of three triazole fungicides, namely, flusilazole, propiconazole, and cyproconazole. The strategy applied starts with evaluating the overall level of concern by comparing exposure estimates to toxicological potential, followed by a combination of in silico tools and literature-derived high-throughput screening assays and computational elaborations to obtain insight into potential toxicological mechanisms and targets in the organism. Additionally, some targeted in vitro tests were evaluated for their utility to confirm suspected mechanisms of toxicity and to generate points of departure. Toxicological mechanisms instead of the current "end point-by-end point" approach should guide the selection of methods and assays that constitute a toolbox for next-generation risk assessment. Comparison of the obtained in silico and in vitro results with data from traditional in vivo testing revealed that, overall, nonanimal methods for hazard identification can produce adequate qualitative hazard information for risk assessment. Follow-up studies are needed to further refine the proposed approach, including the composition of the toolbox, toxicokinetics models, and models for exposure assessment.

Dose-Dependent Effects of Long-Term Administration of Hydrogen Sulfide on Myocardial Ischemia-Reperfusion Injury in Male Wistar Rats: Modulation of RKIP, NF-κB, and Oxidative Stress

Decreased circulating levels of hydrogen sulfide (H₂S) are associated with higher mortality following myocardial ischemia. This study aimed at determining the long-term dose-dependent effects of sodium hydrosulfide (NaSH) administration on myocardial ischemia-reperfusion (IR) injury. Male rats were divided into control and NaSH groups that were treated for 9 weeks with daily intraperitoneal injections of normal saline or NaSH (0.28, 0.56, 1.6, 2.8, and 5.6 mg/kg), respectively. At the end of the study, hearts from all rats were isolated and hemodynamic parameters were recorded during baseline and following IR. In isolated hearts, infarct size, oxidative stress indices as well as mRNA expression of H₂S-, nitric oxide (NO)-producing enzymes, and inflammatory markers were measured. In heart tissue following IR, low doses of NaSH (0.28 and 0.56 mg/kg) had no effect, whereas an intermediate dose (1.6 mg/kg), improved recovery of hemodynamic parameters, decreased infarct size, and decreased oxidative stress. It also increased expression of cystathionine γ-lyase (CSE), Raf kinase inhibitor protein (RKIP), endothelial NO synthase (eNOS), and neuronal NOS (nNOS), as well as decreased expression of inducible NOS (iNOS) and nuclear factor kappa-B (NF-κB). At the high dose of 5.6 mg/kg, NaSH administration was associated with worse recovery of hemodynamic parameters and increased infarct size as well as increased oxidative stress. This dose also decreased expression of CSE, RKIP, and eNOS and increased expression of iNOS and NF-κB. In conclusion, chronic treatment with NaSH has a U-shaped concentration effect on IR injury in heart tissue. An intermediate dose was associated with higher CSE-derived H₂S, lower iNOS-derived NO, lower oxidative stress, and inflammation in heart tissue following IR.

Progress in data interoperability to support computational toxicology and chemical safety evaluation

New approach methodologies (NAMs) in chemical safety evaluation are being explored to address the current public health implications of human environmental exposures to chemicals with limited or no data for assessment. For over a decade since a push toward "Toxicity Testing in the 21st Century," the field has focused on massive data generation efforts to inform computational approaches for preliminary hazard identification, adverse outcome pathways that link molecular initiating events and key events to apical outcomes, and high-throughput approaches to risk-based ratios of bioactivity and exposure to inform relative priority and safety assessment. Projects like the interagency Tox21 program and the US EPA ToxCast program have generated dose-response information on thousands of chemicals, identified and aggregated information from legacy systems, and created tools for access and analysis. The resulting information has been used to develop computational models as viable options for regulatory applications. This progress has introduced challenges in data management that are new, but not unique, to toxicology. Some of the key questions require critical thinking and solutions to promote semantic interoperability, including: (1) identification of bioactivity information from NAMs that might be related to a biological process; (2) identification of legacy hazard information that might be related to a key event or apical outcomes of interest; and, (3) integration of these NAM and traditional data for computational modeling and prediction of complex apical outcomes such as carcinogenesis. This work reviews a number of toxicology-related efforts specifically related to bioactivity and toxicological data interoperability based on the goals established by Findable, Accessible, Interoperable, and Reusable (FAIR) Data Principles. These efforts are essential to enable better integration of NAM and traditional toxicology information to support data-driven toxicology applications.

Quality Control of Quantitative High Throughput Screening Data

Quantitative high throughput screening (qHTS) experiments can generate 1000s of concentration-response profiles to screen compounds for potentially adverse effects. However, potency estimates for a single compound can vary considerably in study designs incorporating multiple concentration-response profiles for each compound. We introduce an automated quality control procedure based on analysis of variance (ANOVA) to identify and filter out compounds with multiple cluster response patterns and improve potency estimation in qHTS assays. Our approach, called Cluster Analysis by Subgroups using ANOVA (CASANOVA), clusters compound-specific response patterns into statistically supported subgroups. Applying CASANOVA to 43 publicly available qHTS data sets, we found that only about 20% of compounds with response values outside of the noise band have single cluster responses. The error rates for incorrectly separating true clusters and incorrectly clumping disparate clusters were both less than 5% in extensive simulation studies. Simulation studies also showed that the bias and variance of concentration at half-maximal response (AC50 ) estimates were usually within 10-fold when using a weighted average approach for potency estimation. In short, CASANOVA effectively sorts out compounds with "inconsistent" response patterns and produces trustworthy AC50 values.

Identifying Attributes That Influence In Vitro-to-In Vivo Concordance by Comparing In Vitro Tox21 Bioactivity Versus In Vivo DrugMatrix Transcriptomic Responses Across 130 Chemicals

Recent efforts aimed at integrating in vitro high-throughput screening (HTS) data into chemical toxicity assessments are necessitating increased understanding of concordance between chemical-induced responses observed in vitro versus in vivo. This investigation set out to (1) measure concordance between in vitro HTS data and transcriptomic responses observed in vivo, focusing on the liver, and (2) identify attributes that can influence concordance. Signal response profiles from 130 substances were compared between in vitro data produced through Tox21 and liver transcriptomic data through DrugMatrix, collected from rats exposed to a chemical for ≤5 days. A global in vitro-to-in vivo comparative analysis based on pathway-level responses resulted in an overall average percent agreement of 79%, ranging on a per-chemical basis between 41% and 100%. Whereas concordance amongst inactive chemicals was high (89%), concordance amongst chemicals showing in vitro activity was only 13%, suggesting that follow-up in vivo and/or orthogonal in vitro assays would improve interpretations of in vitro activity. Attributes identified to influence concordance included experimental design attributes (eg, cell type), target pathways, and physicochemical properties (eg, logP). The attribute that most consistently increased concordance was dose applicability, evaluated by filtering for experimental doses administered to rats that were within 10-fold of those related to likely bioactivity, derived using Tox21 data and high-throughput toxicokinetic modeling. Together, findings suggest that in vitro screening approaches to predict in vivo toxicity are viable particularly when certain attributes are considered, including whether activity versus inactivity is observed, experimental design, chemical properties, and dose applicability.

A review of the genotoxic, mutagenic, and carcinogenic potentials of several lower acrylates

Lower alkyl acrylate monomers include methyl-, ethyl-, n-butyl-, and 2-ethylhexyl acrylate. These acrylates are used in the manufacture of acrylic polymers and copolymers for plastics, food packaging, adhesives, and cosmetic formulations. Although there is limited potential for human environmental exposure, occupational exposure can occur via inhalation and dermal contact. Recently, new genotoxicity data have been generated, along with in silico and in vitro read-cross analyses, for these acrylates. The availability of high-throughput screening (HTS) data through the ToxCast™/Tox21 databases allows for consideration of computational toxicology and organization of these data according to the ten key characteristics of carcinogens. Therefore, we conducted a comprehensive review to evaluate the mechanistic, toxicokinetic, animal, and human data, including HTS data, for characterizing the potential carcinogenicity, mutagenicity, and genotoxicity of these acrylates. Toxicokinetic data demonstrate that these acrylates are metabolized rapidly by carboxylesterase hydrolysis and conjugation with glutathione. HTS data demonstrated an overall lack of bioactivity in cancer-related pathways. Overall, the genotoxicity and mutagenicity data support a cytotoxic, non-genotoxic mechanism for these acrylates. Cancer bioassay studies conducted by the oral, dermal, and inhalation routes in animal models with these acrylates did not show any increase in tumor incidence, with two exceptions. At high doses, and secondary to chronic site-of-contact irritation and corrosion, rodent forestomach tumors were induced by oral gavage dosing with ethyl acrylate, and skin tumors were observed following chronic dermal dosing with 2-ethylhexyl acrylate in C3H/HeJ inbred mice (a strain with deficiencies in wound healing), but not in the outbred NMRI strain. For both dermal and forestomach cancers, tumorigenesis is secondary to high doses and long-term tissue damage, shown to be reversible. With evidence that these chemicals are not genotoxic, and that they cause forestomach and dermal tumors through chronic irritation and regenerative proliferation mechanisms, these acrylates are unlikely to pose a human cancer hazard.

How well can carcinogenicity be predicted by high throughput "characteristics of carcinogens" mechanistic data?

IARC has begun using ToxCast/Tox21 data in efforts to represent key characteristics of carcinogens to organize and weigh mechanistic evidence in cancer hazard determinations and this implicit inference approach also is being considered by USEPA. To determine how well ToxCast/Tox21 data can explicitly predict cancer hazard, this approach was evaluated with statistical analyses and machine learning prediction algorithms. Substances USEPA previously classified as having cancer hazard potential were designated as positives and substances not posing a carcinogenic hazard were designated as negatives. Then ToxCast/Tox21 data were analyzed both with and without adjusting for the cytotoxicity burst effect commonly observed in such assays. Using the same assignments as IARC of ToxCast/Tox21 assays to the seven key characteristics of carcinogens, the ability to predict cancer hazard for each key characteristic, alone or in combination, was found to be no better than chance. Hence, we have little scientific confidence in IARC's inference models derived from current ToxCast/Tox21 assays for key characteristics to predict cancer. This finding supports the need for a more rigorous mode-of-action pathway-based framework to organize, evaluate, and integrate mechanistic evidence with animal toxicity, epidemiological investigations, and knowledge of exposure and dosimetry to evaluate potential carcinogenic hazards and risks to humans.

High-Throughput Screening Data Interpretation in the Context of In Vivo Transcriptomic Responses to Oral Cr(VI) Exposure

The toxicity of hexavalent chromium [Cr(VI)] in drinking water has been studied extensively, and available in vivo and in vitro studies provide a robust dataset for application of advanced toxicological tools to inform the mode of action (MOA). This study aimed to contribute to the understanding of Cr(VI) MOA by evaluating high-throughput screening (HTS) data and other in vitro data relevant to Cr(VI), and comparing these findings to robust in vivo data, including transcriptomic profiles in target tissues. Evaluation of Tox21 HTS data for Cr(VI) identified 11 active assay endpoints relevant to the Ten Key Characteristics of Carcinogens (TKCCs) that have been proposed by other investigators. Four of these endpoints were related to TP53 (tumor protein 53) activation mapping to genotoxicity (KCC#2), and four were related to cell death/proliferation (KCC#10). HTS results were consistent with other in vitro data from the Comparative Toxicogenomics Database. In vitro responses were compared to in vivo transcriptomic responses in the most sensitive target tissue, the duodenum, of mice exposed to ≤ 180 ppm Cr(VI) for 7 and 90 days. Pathways that were altered both in vitro and in vivo included those relevant to cell death/proliferation. In contrast, pathways relevant to p53/DNA damage were identified in vitro but not in vivo. Benchmark dose modeling and phenotypic anchoring of in vivo transcriptomic responses strengthened the finding that Cr(VI) causes cell stress/injury followed by proliferation in the mouse duodenum at high doses. These findings contribute to the body of evidence supporting a non-mutagenic MOA for Cr(VI)-induced intestinal cancer.

Improving the International Agency for Research on Cancer's consideration of mechanistic evidence

BACKGROUND

The International Agency for Research on Cancer (IARC) recently developed a framework for evaluating mechanistic evidence that includes a list of 10 key characteristics of carcinogens. This framework is useful for identifying and organizing large bodies of literature on carcinogenic mechanisms, but it lacks sufficient guidance for conducting evaluations that fully integrate mechanistic evidence into hazard assessments.

OBJECTIVES

We summarize the framework, and suggest approaches to strengthen the evaluation of mechanistic evidence using this framework.

DISCUSSION

While the framework is useful for organizing mechanistic evidence, its lack of guidance for implementation limits its utility for understanding human carcinogenic potential. Specifically, it does not include explicit guidance for evaluating the biological significance of mechanistic endpoints, inter- and intra-individual variability, or study quality and relevance. It also does not explicitly address how mechanistic evidence should be integrated with other realms of evidence. Because mechanistic evidence is critical to understanding human cancer hazards, we recommend that IARC develop transparent and systematic guidelines for the use of this framework so that mechanistic evidence will be evaluated and integrated in a robust manner, and concurrently with other realms of evidence, to reach a final human cancer hazard conclusion.

CONCLUSIONS

IARC does not currently provide a standardized approach to evaluating mechanistic evidence. Incorporating the recommendations discussed here will make IARC analyses of mechanistic evidence more transparent, and lead to assessments of cancer hazards that reflect the weight of the scientific evidence and allow for scientifically defensible decision-making.