Workflow for defining reference chemicals for assessing performance of in vitro assays

Searching for LINCS to Stress: Using Text Mining to Automate Reference Chemical Curation

Adaptive stress response pathways (SRPs) restore cellular homeostasis following perturbation but may activate terminal outcomes like apoptosis, autophagy, or cellular senescence if disruption exceeds critical thresholds. Because SRPs hold the key to vital cellular tipping points, they are targeted for therapeutic interventions and assessed as biomarkers of toxicity. Hence, we are developing a public database of chemicals that perturb SRPs to enable new data-driven tools to improve public health. Here, we report on the automated text-mining pipeline we used to build and curate the first version of this database. We started with 100 reference SRP chemicals gathered from published biomarker studies to bootstrap the database. Second, we used information retrieval to find co-occurrences of reference chemicals with SRP terms in PubMed abstracts and determined pairwise mutual information thresholds to filter biologically relevant relationships. Third, we applied these thresholds to find 1206 putative SRP perturbagens within thousands of substances in the Library of Integrated Network-Based Cellular Signatures (LINCS). To assign SRP activity to LINCS chemicals, domain experts had to manually review at least three publications for each of 1206 chemicals out of 181,805 total abstracts. To accomplish this efficiently, we implemented a machine learning approach to predict SRP classifications from texts to prioritize abstracts. In 5-fold cross-validation testing with a corpus derived from the 100 reference chemicals, artificial neural networks performed the best (F1-macro = 0.678) and prioritized 2479/181,805 abstracts for expert review, which resulted in 457 chemicals annotated with SRP activities. An independent analysis of enriched mechanisms of action and chemical use class supported the text-mined chemical associations (p < 0.05): heat shock inducers were linked with HSP90 and DNA damage inducers to topoisomerase inhibition. This database will enable novel applications of LINCS data to evaluate SRP activities and to further develop tools for biomedical information extraction from the literature.

Navigating Transcriptomic Connectivity Mapping Workflows to Link Chemicals with Bioactivities

Screening new compounds for potential bioactivities against cellular targets is vital for drug discovery and chemical safety. Transcriptomics offers an efficient approach for assessing global gene expression changes, but interpreting chemical mechanisms from these data is often challenging. Connectivity mapping is a potential data-driven avenue for linking chemicals to mechanisms based on the observation that many biological processes are associated with unique gene expression signatures (gene signatures). However, mining the effects of a chemical on gene signatures for biological mechanisms is challenging because transcriptomic data contain thousands of noisy genes. New connectivity mapping approaches seeking to distinguish signal from noise continue to be developed, spurred by the promise of discovering chemical mechanisms, new drugs, and disease targets from burgeoning transcriptomic data. Here, we analyze these approaches in terms of diverse transcriptomic technologies, public databases, gene signatures, pattern-matching algorithms, and statistical evaluation criteria. To navigate the complexity of connectivity mapping, we propose a harmonized scheme to coherently organize and compare published workflows. We first standardize concepts underlying transcriptomic profiles and gene signatures based on various transcriptomic technologies such as microarrays, RNA-Seq, and L1000 and discuss the widely used data sources such as Gene Expression Omnibus, ArrayExpress, and MSigDB. Next, we generalize connectivity mapping as a pattern-matching task for finding similarity between a query (e.g., transcriptomic profile for new chemical) and a reference (e.g., gene signature of known target). Published pattern-matching approaches fall into two main categories: vector-based use metrics like correlation, Jaccard index, etc., and aggregation-based use parametric and nonparametric statistics (e.g., gene set enrichment analysis). The statistical methods for evaluating the performance of different approaches are described, along with comparisons reported in the literature on benchmark transcriptomic data sets. Lastly, we review connectivity mapping applications in toxicology and offer guidance on evaluating chemical-induced toxicity with concentration-response transcriptomic data. In addition to serving as a high-level guide and tutorial for understanding and implementing connectivity mapping workflows, we hope this review will stimulate new algorithms for evaluating chemical safety and drug discovery using transcriptomic data.

A framework for establishing scientific confidence in new approach methodologies

Robust and efficient processes are needed to establish scientific confidence in new approach methodologies (NAMs) if they are to be considered for regulatory applications. NAMs need to be fit for purpose, reliable and, for the assessment of human health effects, provide information relevant to human biology. They must also be independently reviewed and transparently communicated. Ideally, NAM developers should communicate with stakeholders such as regulators and industry to identify the question(s), and specified purpose that the NAM is intended to address, and the context in which it will be used. Assessment of the biological relevance of the NAM should focus on its alignment with human biology, mechanistic understanding, and ability to provide information that leads to health protective decisions, rather than solely comparing NAM-based chemical testing results with those from traditional animal test methods. However, when NAM results are compared to historical animal test results, the variability observed within animal test method results should be used to inform performance benchmarks. Building on previous efforts, this paper proposes a framework comprising five essential elements to establish scientific confidence in NAMs for regulatory use: fitness for purpose, human biological relevance, technical characterization, data integrity and transparency, and independent review. Universal uptake of this framework would facilitate the timely development and use of NAMs by the international community. While this paper focuses on NAMs for assessing human health effects of pesticides and industrial chemicals, many of the suggested elements are expected to apply to other types of chemicals and to ecotoxicological effect assessments.

“In Litero” Screening: Retrospective Evaluation of Clinical Evidence to Establish a Reference List of Human Chemical Respiratory Sensitizers

Despite decades of investigation, test methods to identify respiratory sensitizers remain an unmet regulatory need. In order to support the evaluation of New Approach Methodologies in development, we sought to establish a reference set of low molecular weight respiratory sensitizers based on case reports of occupational asthma. In this context, we have developed an “in litero” approach to identify cases of low molecular weight chemical exposures leading to respiratory sensitization in clinical literature. We utilized the EPA-developed Abstract Sifter literature review tool to maximize the retrieval of publications relevant to respiratory effects in humans for each chemical in a list of chemicals suspected of inducing respiratory sensitization. The literature retrieved for each of these candidate chemicals was sifted to identify relevant case reports and studies, and then evaluated by applying defined selection criteria. Clinical diagnostic criteria were defined around exposure history, respiratory effects, and specific immune response to conclusively demonstrate occupational asthma as a result of sensitization, rather than irritation. This approach successfully identified 28 chemicals that can be considered as human respiratory sensitizers and used to evaluate the performance of NAMs as part of a weight of evidence approach to identify novel respiratory sensitizers. Further, these results have immediate implications for the development and refinement of predictive tools to distinguish between skin and respiratory sensitizers. A comparison of the protein binding mechanisms of our identified “in litero” clinical respiratory sensitizers shows that acylation is a prevalent protein binding mechanism, in contrast to Michael addition and Schiff base formation common to skin sensitizers. Overall, this approach provides an exemplary method to evaluate and apply human data as part of the weight of evidence when establishing reference chemical lists.

Identifying candidate reference chemicals for in vitro testing of the retinoid pathway for predictive developmental toxicity

Evaluating chemicals for potential in vivo toxicity based on their in vitro bioactivity profile is an important step toward animal- free testing. A compendium of reference chemicals and data describing their bioactivity on specific molecular targets, cellular pathways, and biological processes is needed to bolster confidence in the predictive value of in vitro hazard detection. Endogenous signaling by all-trans retinoic acid (ATRA) is an important pathway in developmental processes and toxicities. Employing data extraction methods and advanced literature extraction tools, we assembled a set of candidate reference chemicals with demonstrated activity on ten protein family targets in the retinoid system. The compendium was culled from Protein Data Bank, ChEMBL, ToxCast/Tox21, and the biomedical literature in PubMed. Finally, we performed a case study on one chemical in our collection, citral, an inhibitor of endogenous ATRA production, to determine whether the literature supports an adverse outcome pathway explaining the compound’s developmental toxicity initiated by disruption of the retinoid pathway. We also deliver an updated Abstract Sifter tool populated with these reference compounds and complex search terms designed to query the literature for the downstream consequences to support concordance with targeted retinoid pathway disruption.

Predicting molecular initiating events using chemical target annotations and gene expression

BACKGROUND

The advent of high-throughput transcriptomic screening technologies has resulted in a wealth of publicly available gene expression data associated with chemical treatments. From a regulatory perspective, data sets that cover a large chemical space and contain reference chemicals offer utility for the prediction of molecular initiating events associated with chemical exposure. Here, we integrate data from a large compendium of transcriptomic responses to chemical exposure with a comprehensive database of chemical-protein associations to train binary classifiers that predict mechanism(s) of action from transcriptomic responses. First, we linked reference chemicals present in the LINCS L1000 gene expression data collection to chemical identifiers in RefChemDB, a database of chemical-protein interactions. Next, we trained binary classifiers on MCF7 human breast cancer cell line derived gene expression profiles and chemical-protein labels using six classification algorithms to identify optimal analysis parameters. To validate classifier accuracy, we used holdout data sets, training-excluded reference chemicals, and empirical significance testing of null models derived from permuted chemical-protein associations. To identify classifiers that have variable predicting performance across training data derived from different cellular contexts, we trained a separate set of binary classifiers on the PC3 human prostate cancer cell line.

RESULTS

We trained classifiers using expression data associated with chemical treatments linked to 51 molecular initiating events. This analysis identified and validated 9 high-performing classifiers with empirical p-values lower than 0.05 and internal accuracies ranging from 0.73 to 0.94 and holdout accuracies of 0.68 to 0.92. High-ranking predictions for training-excluded reference chemicals demonstrating that predictive accuracy extends beyond the set of chemicals used in classifier training. To explore differences in classifier performance as a function of training data cellular context, MCF7-trained classifier accuracies were compared to classifiers trained on the PC3 gene expression data for the same molecular initiating events.

CONCLUSIONS

This methodology can offer insight in prioritizing candidate perturbagens of interest for targeted screens. This approach can also help guide the selection of relevant cellular contexts for screening classes of candidate perturbagens using cell line specific model performance.

High-throughput toxicogenomic screening of chemicals in the environment using metabolically competent hepatic cell cultures

The ToxCast in vitro screening program has provided concentration-response bioactivity data across more than a thousand assay endpoints for thousands of chemicals found in our environment and commerce. However, most ToxCast screening assays have evaluated individual biological targets in cancer cell lines lacking integrated physiological functionality (such as receptor signaling, metabolism). We evaluated differentiated HepaRGTM cells, a human liver-derived cell model understood to effectively model physiologically relevant hepatic signaling. Expression of 93 gene transcripts was measured by quantitative polymerase chain reaction using Fluidigm 96.96 dynamic arrays in response to 1060 chemicals tested in eight-point concentration-response. A Bayesian framework quantitatively modeled chemical-induced changes in gene expression via six transcription factors including: aryl hydrocarbon receptor, constitutive androstane receptor, pregnane X receptor, farnesoid X receptor, androgen receptor, and peroxisome proliferator-activated receptor alpha. For these chemicals the network model translates transcriptomic data into Bayesian inferences about molecular targets known to activate toxicological adverse outcome pathways. These data also provide new insights into the molecular signaling network of HepaRGTM cell cultures.

Progress in data interoperability to support computational toxicology and chemical safety evaluation

New approach methodologies (NAMs) in chemical safety evaluation are being explored to address the current public health implications of human environmental exposures to chemicals with limited or no data for assessment. For over a decade since a push toward "Toxicity Testing in the 21st Century," the field has focused on massive data generation efforts to inform computational approaches for preliminary hazard identification, adverse outcome pathways that link molecular initiating events and key events to apical outcomes, and high-throughput approaches to risk-based ratios of bioactivity and exposure to inform relative priority and safety assessment. Projects like the interagency Tox21 program and the US EPA ToxCast program have generated dose-response information on thousands of chemicals, identified and aggregated information from legacy systems, and created tools for access and analysis. The resulting information has been used to develop computational models as viable options for regulatory applications. This progress has introduced challenges in data management that are new, but not unique, to toxicology. Some of the key questions require critical thinking and solutions to promote semantic interoperability, including: (1) identification of bioactivity information from NAMs that might be related to a biological process; (2) identification of legacy hazard information that might be related to a key event or apical outcomes of interest; and, (3) integration of these NAM and traditional data for computational modeling and prediction of complex apical outcomes such as carcinogenesis. This work reviews a number of toxicology-related efforts specifically related to bioactivity and toxicological data interoperability based on the goals established by Findable, Accessible, Interoperable, and Reusable (FAIR) Data Principles. These efforts are essential to enable better integration of NAM and traditional toxicology information to support data-driven toxicology applications.