Implementing systematic review at the National Toxicology Program: status and next steps

Ambient temperature and mental health: a systematic review and meta-analysis

BACKGROUND

Increasing evidence indicates that ambient outdoor temperature could affect mental health, which is especially concerning in the context of climate change. We aimed to comprehensively analyse the current evidence regarding the associations between ambient temperature and mental health outcomes.

METHODS

We did a systematic review and meta-analysis of the evidence regarding associations between ambient outdoor temperature and changes in mental health outcomes. We searched WebOfScience, Embase, PsychINFO, and PubMed for articles published from database origin up to April 7, 2022. Eligible articles were epidemiological, observational studies in humans of all ages, which evaluated real-world responses to ambient outdoor temperature, and had mental health as a documented outcome; studies of manipulated or controlled temperature or those with only physical health outcomes were excluded. All eligible studies were synthesised qualitatively. If three or more studies reported the same or equivalent effect statistics and if they had equivalent exposure, outcome, and metrics, the studies were pooled in a random-effects meta-analysis. The risk of bias for individual studies was assessed using the Newcastle-Ottawa Scale. The quality of evidence across studies was assessed using the Office of Health Assessment and Translation (OHAT) approach.

FINDINGS

114 studies were included in the systematic review, of which 19 were suitable for meta-analysis. Three meta-analyses were conducted for suicide outcomes: a 1°C increase in mean monthly temperature was associated with an increase in incidence of 1·5% (95% CI 0·8-2·2, p<0·001; n=1 563 109, seven effects pooled from three studies); a 1°C increase in mean daily temperature was associated with an increase in incidence of 1·7% (0·3-3·0, p=0·014; n=113 523, five effects pooled from five studies); and a 1°C increase in mean monthly temperature was associated with a risk ratio of 1·01 (95% CI 1·00-1·01, p<0·001; n=111 794, six effects pooled from three studies). Three meta-analyses were conducted for hospital attendance or admission for mental illness: heatwaves versus non-heatwave periods were associated with an increase in incidence of 9·7% (95% CI 7·6-11·9, p<0·001; n=362 086, three studies); the risk ratio at the 99th percentile of daily mean temperature compared with the 50th percentile was 1·02 (95% CI 1·01-1·03, p=0·006; n=532 296, three studies); and no significant association was found between a 10°C increase in daily mean temperature and hospital attendance. In a qualitative narrative synthesis, we found that ambient outdoor temperature (including absolute temperatures, temperature variability, and heatwaves) was positively associated with attempted and completed suicides (86 studies), hospital attendance or admission for mental illness (43 studies), and worse outcomes for community mental health and wellbeing (19 studies), but much of the evidence was of low certainty with high heterogeneity.

INTERPRETATION

Increased temperature and temperature variability could be associated with increased cases of suicide and suicidal behaviour, hospital attendance or admission for mental illness, and poor community health and wellbeing. Climate change is likely to increase temperature anomalies, variability, and heatwaves as well as average temperatures; as such, health system leaders and policy makers must be adequately prepared and should develop adaptation strategies. More high-quality, standardised research is required to improve our understanding of these effects.

FUNDING

None.

Does paraquat cause Parkinson's disease? A review of reviews

To examine the extent to which a consensus exists in the scientific community regarding the relationship between exposure to paraquat and Parkinson's disease, a critical review of reviews was undertaken focusing on reviews published between 2006 and the present that offered opinions on the issue of causation. Systematic searches were undertaken of scientific databases along with searches of published bibliographies to identify English language reviews on the topic of paraquat and Parkinson's disease including those on the broader topic of environmental and occupational risk factors for Parkinson's disease. Of the 269 publications identified in the searches, there were twelve reviews, some with meta-analyses, that met the inclusion criteria. Information on methods used by the reviewers, if any, and source of funding was collected; the quality of the reviews was considered. No author of any published review stated that it has been established that exposure to paraquat causes Parkinson's disease, regardless of methods used and independent of funding source. A consensus exists in the scientific community that the available evidence does not warrant a claim that paraquat causes Parkinson's disease. Future research on this topic should focus on improving the quality of epidemiological studies including better exposure measures and identifying specific mechanisms of action. Future reviews of emerging evidence should be structured as systematic narrative reviews with meta-analysis if appropriate.

Novel text analytics approach to identify relevant literature for human health risk assessments: A pilot study with health effects of in utero exposures

Systematic reviews involve mining literature databases to identify relevant studies. Identifying potentially relevant studies can be informed by computational tools comparing text similarity between candidate studies and selected key (i.e., seed) references. Challenge Using computational approaches to identify relevant studies for risk assessments is challenging, as these assessments examine multiple chemical effects across lifestages (e.g., human health risk assessments) or specific effects of multiple chemicals (e.g., cumulative risk). The broad scope of potentially relevant literature can make selection of seed references difficult. Approach We developed a generalized computational scoping strategy to identify human health relevant studies for multiple chemicals and multiple effects. We used semi-supervised machine learning to prioritize studies to review manually with training data derived from references cited in the hazard identification sections of several US EPA Integrated Risk Information System (IRIS) assessments. These generic training data or seed studies were clustered with the unclassified corpus to group studies based on text similarity. Clusters containing a high proportion of seed studies were prioritized for manual review. Chemical names were removed from seed studies prior to clustering resulting in a generic, chemical-independent method for identifying potentially human health relevant studies. We developed a case study that focused on identifying the array of chemicals that have been studied with respect to in utero exposure to test the recall of this novel literature searching strategy. We then evaluated the general strategy of using generic, chemical-independent training data with two previous IRIS assessments by comparing studies predicted relevant to those used in the assessments (i.e., total relevant). Outcome A keyword search designed to retrieve studies that examined the in utero effects of environmental chemicals identified over 54,000 candidate references. Clustering algorithms were applied using 1456 studies from multiple IRIS assessments with chemical names removed as training data or seeds (i.e., semi-supervised learning). Using a six-algorithm ensemble approach 2602 articles, or approximately 5% of candidate references, were "voted" relevant by four or more clustering algorithms and manual review confirmed nearly 50% of these studies were relevant. Further evaluations on two IRIS assessments, using a nine-algorithm ensemble approach and a set of generic, chemical-independent, externally-derived seed studies correctly identified 77-83% of hazard identification studies published in the assessments and eliminated the need to manually screen more than 75% of search results on average. Limitations The chemical-independent approach used to build the training literature set provides a broad and unbiased picture across a variety of endpoints and environmental exposures but does not systematically identify all available data. Variance between actual and predicted relevant studies will be greater because of the external and non-random origin of seed study selection. This approach depends on access to readily available generic training data that can be used to locate relevant references in an unclassified corpus. Impact A generic approach to identifying human health relevant studies could be an important first step in literature evaluation for risk assessments. This initial scoping approach could facilitate faster literature evaluation by focusing reviewer efforts, as well as potentially minimize reviewer bias in selection of key studies. Using externally-derived training data has applicability particularly for databases with very low search precision where identifying training data may be cost-prohibitive.

Overview of known plastic packaging-associated chemicals and their hazards

Global plastics production has reached 380 million metric tons in 2015, with around 40% used for packaging. Plastic packaging is diverse and made of multiple polymers and numerous additives, along with other components, such as adhesives or coatings. Further, packaging can contain residues from substances used during manufacturing, such as solvents, along with non-intentionally added substances (NIAS), such as impurities, oligomers, or degradation products. To characterize risks from chemicals potentially released during manufacturing, use, disposal, and/or recycling of packaging, comprehensive information on all chemicals involved is needed. Here, we present a database of Chemicals associated with Plastic Packaging (CPPdb), which includes chemicals used during manufacturing and/or present in final packaging articles. The CPPdb lists 906 chemicals likely associated with plastic packaging and 3377 substances that are possibly associated. Of the 906 chemicals likely associated with plastic packaging, 63 rank highest for human health hazards and 68 for environmental hazards according to the harmonized hazard classifications assigned by the European Chemicals Agency within the Classification, Labeling and Packaging (CLP) regulation implementing the United Nations' Globally Harmonized System (GHS). Further, 7 of the 906 substances are classified in the European Union as persistent, bioaccumulative, and toxic (PBT), or very persistent, very bioaccumulative (vPvB), and 15 as endocrine disrupting chemicals (EDC). Thirty-four of the 906 chemicals are also recognized as EDC or potential EDC in the recent EDC report by the United Nations Environment Programme. The identified hazardous chemicals are used in plastics as monomers, intermediates, solvents, surfactants, plasticizers, stabilizers, biocides, flame retardants, accelerators, and colorants, among other functions. Our work was challenged by a lack of transparency and incompleteness of publicly available information on both the use and toxicity of numerous substances. The most hazardous chemicals identified here should be assessed in detail as potential candidates for substitution.

A review of methods used for hazard identification and risk assessment of environmental hazards

BACKGROUND

Approximately one quarter of all deaths globally are attributed to living or working in an unhealthy environment, with household and ambient air pollution, along with exposures to ultraviolet radiation and chemicals amongst the leading causes. At present there are no international standards for assessing the risks of these environmental hazards. The use of heterogeneous methods to identify health risks from environmental hazards may reduce the level of confidence the public has in the conclusions that are made.

OBJECTIVES

To describe and compare the processes and methods used by national and international organisations that conduct hazard identification and/or risk assessment (HI/RA) of environmental hazards and to identify knowledge gaps to inform the development of future methods.

METHODS

We searched the websites of 19 organisations (ten national, five international and four World Health Organization (WHO) units) and extracted data from all relevant, publicly available resources which described the processes and methods used in HI/RA of environmental hazards. We contacted each organisation for any additional information.

RESULTS

Five organisations were excluded from further analysis: three made recommendations but did not conduct HI/RA; one used heterogenous methods across their reviews for HI; and one WHO unit did not have any published guidelines. Of the 14 organisations analysed, five (36%) describe the process for establishing the questions to be answered in the assessments. Only one (7%) organisation uses systematic review methods, although five (36%) state that they use such methods. Ten (71%) assess the scientific quality of the included studies, however only three (21%) use explicit criteria. Only three (21%) organisations assess the quality of the body of evidence using explicit criteria. Four (29%) organisations describe the process for making the final HI conclusions and three (38%) the final RA conclusions. Eight (57%) have a conflict of interest policy and seven (50%) organisations describe a process for managing them. The US Office of Health Assessment and Translation and the World Health Organisation meet the most criteria for describing their processes and methods.

CONCLUSIONS

The processes and methods used by organisations conducting HI/RA of environmental hazards are inconsistent. There is a need for empirically based tools and methods to be adopted for the evaluation and synthesis of evidence, and the formulation of conclusions across all organisations that conduct HI or RA. These tools and methods will lead to increased transparency, comparability and validity of the assessments.

Is it possible to overcome issues of external validity in preclinical animal research? Why most animal models are bound to fail

Background

The pharmaceutical industry is in the midst of a productivity crisis and rates of translation from bench to bedside are dismal. Patients are being let down by the current system of drug discovery; of the several 1000 diseases that affect humans, only a minority have any approved treatments and many of these cause adverse reactions in humans. A predominant reason for the poor rate of translation from bench to bedside is generally held to be the failure of preclinical animal models to predict clinical efficacy and safety. Attempts to explain this failure have focused on problems of internal validity in preclinical animal studies (e.g. poor study design, lack of measures to control bias). However there has been less discussion of another key factor that influences translation, namely the external validity of preclinical animal models.

Review of problems of external validity

External validity is the extent to which research findings derived in one setting, population or species can be reliably applied to other settings, populations and species. This paper argues that the reliable translation of findings from animals to humans will only occur if preclinical animal studies are both internally and externally valid. We review several key aspects that impact external validity in preclinical animal research, including unrepresentative animal samples, the inability of animal models to mimic the complexity of human conditions, the poor applicability of animal models to clinical settings and animal–human species differences. We suggest that while some problems of external validity can be overcome by improving animal models, the problem of species differences can never be overcome and will always undermine external validity and the reliable translation of preclinical findings to humans.

Conclusion

We conclude that preclinical animal models can never be fully valid due to the uncertainties introduced by species differences. We suggest that even if the next several decades were spent improving the internal and external validity of animal models, the clinical relevance of those models would, in the end, only improve to some extent. This is because species differences would continue to make extrapolation from animals to humans unreliable. We suggest that to improve clinical translation and ultimately benefit patients, research should focus instead on human-relevant research methods and technologies.

Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned with the Zebrafish Embryotoxicity Test

Systematic review methodology is a means of addressing specific questions through structured, consistent, and transparent examinations of the relevant scientific evidence. This methodology has been used to advantage in clinical medicine, and is being adapted for use in other disciplines. Although some applications to toxicology have been explored, especially for hazard identification, the present preparatory study is, to our knowledge, the first attempt to adapt it to the assessment of toxicological test methods. As our test case, we chose the zebrafish embryotoxicity test (ZET) for developmental toxicity and its mammalian counterpart, the standard mammalian prenatal development toxicity study, focusing the review on how well the ZET predicts the presence or absence of chemical-induced prenatal developmental toxicity observed in mammalian studies. An interdisciplinary team prepared a systematic review protocol and adjusted it throughout this piloting phase, where needed. The final protocol was registered and will guide the main study (systematic review), which will execute the protocol to comprehensively answer the review question. The goal of this preparatory study was to translate systematic review methodology to the assessment of toxicological test method performance. Consequently, it focused on the methodological issues encountered, whereas the main study will report substantive findings. These relate to numerous systematic review steps, but primarily to searching and selecting the evidence. Applying the lessons learned to these challenges can improve not only our main study, but may also be helpful to others seeking to use systematic review methodology to compare toxicological test methods. We conclude with a series of recommendations that, if adopted, would help improve the quality of the published literature, and make conducting systematic reviews of toxicological studies faster and easier over time.

Facilitating healthcare decisions by assessing the certainty in the evidence from preclinical animal studies

Laboratory animal studies are used in a wide range of human health related research areas, such as basic biomedical research, drug research, experimental surgery and environmental health. The results of these studies can be used to inform decisions regarding clinical research in humans, for example the decision to proceed to clinical trials. If the research question relates to potential harms with no expectation of benefit (e.g., toxicology), studies in experimental animals may provide the only relevant or controlled data and directly inform clinical management decisions. Systematic reviews and meta-analyses are important tools to provide robust and informative evidence summaries of these animal studies. Rating how certain we are about the evidence could provide important information about the translational probability of findings in experimental animal studies to clinical practice and probably improve it. Evidence summaries and certainty in the evidence ratings could also be used (1) to support selection of interventions with best therapeutic potential to be tested in clinical trials, (2) to justify a regulatory decision limiting human exposure (to drug or toxin), or to (3) support decisions on the utility of further animal experiments. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach is the most widely used framework to rate the certainty in the evidence and strength of health care recommendations. Here we present how the GRADE approach could be used to rate the certainty in the evidence of preclinical animal studies in the context of therapeutic interventions. We also discuss the methodological challenges that we identified, and for which further work is needed. Examples are defining the importance of consistency within and across animal species and using GRADE's indirectness domain as a tool to predict translation from animal models to humans.

Using systematic review in occupational safety and health

Evaluation of scientific evidence is critical in developing recommendations to reduce risk. Healthcare was the first scientific field to employ a systematic review approach for synthesizing research findings to support evidence-based decision-making and it is still the largest producer and consumer of systematic reviews. Systematic reviews in the field of occupational safety and health are being conducted, but more widespread use and adoption would strengthen assessments. In 2016, NIOSH asked RAND to develop a framework for applying the traditional systematic review elements to the field of occupational safety and health. This paper describes how essential systematic review elements can be adapted for use in occupational systematic reviews to enhance their scientific quality, objectivity, transparency, reliability, utility, and acceptability.

A primer on systematic reviews in toxicology

Systematic reviews, pioneered in the clinical field, provide a transparent, methodologically rigorous and reproducible means of summarizing the available evidence on a precisely framed research question. Having matured to a well-established approach in many research fields, systematic reviews are receiving increasing attention as a potential tool for answering toxicological questions. In the larger framework of evidence-based toxicology, the advantages and obstacles of, as well as the approaches for, adapting and adopting systematic reviews to toxicology are still being explored. To provide the toxicology community with a starting point for conducting or understanding systematic reviews, we herein summarized available guidance documents from various fields of application. We have elaborated on the systematic review process by breaking it down into ten steps, starting with planning the project, framing the question, and writing and publishing the protocol, and concluding with interpretation and reporting. In addition, we have identified the specific methodological challenges of toxicological questions and have summarized how these can be addressed. Ultimately, this primer is intended to stimulate scientific discussions of the identified issues to fuel the development of toxicology-specific methodology and to encourage the application of systematic review methodology to toxicological issues.

An Integrated Experimental Design for the Assessment of Multiple Toxicological End Points in Rat Bioassays

BACKGROUND

For nearly five decades long-term studies in rodents have been the accepted benchmark for assessing chronic long-term toxic effects, particularly carcinogenicity, of chemicals. The European Food Safety Authority (EFSA) and the World Health Organization (WHO) have pointed out that the current set of internationally utilized test methods capture only some of the potential adverse effects associated with exposures to these agents over the lifetime.

OBJECTIVES

In this paper, we propose the adaption of the carcinogenicity bioassay to integrate additional protocols for comprehensive long-term toxicity assessment that includes developmental exposures and long-term outcomes, capable of generating information on a broad spectrum of different end points.

DISCUSSION

An integrated study design based on a stepwise process is described that includes the priority end points of the Economic Co-operation and Development and the National Toxicology Program guidelines on carcinogenicity and chronic toxicity and developmental and reproductive toxicity. Integrating a comprehensive set of relevant toxicological end points in a single protocol represents an opportunity to optimize animal use in accordance with the 3Rs (replacement, reduction and refinement). This strategy has the potential to provide sufficient data on multiple windows of susceptibility of specific interest for risk assessments and public health decision-making by including prenatal, lactational, neonatal exposures and evaluating outcomes over the lifespan.

CONCLUSION

This integrated study design is efficient in that the same generational cohort of rats used for evaluating long-term outcomes can be monitored in satellite parallel experiments to measure biomarkers and other parameters related to system-specific responses including metabolic alterations and endocrine disturbances. Citation: Manservisi F, Babot Marquillas C, Buscaroli A, Huff J, Lauriola M, Mandrioli D, Manservigi M, Panzacchi S, Silbergeld EK, Belpoggi F. 2017. An integrated experimental design for the assessment of multiple toxicological end points in rat bioassays. Environ Health Perspect 125:289-295; http://dx.doi.org/10.1289/EHP419.

Commemorating Toxicology at the National Institute of Environmental Health Sciences on the Occasion of Its 50th Anniversary

In 1978, the National Toxicology Program (NTP) was established and headquartered at the National Institute of Environmental Health Sciences (NIEHS) in Research Triangle Park, North Carolina. On the occasion of the 50th Anniversary of the NIEHS, this article documents some of the historical and current NTP programs and scientific advances that have been made possible through this long-standing relationship.

The inadequacies of pre-market chemical risk assessment's toxicity studies-the implications

Industry provides essentially all the data for most (pre-market) chemical risk assessments (RA); academics study a chemical once it is marketed. For two randomly-chosen high production chemicals, despite new European Union mandates to evaluate all data, just 13% of the herbicide bentazon and 15% of the flame-retardant hexabromocyclododecane's published toxicity studies were found in their pre-market RA, and a systematic review on bentazon concludes it has greater hazards than indicated in its RA. More important, for both, academia's toxicity studies were designated as lower quality than industries were, despite showing hazards at lower doses. The accuracy of industry's test methods is analyzed and found to be replicable but insensitive, thus inaccurate. The synthetic pharmaceutical industry originated them, and by 1983 the Organization for Economic Cooperation & Development mandated their test guidelines (TG) methods be accepted for any new study for pre-market RA. For existing studies, industry's "Klimisch" criterion is universally used to evaluate quality, but it only states that TG studies produce the best data. However, no TG can answer the realistic exposure effect hypotheses of academics; therefore, crucially in pre-market RA, tens of thousands of published experimental findings (increasingly at low dose) are ignored to determine the safe dose. Few appreciate this, so scientific debate on the most accurate elements of toxicity tests is urgently indicated. Copyright © 2016 John Wiley & Sons, Ltd.

What is the evidence that point sources of anthropogenic effluent increase antibiotic resistance in the environment? Protocol for a systematic review

Herein we describe a protocol for a systematic review of the evidence on whether point sources of anthropogenic effluent are associated with an increase in antibiotic resistance in the adjacent environment. The review question was based on the Population, Exposure, Comparator, Outcome, Study Design (PECOS) framework as follows: Is the prevalence or concentration of antibiotic resistant bacteria or resistance genes (O) in soil, water, air or free-living wildlife (P) higher in close proximity to, or downstream from, known or suspected sources of anthropogenic effluent (E) compared to areas more distant from or upstream from these sources (C)? A comprehensive search strategy was created to capture all relevant, published literature. Criteria for two stages of eligibility screening were developed to exclude publications that were not relevant to the question, and determine if the study used a design that permitted estimation of an association between a source and levels of resistance. A decision matrix was created for assessment of risk of bias to internal validity due to sample selection bias, information bias, and confounding. The goal of this protocol is to provide a method for determining the state of knowledge about the effect of point sources on antibiotic resistance in the environment.

Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: A scoping review

Assessments of methodological and reporting quality are critical to adequately judging the credibility of a study's conclusions and to gauging its potential reproducibility. To aid those seeking to assess the methodological or reporting quality of studies relevant to toxicology, we conducted a scoping review of the available guidance with respect to four types of studies: in vivo and in vitro, (quantitative) structure-activity relationships ([Q]SARs), physico-chemical, and human observational studies. Our aims were to identify the available guidance in this diverse literature, briefly summarize each document, and distill the common elements of these documents for each study type. In general, we found considerable guidance for in vivo and human studies, but only one paper addressed in vitro studies exclusively. The guidance for (Q)SAR studies and physico-chemical studies was scant but authoritative. There was substantial overlap across guidance documents in the proposed criteria for both methodological and reporting quality. Some guidance documents address toxicology research directly, whereas others address preclinical research generally or clinical research and therefore may not be fully applicable to the toxicology context without some translation. Another challenge is the degree to which assessments of methodological quality in toxicology should focus on risk of bias - as in clinical medicine and healthcare - or be broadened to include other quality measures, such as confirming the identity of test substances prior to exposure. Our review is intended primarily for those in toxicology and risk assessment seeking an entry point into the extensive and diverse literature on methodological and reporting quality applicable to their work.

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations

Systematic review (SR) is a rigorous, protocol-driven approach designed to minimise error and bias when summarising the body of research evidence relevant to a specific scientific question. Taking as a comparator the use of SR in synthesising research in healthcare, we argue that SR methods could also pave the way for a "step change" in the transparency, objectivity and communication of chemical risk assessments (CRA) in Europe and elsewhere. We suggest that current controversies around the safety of certain chemicals are partly due to limitations in current CRA procedures which have contributed to ambiguity about the health risks posed by these substances. We present an overview of how SR methods can be applied to the assessment of risks from chemicals, and indicate how challenges in adapting SR methods from healthcare research to the CRA context might be overcome. Regarding the latter, we report the outcomes from a workshop exploring how to increase uptake of SR methods, attended by experts representing a wide range of fields related to chemical toxicology, risk analysis and SR. Priorities which were identified include: the conduct of CRA-focused prototype SRs; the development of a recognised standard of reporting and conduct for SRs in toxicology and CRA; and establishing a network to facilitate research, communication and training in SR methods. We see this paper as a milestone in the creation of a research climate that fosters communication between experts in CRA and SR and facilitates wider uptake of SR methods into CRA.

How credible are the study results? Evaluating and applying internal validity tools to literature-based assessments of environmental health hazards

Environmental health hazard assessments are routinely relied upon for public health decision-making. The evidence base used in these assessments is typically developed from a collection of diverse sources of information of varying quality. It is critical that literature-based evaluations consider the credibility of individual studies used to reach conclusions through consistent, transparent and accepted methods. Systematic review procedures address study credibility by assessing internal validity or "risk of bias" - the assessment of whether the design and conduct of a study compromised the credibility of the link between exposure/intervention and outcome. This paper describes the commonalities and differences in risk-of-bias methods developed or used by five groups that conduct or provide methodological input for performing environmental health hazard assessments: the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group, the Navigation Guide, the National Toxicology Program's (NTP) Office of Health Assessment and Translation (OHAT) and Office of the Report on Carcinogens (ORoC), and the Integrated Risk Information System of the U.S. Environmental Protection Agency (EPA-IRIS). Each of these groups have been developing and applying rigorous assessment methods for integrating across a heterogeneous collection of human and animal studies to inform conclusions on potential environmental health hazards. There is substantial consistency across the groups in the consideration of risk-of-bias issues or "domains" for assessing observational human studies. There is a similar overlap in terms of domains addressed for animal studies; however, the groups differ in the relative emphasis placed on different aspects of risk of bias. Future directions for the continued harmonization and improvement of these methods are also discussed.

Using systematic reviews for hazard and risk assessment of endocrine disrupting chemicals

The possibility that endocrine disrupting chemicals (EDCs) in our environment contribute to hormonally related effects and diseases observed in human and wildlife populations has caused concern among decision makers and researchers alike. EDCs challenge principles traditionally applied in chemical risk assessment and the identification and assessment of these compounds has been a much debated topic during the last decade. State of the science reports and risk assessments of potential EDCs have been criticized for not using systematic and transparent approaches in the evaluation of evidence. In the fields of medicine and health care, systematic review methodologies have been developed and used to enable objectivity and transparency in the evaluation of scientific evidence for decision making. Lately, such approaches have also been promoted for use in the environmental health sciences and risk assessment of chemicals. Systematic review approaches could provide a tool for improving the evaluation of evidence for decision making regarding EDCs, e.g. by enabling systematic and transparent use of academic research data in this process. In this review we discuss the advantages and challenges of applying systematic review methodology in the identification and assessment of EDCs.

Risk assessment's insensitive toxicity testing may cause it to fail

BACKGROUND

Risk assessment of chemicals and other agents must be accurate to protect health. We analyse the determinants of a sensitive chronic toxicity study, risk assessment's most important test. Manufacturers originally generate data on the properties of a molecule, and if government approval is needed to market it, laws globally require toxicity data to be generated using Test Guidelines (TG), i.e. test methods of the Organisation for Economic Cooperation and Development (OECD), or their equivalent. TGs have advantages, but they test close-to-poisonous doses for chronic exposures and have other insensitivities, such as not testing disease latency. This and the fact that academic investigators will not be constrained by such artificial methods, created a de facto total ban of academia's diverse and sensitive toxicity tests from most risk assessment.

OBJECTIVE

To start and sustain a dialogue between regulatory agencies and academic scientists (secondarily, industry and NGOs) whose goals would be to (1) agree on the determinants of accurate toxicity tests and (2) implement them (via the OECD).

DISCUSSION

We analyse the quality of the data produced by these incompatible paradigms: regulatory and academic toxicology; analyse the criteria used to designate data quality in risk assessment; and discuss accurate chronic toxicity test methods.

CONCLUSION

There are abundant modern experimental methods (and rigorous epidemiology), and an existing systematic review system, to at long last allow academia's toxicity studies to be used in most risk assessments.

The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes

BACKGROUND

Synthesizing what is known about the environmental drivers of health is instrumental to taking prevention-oriented action. Methods of research synthesis commonly used in environmental health lag behind systematic review methods developed in the clinical sciences over the past 20 years.

OBJECTIVES

We sought to develop a proof of concept of the "Navigation Guide," a systematic and transparent method of research synthesis in environmental health.

DISCUSSION

The Navigation Guide methodology builds on best practices in research synthesis in evidence-based medicine and environmental health. Key points of departure from current methods of expert-based narrative review prevalent in environmental health include a prespecified protocol, standardized and transparent documentation including expert judgment, a comprehensive search strategy, assessment of "risk of bias," and separation of the science from values and preferences. Key points of departure from evidence-based medicine include assigning a "moderate" quality rating to human observational studies and combining diverse evidence streams.

CONCLUSIONS

The Navigation Guide methodology is a systematic and rigorous approach to research synthesis that has been developed to reduce bias and maximize transparency in the evaluation of environmental health information. Although novel aspects of the method will require further development and validation, our findings demonstrated that improved methods of research synthesis under development at the National Toxicology Program and under consideration by the U.S. Environmental Protection Agency are fully achievable. The institutionalization of robust methods of systematic and transparent review would provide a concrete mechanism for linking science to timely action to prevent harm.

Systematic review and evidence integration for literature-based environmental health science assessments

BACKGROUND

Systematic-review methodologies provide objectivity and transparency to the process of collecting and synthesizing scientific evidence in reaching conclusions on specific research questions. There is increasing interest in applying these procedures to address environmental health questions.

OBJECTIVES

The goal was to develop a systematic-review framework to address environmental health questions by extending approaches developed for clinical medicine to handle the breadth of data relevant to environmental health sciences (e.g., human, animal, and mechanistic studies).

METHODS

The Office of Health Assessment and Translation (OHAT) adapted guidance from authorities on systematic-review and sought advice during development of the OHAT Approach through consultation with technical experts in systematic review and human health assessments, as well as scientific advisory groups and the public. The method was refined by considering expert and public comments and through application to case studies.

RESULTS AND DISCUSSION

Here we present a seven-step framework for systematic review and evidence integration for reaching hazard identification conclusions: 1) problem formulation and protocol development, 2) search for and select studies for inclusion, 3) extract data from studies, 4) assess the quality or risk of bias of individual studies, 5) rate the confidence in the body of evidence, 6) translate the confidence ratings into levels of evidence, and 7) integrate the information from different evidence streams (human, animal, and "other relevant data" including mechanistic or in vitro studies) to develop hazard identification conclusions.

CONCLUSION

The principles of systematic review can be successfully applied to environmental health questions to provide greater objectivity and transparency to the process of developing conclusions.

Risk communication and decision tools for children's health protection

Scientific discovery linking the environment to beneficial and adverse health children's health outcomes is rapidly expanding, leading scientists and health professionals to call for timely action to prevent harm and secure benefits. A robust method to synthesize what is known about the environmental drivers of health is a foundational step to making the science actionable by individuals and decision-makers. To meet this need, a methodology called the Navigation Guide was crafted by a collaboration of 22 clinical and environmental health scientists. The Navigation Guide proceeds from methods of research synthesis used in clinical settings but accounts for differences between environmental and clinical health sciences related to the evidence-base and decision-context. The methodology can be used to develop evidence profiles that provide simple, transparent summaries, such as practice guidelines or other evidence-based recommendations for prevention. Establishing proof-of-concept of the method is underway. Development of the Navigation Guide is extremely timely as it coincides with growing recognition of the need for updated methods in risk assessment. The costs in 2008 to the US healthcare system for treatment of childhood illnesses linked to toxic environmental exposures is conservatively estimated to be over $76 billion, and it is anticipated that US healthcare policy decisions will increasingly rely on systematic reviews of the evidence. The Navigation Guide is poised to provide a methodological bridge to link healthcare decision-making to efforts to reduce toxic environmental exposures. The institutionalization of the Navigation Guide would provide a concrete mechanism for linking science to action to protect children's health.