1
|
Schrecker T. Downing the Master's Tools? New Research Strategies to Address Social Determinants of Health Inequalities. INTERNATIONAL JOURNAL OF SOCIAL DETERMINANTS OF HEALTH AND HEALTH SERVICES 2023:27551938231161932. [PMID: 37032455 DOI: 10.1177/27551938231161932] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
A dramatic increase in the volume of research literature referencing social determinants of health (SDH) since the report of the World Health Organization Commission on the topic in 2008 has not been matched by expansion of policies and interventions to reduce health inequalities by way of SDH. This article argues that familiar hierarchies of evidence that privilege clinical epidemiology as used in evidence-based medicine are inappropriate to address SDH. They misunderstand both the range of relevant evidence and the value-based nature of standards of proof. A richer conceptual armamentarium is available; it includes several applications of the concepts of epidemiological worlds and the lifecourse, which are explained in the article. A more appropriate evidentiary approach to SDH and health inequalities requires "downing the master's tools," to adapt Audre Lorde's phrase, and instead applying a multidisciplinary approach to assessing the evidence that adequately reflects the complexity of the relevant causal pathways. Doing so is made more difficult by the power structures that shape research priorities, yet it is essential.
Collapse
Affiliation(s)
- Ted Schrecker
- Emeritus Professor of Global Health Policy, Faculty of Medical Sciences, Newcastle University, Population Health Sciences Institute, Newcastle upon Tyne, UK
| |
Collapse
|
2
|
Woodruff TJ, Rayasam SDG, Axelrad DA, Koman PD, Chartres N, Bennett DH, Birnbaum LS, Brown P, Carignan CC, Cooper C, Cranor CF, Diamond ML, Franjevic S, Gartner EC, Hattis D, Hauser R, Heiger-Bernays W, Joglekar R, Lam J, Levy JI, MacRoy PM, Maffini MV, Marquez EC, Morello-Frosch R, Nachman KE, Nielsen GH, Oksas C, Abrahamsson DP, Patisaul HB, Patton S, Robinson JF, Rodgers KM, Rossi MS, Rudel RA, Sass JB, Sathyanarayana S, Schettler T, Shaffer RM, Shamasunder B, Shepard PM, Shrader-Frechette K, Solomon GM, Subra WA, Vandenberg LN, Varshavsky JR, White RF, Zarker K, Zeise L. A science-based agenda for health-protective chemical assessments and decisions: overview and consensus statement. Environ Health 2023; 21:132. [PMID: 36635734 PMCID: PMC9835243 DOI: 10.1186/s12940-022-00930-3] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/27/2022] [Indexed: 06/17/2023]
Abstract
The manufacture and production of industrial chemicals continues to increase, with hundreds of thousands of chemicals and chemical mixtures used worldwide, leading to widespread population exposures and resultant health impacts. Low-wealth communities and communities of color often bear disproportionate burdens of exposure and impact; all compounded by regulatory delays to the detriment of public health. Multiple authoritative bodies and scientific consensus groups have called for actions to prevent harmful exposures via improved policy approaches. We worked across multiple disciplines to develop consensus recommendations for health-protective, scientific approaches to reduce harmful chemical exposures, which can be applied to current US policies governing industrial chemicals and environmental pollutants. This consensus identifies five principles and scientific recommendations for improving how agencies like the US Environmental Protection Agency (EPA) approach and conduct hazard and risk assessment and risk management analyses: (1) the financial burden of data generation for any given chemical on (or to be introduced to) the market should be on the chemical producers that benefit from their production and use; (2) lack of data does not equate to lack of hazard, exposure, or risk; (3) populations at greater risk, including those that are more susceptible or more highly exposed, must be better identified and protected to account for their real-world risks; (4) hazard and risk assessments should not assume existence of a "safe" or "no-risk" level of chemical exposure in the diverse general population; and (5) hazard and risk assessments must evaluate and account for financial conflicts of interest in the body of evidence. While many of these recommendations focus specifically on the EPA, they are general principles for environmental health that could be adopted by any agency or entity engaged in exposure, hazard, and risk assessment. We also detail recommendations for four priority areas in companion papers (exposure assessment methods, human variability assessment, methods for quantifying non-cancer health outcomes, and a framework for defining chemical classes). These recommendations constitute key steps for improved evidence-based environmental health decision-making and public health protection.
Collapse
Affiliation(s)
- Tracey J Woodruff
- Program On Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 490 Illinois Street, Floor 10, Box 0132, San Francisco, CA, 94143, USA.
| | - Swati D G Rayasam
- Program On Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 490 Illinois Street, Floor 10, Box 0132, San Francisco, CA, 94143, USA
| | | | - Patricia D Koman
- Department of Environmental Health Sciences, School of Public Health, University of Michigan, Ann Arbor, MI, USA
| | - Nicholas Chartres
- Program On Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 490 Illinois Street, Floor 10, Box 0132, San Francisco, CA, 94143, USA
| | - Deborah H Bennett
- Department of Public Health Sciences, University of California, Davis, Davis, CA, USA
| | - Linda S Birnbaum
- National Institutes of Environmental Health Sciences and National Toxicology Program, Research Triangle Park, NC, USA
- Duke University, Durham, NC, USA
| | - Phil Brown
- Social Science Environmental Health Research Institute, Northeastern University, Boston, MA, USA
| | - Courtney C Carignan
- Department of Food Science and Human Nutrition, Department of Pharmacology and Toxicology, Michigan State University, East Lansing, MI, USA
| | - Courtney Cooper
- Program On Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 490 Illinois Street, Floor 10, Box 0132, San Francisco, CA, 94143, USA
| | - Carl F Cranor
- Department of Philosophy, University of California, Riverside, Riverside, CA, USA
- Environmental Toxicology Graduate Program, College of Natural and Agricultural Sciences, University of California, Riverside, Riverside, CA, USA
| | - Miriam L Diamond
- Department of Earth Sciences, University of Toronto, Toronto, ON, Canada
- School of the Environment, University of Toronto, Toronto, ON, Canada
| | | | | | - Dale Hattis
- The George Perkins Marsh Institute, Clark University, Worcester, MA, USA
| | - Russ Hauser
- Department of Environmental Health, T.H. Chan School of Public Health, Harvard University, Boston, MA, USA
| | - Wendy Heiger-Bernays
- Department of Environmental Health, School of Public Health, Boston University, Boston, MA, USA
| | | | - Juleen Lam
- Department of Public Health, California State University, East Bay, Hayward, CA, USA
| | - Jonathan I Levy
- Department of Environmental Health, School of Public Health, Boston University, Boston, MA, USA
| | | | | | | | - Rachel Morello-Frosch
- School of Public Health, University of California, Berkeley, Berkeley, CA, USA
- Department of Environmental Science, Policy and Management, University of California, Berkeley, Berkeley, CA, USA
| | - Keeve E Nachman
- Department of Environmental Health and Engineering, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
- Johns Hopkins Risk Sciences and Public Policy Institute, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
| | - Greylin H Nielsen
- Department of Environmental Health, School of Public Health, Boston University, Boston, MA, USA
| | - Catherine Oksas
- School of Medicine, University of California, San Francisco, CA, USA
| | - Dimitri Panagopoulos Abrahamsson
- Program On Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 490 Illinois Street, Floor 10, Box 0132, San Francisco, CA, 94143, USA
| | - Heather B Patisaul
- Department of Biological Sciences, Center for Human Health and the Environment, North Carolina State University, Raleigh, NC, USA
| | | | - Joshua F Robinson
- Program On Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, 490 Illinois Street, Floor 10, Box 0132, San Francisco, CA, 94143, USA
- Center for Reproductive Sciences, Department of Obstetrics, Gynecology & Reproductive Sciences, University of California, San Francisco, San Francisco, CA, USA
| | | | | | | | | | - Sheela Sathyanarayana
- Department of Pediatrics, University of Washington, Seattle, WA, USA
- Department of Child Health, Behavior, and Development, Seattle Children's Research Institute, Seattle, WA, USA
| | - Ted Schettler
- Science and Environmental Health Network, Ames, IA, USA
| | - Rachel M Shaffer
- Department of Environmental and Occupational Health Sciences, University of Washington School of Public Health, Seattle, USA
| | - Bhavna Shamasunder
- Department of Urban & Environmental Policy and Public Health, Occidental College, Los Angeles, CA, USA
| | | | - Kristin Shrader-Frechette
- Department of Biological Sciences, University of Notre Dame, Notre Dame, IN, USA
- Department of Philosophy, University of Notre Dame, Notre Dame, IN, USA
| | - Gina M Solomon
- School of Medicine, University of California, San Francisco, CA, USA
- Public Health Institute, Oakland, CA, USA
| | - Wilma A Subra
- Louisiana Environmental Action Network, Baton Rouge, LA, USA
| | - Laura N Vandenberg
- Department of Environmental Health Sciences, School of Public Health & Health Sciences, University of Massachusetts, Amherst, Amherst, MA, USA
| | - Julia R Varshavsky
- Department of Health Sciences, Northeastern University, Boston, MA, USA
- Department of Civil and Environmental Engineering, Northeastern University, Boston, MA, USA
| | - Roberta F White
- Department of Environmental Health, School of Public Health, Boston University, Boston, MA, USA
| | - Ken Zarker
- Washington State Department of Ecology, Olympia, WA, USA
| | - Lauren Zeise
- Office of Environmental Health Hazard Assessment, California Environmental Protection Agency, Oakland, CA, USA
| |
Collapse
|
3
|
Sarpa M, Friedrich K. Exposição a agrotóxicos e desenvolvimento de câncer no contexto da saúde coletiva: o papel da agroecologia como suporte às políticas públicas de prevenção do câncer. SAÚDE EM DEBATE 2022. [DOI: 10.1590/0103-11042022e227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
RESUMO Atualmente, a agricultura brasileira é caracterizada pelo crescente consumo de agrotóxicos e fertilizantes químicos, inserindo-se no modelo de produção baseado nos fundamentos do agronegócio. As novas técnicas de cultivo baseadas no agronegócio resultaram na expansão das monoculturas sobre os ecossistemas naturais, com o consequente desmatamento, desequilíbrio e perda da biodiversidade; e o aumento da contaminação do solo, da água e do ar pelos agrotóxicos. No que tange à saúde humana, a literatura científica tem demonstrado que a contaminação química decorrente do uso de agrotóxicos na agricultura implica adoecimento dos trabalhadores rurais expostos ocupacionalmente aos agrotóxicos, dos moradores da área rural, além de consumidores de alimentos contendo resíduos de agrotóxicos. Entre os efeitos sobre a saúde humana associados à exposição a agrotóxicos, os mais preocupantes são as intoxicações crônicas, caracterizadas por infertilidade, abortos, malformações congênitas, neurotoxicidade, desregulação hormonal, imunotoxicidade, genotoxicidade e câncer. Sendo assim, neste ensaio, apresenta-se uma revisão narrativa com dados presentes na literatura científica nacional e internacional referentes à associação entre a exposição a agrotóxicos e o desenvolvimento de câncer no contexto da saúde coletiva e o papel da alimentação saudável e da agroecologia como suporte às políticas públicas de prevenção do câncer.
Collapse
Affiliation(s)
- Marcia Sarpa
- Instituto Nacional de Câncer José Alencar Gomes da Silva (Inca), Brasil
| | - Karen Friedrich
- Fundação Oswaldo Cruz (Fiocruz), Escola Nacional de Saúde Pública Sergio Arouca (Ensp), Brasil
| |
Collapse
|
4
|
Marí-Bauset S, Peraita-Costa I, Donat-Vargas C, Llopis-González A, Marí-Sanchis A, Llopis-Morales J, Morales Suárez-Varela M. Systematic review of prenatal exposure to endocrine disrupting chemicals and autism spectrum disorder in offspring. AUTISM : THE INTERNATIONAL JOURNAL OF RESEARCH AND PRACTICE 2022; 26:6-32. [PMID: 34412519 DOI: 10.1177/13623613211039950] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
LAY ABSTRACT Autism spectrum disorders comprise a complex group with many subtypes of behaviorally defined neurodevelopmental abnormalities in two core areas: deficits in social communication and fixated, restricted, repetitive, or stereotyped behaviors and interests each with potential unique risk factors and characteristics. The underlying mechanisms and the possible causes of autism spectrum disorder remain elusive and while increased prevalence is undoubtable, it is unclear if it is a reflection of diagnostic improvement or emerging risk factors such as endocrine disrupting chemicals. Epidemiological studies, which are used to study the relation between endocrine disrupting chemicals and autism spectrum disorder, can have inherent methodological challenges that limit the quality and strength of their findings. The objective of this work is to systematically review the treatment of these challenges and assess the quality and strength of the findings in the currently available literature. The overall quality and strength were "moderate" and "limited," respectively. Risk of bias due to the exclusion of potential confounding factors and the lack of accuracy of exposure assessment methods were the most prevalent. The omnipresence of endocrine disrupting chemicals and the biological plausibility of the association between prenatal exposure and later development of autism spectrum disorder highlight the need to carry out well-designed epidemiological studies that overcome the methodological challenges observed in the currently available literature in order to be able to inform public policy to prevent exposure to these potentially harmful chemicals and aid in the establishment of predictor variables to facilitate early diagnosis of autism spectrum disorder and improve long-term outcomes.
Collapse
Affiliation(s)
- Salvador Marí-Bauset
- Unit of Public Health and Environmental Care, Department of Preventive Medicine and Public Health, Food Sciences, Toxicology and Forensic Medicine, University of Valencia, Spain
| | - Isabel Peraita-Costa
- Unit of Public Health and Environmental Care, Department of Preventive Medicine and Public Health, Food Sciences, Toxicology and Forensic Medicine, University of Valencia, Spain
- CIBER in Epidemiology and Public Health (CIBERESP), Institute of Health Carlos III, Spain
| | | | - Agustín Llopis-González
- Unit of Public Health and Environmental Care, Department of Preventive Medicine and Public Health, Food Sciences, Toxicology and Forensic Medicine, University of Valencia, Spain
- CIBER in Epidemiology and Public Health (CIBERESP), Institute of Health Carlos III, Spain
| | | | - Juan Llopis-Morales
- Unit of Public Health and Environmental Care, Department of Preventive Medicine and Public Health, Food Sciences, Toxicology and Forensic Medicine, University of Valencia, Spain
| | - María Morales Suárez-Varela
- Unit of Public Health and Environmental Care, Department of Preventive Medicine and Public Health, Food Sciences, Toxicology and Forensic Medicine, University of Valencia, Spain
- CIBER in Epidemiology and Public Health (CIBERESP), Institute of Health Carlos III, Spain
| |
Collapse
|
5
|
Soskolne CL, Kramer S, Ramos-Bonilla JP, Mandrioli D, Sass J, Gochfeld M, Cranor CF, Advani S, Bero LA. Toolkit for detecting misused epidemiological methods. Environ Health 2021; 20:90. [PMID: 34412643 PMCID: PMC8375462 DOI: 10.1186/s12940-021-00771-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Accepted: 07/09/2021] [Indexed: 05/08/2023]
Abstract
BACKGROUND Critical knowledge of what we know about health and disease, risk factors, causation, prevention, and treatment, derives from epidemiology. Unfortunately, its methods and language can be misused and improperly applied. A repertoire of methods, techniques, arguments, and tactics are used by some people to manipulate science, usually in the service of powerful interests, and particularly those with a financial stake related to toxic agents. Such interests work to foment uncertainty, cast doubt, and mislead decision makers by seeding confusion about cause-and-effect relating to population health. We have compiled a toolkit of the methods used by those whose interests are not aligned with the public health sciences. Professional epidemiologists, as well as those who rely on their work, will thereby be more readily equipped to detect bias and flaws resulting from financial conflict-of-interest, improper study design, data collection, analysis, or interpretation, bringing greater clarity-not only to the advancement of knowledge, but, more immediately, to policy debates. METHODS The summary of techniques used to manipulate epidemiological findings, compiled as part of the 2020 Position Statement of the International Network for Epidemiology in Policy (INEP) entitled Conflict-of-Interest and Disclosure in Epidemiology, has been expanded and further elucidated in this commentary. RESULTS Some level of uncertainty is inherent in science. However, corrupted and incomplete literature contributes to confuse, foment further uncertainty, and cast doubt about the evidence under consideration. Confusion delays scientific advancement and leads to the inability of policymakers to make changes that, if enacted, would-supported by the body of valid evidence-protect, maintain, and improve public health. An accessible toolkit is provided that brings attention to the misuse of the methods of epidemiology. Its usefulness is as a compendium of what those trained in epidemiology, as well as those reviewing epidemiological studies, should identify methodologically when assessing the transparency and validity of any epidemiological inquiry, evaluation, or argument. The problems resulting from financial conflicting interests and the misuse of scientific methods, in conjunction with the strategies that can be used to safeguard public health against them, apply not only to epidemiologists, but also to other public health professionals. CONCLUSIONS This novel toolkit is for use in protecting the public. It is provided to assist public health professionals as gatekeepers of their respective specialty and subspecialty disciplines whose mission includes protecting, maintaining, and improving the public's health. It is intended to serve our roles as educators, reviewers, and researchers.
Collapse
Affiliation(s)
- Colin L Soskolne
- School of Public Health, University of Alberta, Edmonton, AB, Canada.
| | - Shira Kramer
- Epidemiology International, Hunt Valley, MD, USA
| | | | - Daniele Mandrioli
- Cesare Maltoni Cancer Research Centre, Ramazzini Institute, Bologna, Italy
| | - Jennifer Sass
- Natural Resources Defense Council, Washington, DC, USA
- George Washington University, Washington, DC, USA
| | - Michael Gochfeld
- Environmental and Occupational Health Sciences Institute, Rutgers Biomedical and Health Sciences, Newark, NJ, USA
| | - Carl F Cranor
- Departments of Philosophy and Environmental Toxicology, University of California, Riverside, CA, USA
| | - Shailesh Advani
- Terasaki Institute of Biomedical Innovation, Los Angeles, CA, USA
- Georgetown University School of Medicine, Washington, DC, USA
| | - Lisa A Bero
- Center for Bioethics and Humanities, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|
6
|
Johnson PI, Sutton P, Koustas E, Vesterinen HM, Woodruff TJ. Response to correspondence by Heather Lynch, Julie Goodman and Nancy Beck Re: "Application of the Navigation Guide systematic review methodology to the evidence for developmental and reproductive toxicity of triclosan". ENVIRONMENT INTERNATIONAL 2017; 102:76-78. [PMID: 28236502 PMCID: PMC6685544 DOI: 10.1016/j.envint.2017.02.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/04/2017] [Revised: 02/13/2017] [Accepted: 02/14/2017] [Indexed: 06/06/2023]
Affiliation(s)
- Paula I Johnson
- University of California San Francisco, Program on Reproductive Health and the Environment, CA, USA.
| | - Patrice Sutton
- University of California San Francisco, Program on Reproductive Health and the Environment, CA, USA
| | - Erica Koustas
- ORISE Post-doctoral Fellowship, U.S. Environmental Protection Agency, Office of Policy, National Center for Environmental Economics, Washington, D.C., USA
| | - Hanna M Vesterinen
- University of California San Francisco, Program on Reproductive Health and the Environment, CA, USA
| | - Tracey J Woodruff
- University of California San Francisco, Program on Reproductive Health and the Environment, CA, USA
| |
Collapse
|
7
|
Metayer C, Dahl G, Wiemels J, Miller M. Childhood Leukemia: A Preventable Disease. Pediatrics 2016; 138:S45-S55. [PMID: 27940977 PMCID: PMC5080868 DOI: 10.1542/peds.2015-4268h] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/16/2016] [Indexed: 01/26/2023] Open
Abstract
In contrast to most pediatric cancers, there is a growing body of literature, nationally and internationally, that has implicated the role of several environmental indoor and outdoor hazards in the etiology of childhood leukemia. For example, exposures to solvents, traffic, pesticides, and tobacco smoke have consistently demonstrated positive associations with the risk of developing childhood leukemia. Intake of vitamins and folate supplementation during the preconception period or pregnancy has been demonstrated to have a protective effect. Despite the strength of these findings, the dissemination of this knowledge to clinicians has been limited. Some children may be more vulnerable than others as documented by the high and increasing incidence of childhood leukemia in Hispanics. To protect children's health, it is prudent to establish programs to alter exposure to those factors with well-established associations with leukemia risk rather than to suspend judgment until no uncertainty remains. This is particularly true because other serious health outcomes (both negative and positive) have been associated with the same exposures. We draw from historical examples to put in perspective the arguments of association versus causation, as well as to discuss benefits versus risks of immediate and long-term preventive actions.
Collapse
Affiliation(s)
- Catherine Metayer
- School of Public Health, University of California, Berkeley, Berkeley, California;
| | - Gary Dahl
- School of Medicine, Stanford University, Stanford, California; and
| | - Joe Wiemels
- Department of Epidemiology and Biostatistics, and
| | - Mark Miller
- Western States Pediatric Environmental Health Specialty Unit, University of California, San Francisco, San Francisco, California
| |
Collapse
|
8
|
Grandjean P. Toxicology research for precautionary decision-making and the role of Human & Experimental Toxicology. Hum Exp Toxicol 2015; 34:1231-7. [PMID: 26614810 DOI: 10.1177/0960327115601762] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
A key aim of toxicology is the prevention of adverse effects due to toxic hazards. Therefore, the dissemination of toxicology research findings must confront two important challenges: one being the lack of information on the vast majority of potentially toxic industrial chemicals and the other being the strict criteria for scientific proof usually required for decision-making in regard to prevention. The present study ascertains the coverage of environmental chemicals in four volumes of Human & Experimental Toxicology and the presentation and interpretation of research findings in published articles. Links in SciFinder showed that the 530 articles published in four selected volumes between 1984 and 2014 primarily dealt with metals (126 links) and other toxicants that have received substantial attention in the past. Thirteen compounds identified by US authorities in 2006 as high-priority substances, for which toxicology documentation is badly needed, were not covered in the journal issues at all. When reviewing published articles, reliance on p values was standard, and non-significant findings were often called 'negative.' This tradition may contribute to the perceived need to extend existing research on toxic hazards that have already been well characterized. Several sources of bias towards the null hypothesis can affect toxicology research, but are generally not considered, thus adding to the current inclination to avoid false positive findings. In this regard, toxicology is particularly prone to bias because of the known paucity of false positives and, in particular, the existence of a vast number of toxic hazards which by default are considered innocuous due to lack of documentation. The Precautionary Principle could inspire decision-making on the basis of incomplete documentation and should stimulate a change in toxicology traditions and in toxicology research publication.
Collapse
Affiliation(s)
- P Grandjean
- Department of Public Health, University of Southern Denmark, Odense C, Denmark Department of Environmental Health, Harvard School of Public Health, Boston, MA, USA
| |
Collapse
|
9
|
Futran Fuhrman V, Tal A, Arnon S. Why endocrine disrupting chemicals (EDCs) challenge traditional risk assessment and how to respond. JOURNAL OF HAZARDOUS MATERIALS 2015; 286:589-611. [PMID: 25646754 DOI: 10.1016/j.jhazmat.2014.12.012] [Citation(s) in RCA: 81] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2014] [Revised: 12/02/2014] [Accepted: 12/08/2014] [Indexed: 05/11/2023]
Abstract
Endocrine disrupting compounds (EDCs) are a diverse group of "chemicals of emerging concern" which have attracted much interest from the research community since the 1990s. Today there is still no definitive risk assessment tool for EDCs. While some decision making organizations have attempted to design methodology guidelines to evaluate the potential risk from this broadly defined group of constituents, risk assessors still face many uncertainties and unknowns. Until a risk assessment paradigm is designed specifically for EDCs and is vetted by the field, traditional risk assessment tools may be used with caution to evaluate EDCs. In doing so, each issue of contention should be addressed with transparency in order to leverage available information and technology without sacrificing integrity or accuracy. The challenges that EDCs pose to traditional risk assessment are described in this article to assist in this process.
Collapse
Affiliation(s)
- Vivian Futran Fuhrman
- Institute for Dryland, Environmental and Desert Research, The Jacob Blaustein Institutes for Desert Research, Ben-Gurion University of the Negev, Sede Boqer 84990, Israel.
| | - Alon Tal
- Institute for Dryland, Environmental and Desert Research, The Jacob Blaustein Institutes for Desert Research, Ben-Gurion University of the Negev, Sede Boqer 84990, Israel.
| | - Shai Arnon
- Zuckerberg Institute for Water Research, The Jacob Blaustein Institutes for Desert Research, Ben-Gurion University of the Negev, Sede Boqer 84990, Israel.
| |
Collapse
|
10
|
Lam J, Koustas E, Sutton P, Johnson PI, Atchley DS, Sen S, Robinson KA, Axelrad DA, Woodruff TJ. The Navigation Guide - evidence-based medicine meets environmental health: integration of animal and human evidence for PFOA effects on fetal growth. ENVIRONMENTAL HEALTH PERSPECTIVES 2014; 122:1040-51. [PMID: 24968389 PMCID: PMC4181930 DOI: 10.1289/ehp.1307923] [Citation(s) in RCA: 158] [Impact Index Per Article: 14.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2013] [Accepted: 06/20/2014] [Indexed: 05/03/2023]
Abstract
BACKGROUND The Navigation Guide is a novel systematic review method to synthesize scientific evidence and reach strength of evidence conclusions for environmental health decision making. OBJECTIVE Our aim was to integrate scientific findings from human and nonhuman studies to determine the overall strength of evidence for the question "Does developmental exposure to perfluorooctanoic acid (PFOA) affect fetal growth in humans?" METHODS We developed and applied prespecified criteria to systematically and transparently a) rate the quality of the scientific evidence as "high," "moderate," or "low"; b) rate the strength of the human and nonhuman evidence separately as "sufficient," "limited," "moderate," or "evidence of lack of toxicity"; and c) integrate the strength of the human and nonhuman evidence ratings into a strength of the evidence conclusion. RESULTS We identified 18 epidemiology studies and 21 animal toxicology studies relevant to our study question. We rated both the human and nonhuman mammalian evidence as "moderate" quality and "sufficient" strength. Integration of these evidence ratings produced a final strength of evidence rating in which review authors concluded that PFOA is "known to be toxic" to human reproduction and development based on sufficient evidence of decreased fetal growth in both human and nonhuman mammalian species. CONCLUSION We concluded that developmental exposure to PFOA adversely affects human health based on sufficient evidence of decreased fetal growth in both human and nonhuman mammalian species. The results of this case study demonstrate the application of a systematic and transparent methodology, via the Navigation Guide, for reaching strength of evidence conclusions in environmental health.
Collapse
Affiliation(s)
- Juleen Lam
- Oak Ridge Institute for Science and Education (ORISE) Postdoctoral Fellow, National Center for Environmental Economics, Office of Policy, U.S. Environmental Protection Agency, Washington, DC, USA
| | | | | | | | | | | | | | | | | |
Collapse
|
11
|
Woodruff TJ, Sutton P. The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. ENVIRONMENTAL HEALTH PERSPECTIVES 2014; 122:1007-14. [PMID: 24968373 PMCID: PMC4181919 DOI: 10.1289/ehp.1307175] [Citation(s) in RCA: 292] [Impact Index Per Article: 26.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2013] [Accepted: 02/24/2014] [Indexed: 05/17/2023]
Abstract
BACKGROUND Synthesizing what is known about the environmental drivers of health is instrumental to taking prevention-oriented action. Methods of research synthesis commonly used in environmental health lag behind systematic review methods developed in the clinical sciences over the past 20 years. OBJECTIVES We sought to develop a proof of concept of the "Navigation Guide," a systematic and transparent method of research synthesis in environmental health. DISCUSSION The Navigation Guide methodology builds on best practices in research synthesis in evidence-based medicine and environmental health. Key points of departure from current methods of expert-based narrative review prevalent in environmental health include a prespecified protocol, standardized and transparent documentation including expert judgment, a comprehensive search strategy, assessment of "risk of bias," and separation of the science from values and preferences. Key points of departure from evidence-based medicine include assigning a "moderate" quality rating to human observational studies and combining diverse evidence streams. CONCLUSIONS The Navigation Guide methodology is a systematic and rigorous approach to research synthesis that has been developed to reduce bias and maximize transparency in the evaluation of environmental health information. Although novel aspects of the method will require further development and validation, our findings demonstrated that improved methods of research synthesis under development at the National Toxicology Program and under consideration by the U.S. Environmental Protection Agency are fully achievable. The institutionalization of robust methods of systematic and transparent review would provide a concrete mechanism for linking science to timely action to prevent harm.
Collapse
Affiliation(s)
- Tracey J Woodruff
- Program on Reproductive Health and the Environment, University of California, San Francisco, Oakland, California, USA
| | | |
Collapse
|
12
|
Grandjean P, Ozonoff D. Transparency and translation of science in a modern world. Environ Health 2013; 12:70. [PMID: 23981514 PMCID: PMC3765922 DOI: 10.1186/1476-069x-12-70] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2013] [Accepted: 08/07/2013] [Indexed: 05/20/2023]
Abstract
The co-Editors-in-Chief of Environmental Health respond to an unusual initiative taken by editors of 14 toxicology journals to influence pending decisions by the European Commission to establish a framework for regulating chemicals that pose a hazard to normal function of the endocrine system. This initiative is also the subject of this Commentary in this journal by authors who recently reviewed the subject and who point out inaccuracies in the toxicology editors' critique. The dispute is about potential public policy development, rather than on science translation and research opportunities and priorities. The toxicology journal editors recommend that chemicals be examined in depth one by one, ignoring modern achievements in biomedical research that would allow new understanding of the effects of classes of toxic substances in complex biological systems. Concerns about policy positions framed as scientific ones are especially important in a time with shrinking public support for biomedical research affects priorities. In such a setting, conflict of interest declarations are important, especially in research publications that address issues of public concern and where financial and other interests may play a role. Science relies on trust, and reasonable disclosure of financial or other potential conflicts is therefore essential. This need has been emphasized by recent discoveries of hidden financial conflicts in publications in toxicology journals, thus misleading readers and the public about the safety of particular industrial products. The transparency provided by Environmental Health includes open access and open peer review, with reader access to reviews, including the identity of reviewers and their statements on possible conflicts of interest. However, the editors of the 14 toxicology journals did not provide any information on potential conflicts of interest, an oversight that needs to be corrected.
Collapse
Affiliation(s)
- Philippe Grandjean
- Department of Environmental Medicine, University of Southern Denmark, 5000 Odense, Denmark
- Department of Environmental Health, Harvard School of Public Health, Boston, MA USA
| | - David Ozonoff
- Department of Environmental Health, Boston University School of Public Health, Boston, MA USA
| |
Collapse
|
13
|
Money CD, Tomenson JA, Penman MG, Boogaard PJ, Jeffrey Lewis R. A systematic approach for evaluating and scoring human data. Regul Toxicol Pharmacol 2013; 66:241-7. [DOI: 10.1016/j.yrtph.2013.03.011] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2012] [Revised: 03/20/2013] [Accepted: 03/22/2013] [Indexed: 10/27/2022]
|
14
|
Vogt R, Bennett D, Cassady D, Frost J, Ritz B, Hertz-Picciotto I. Cancer and non-cancer health effects from food contaminant exposures for children and adults in California: a risk assessment. Environ Health 2012; 11:83. [PMID: 23140444 PMCID: PMC3551655 DOI: 10.1186/1476-069x-11-83] [Citation(s) in RCA: 45] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2011] [Accepted: 10/18/2012] [Indexed: 05/19/2023]
Abstract
BACKGROUND In the absence of current cumulative dietary exposure assessments, this analysis was conducted to estimate exposure to multiple dietary contaminants for children, who are more vulnerable to toxic exposure than adults. METHODS We estimated exposure to multiple food contaminants based on dietary data from preschool-age children (2-4 years, n=207), school-age children (5-7 years, n=157), parents of young children (n=446), and older adults (n=149). We compared exposure estimates for eleven toxic compounds (acrylamide, arsenic, lead, mercury, chlorpyrifos, permethrin, endosulfan, dieldrin, chlordane, DDE, and dioxin) based on self-reported food frequency data by age group. To determine if cancer and non-cancer benchmark levels were exceeded, chemical levels in food were derived from publicly available databases including the Total Diet Study. RESULTS Cancer benchmark levels were exceeded by all children (100%) for arsenic, dieldrin, DDE, and dioxins. Non-cancer benchmarks were exceeded by >95% of preschool-age children for acrylamide and by 10% of preschool-age children for mercury. Preschool-age children had significantly higher estimated intakes of 6 of 11 compounds compared to school-age children (p<0.0001 to p=0.02). Based on self-reported dietary data, the greatest exposure to pesticides from foods included in this analysis were tomatoes, peaches, apples, peppers, grapes, lettuce, broccoli, strawberries, spinach, dairy, pears, green beans, and celery. CONCLUSIONS Dietary strategies to reduce exposure to toxic compounds for which cancer and non-cancer benchmarks are exceeded by children vary by compound. These strategies include consuming organically produced dairy and selected fruits and vegetables to reduce pesticide intake, consuming less animal foods (meat, dairy, and fish) to reduce intake of persistent organic pollutants and metals, and consuming lower quantities of chips, cereal, crackers, and other processed carbohydrate foods to reduce acrylamide intake.
Collapse
Affiliation(s)
- Rainbow Vogt
- UC Davis, Department of Public Health Sciences, 1 Shields Avenue, Davis, CA, 95616, USA
| | - Deborah Bennett
- UC Davis, Department of Public Health Sciences, 1 Shields Avenue, Davis, CA, 95616, USA
| | - Diana Cassady
- UC Davis, Department of Public Health Sciences, 1 Shields Avenue, Davis, CA, 95616, USA
| | - Joshua Frost
- UC Davis, Department of Public Health Sciences, 1 Shields Avenue, Davis, CA, 95616, USA
| | - Beate Ritz
- UC Los Angeles, Department of Epidemiology, 100 Stein Plaza Driveway, Los Angeles, CA, 90095, USA
| | - Irva Hertz-Picciotto
- UC Davis, Department of Public Health Sciences, 1 Shields Avenue, Davis, CA, 95616, USA
| |
Collapse
|
15
|
Abstract
The magic word 'complexity' has been buzzing around in science, policy and society for quite some time now. There seems to be a common feel for a 'new way' of doing things, for overcoming the limits of tradition. From the combined perspective of critical complexity thinking and environment and health practice we want to contribute to the development of alternative routines that may help overcome the limitations of traditional environment and health science. On the one hand traditional environment and health science is too self-confident with respect to potential scientific insight in environment and health problems: complexity condemns us to limited and ambiguous knowledge and the need for simplification. A more modest attitude would be more realistic from that point of view. On the other hand from a problem solving perspective more boldness is required. Waiting for Godot (perfect undisputed knowledge) will not help us with respect to the challenges posed to society by environment and health problems. A sense of urgency is legitimate: the paralysis by traditional analysis should be resolved. Nevertheless this sense of urgency should not withhold us from investing in the problem solving quality of our endeavour; quality takes time, fastness from a quality perspective often leads us to a standstill. We propose the concept of critical complexification of environment and health practice that will enable the integration of relevant actors and factors in a pragmatic manner. We will illustrate this with practical examples and especially draw attention to the practical complexities involved, confronting us not only with fundamental questions, but also with fundamental challenges.
Collapse
Affiliation(s)
- Hans Keune
- Research Institute for Nature and Forest (INBO), Brussels, Belgium.
| |
Collapse
|
16
|
When Work is Related to Disease, What Establishes Evidence for a Causal Relation? Saf Health Work 2012; 3:110-6. [PMID: 22993715 PMCID: PMC3440459 DOI: 10.5491/shaw.2012.3.2.110] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2011] [Revised: 10/17/2011] [Accepted: 03/28/2012] [Indexed: 11/08/2022] Open
Abstract
Establishing a causal relationship between factors at work and disease is difficult for occupational physicians and researchers. This paper seeks to provide arguments for the judgement of evidence of causality in observational studies that relate work factors to disease. I derived criteria for the judgement of evidence of causality from the following sources: the criteria list of Hill, the approach by Rothman, the methods used by International Agency for Research on Cancer (IARC), and methods used by epidemiologists. The criteria are applied to two cases of putative occupational diseases; breast cancer caused by shift work and aerotoxic syndrome. Only three of the Hill criteria can be applied to an actual study. Rothman stresses the importance of confounding and alternative explanations than the putative cause. IARC closely follows Hill, but they also incorporate other than epidemiological evidence. Applied to shift work and breast cancer, these results have found moderate evidence for a causal relationship, but applied to the aerotoxic syndrome, there is an absence of evidence of causality. There are no ready to use algorithms for judgement of evidence of causality. Criteria from different sources lead to similar results and can make a conclusion of causality more or less likely.
Collapse
|
17
|
Abstract
Occupational Safety and Health (OSH) professionals should use scientific evidence to support their decisions in policy and practice. Although examples from practice show that progress has been made in evidence-based decision making, there is a challenge to improve and extend the facilities that support knowledge translation in practice. A knowledge infrastructure that supports OSH practice should include scientific research, systematic reviews, practice guidelines, and other tools for professionals such as well accessible virtual libraries and databases providing knowledge, quality tools, and good learning materials. A good infrastructure connects facilities with each other and with practice. Training and education is needed for OSH professionals in the use of evidence to improve effectiveness and efficiency. New initiatives show that occupational health can profit from intensified international collaboration to establish a good functioning knowledge infrastructure.
Collapse
|
18
|
Maslanyj M, Lightfoot T, Schüz J, Sienkiewicz Z, McKinlay A. A precautionary public health protection strategy for the possible risk of childhood leukaemia from exposure to power frequency magnetic fields. BMC Public Health 2010; 10:673. [PMID: 21054823 PMCID: PMC3091578 DOI: 10.1186/1471-2458-10-673] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2010] [Accepted: 11/05/2010] [Indexed: 11/12/2022] Open
Abstract
Background Epidemiological evidence showing a consistent association between the risk of childhood leukaemia and exposure to power frequency magnetic fields has been accumulating. This debate considers the additional precautionary intervention needed to manage this risk, when it exceeds the protection afforded by the exposure guidelines as recommended by the International Commission on Non-Ionizing Radiation Protection. Methods The Bradford-Hill Criteria are guidelines for evaluating the scientific evidence that low frequency magnetic fields cause childhood leukaemia. The criteria are used for assessing the strength of scientific evidence and here have been applied to considering the strength of evidence that exposures to extremely low frequency magnetic fields may increase the risk of childhood leukaemia. The applicability of precaution is considered using the risk management framework outlined in a European Commission (EC) communication on the Precautionary Principle. That communication advises that measures should be proportionate, non-discriminatory, consistent with similar measures already taken, based on an examination of the benefits and costs of action and inaction, and subject to review in the light of new scientific findings. Results The main evidence for a risk is an epidemiological association observed in several studies and meta-analyses; however, the number of highly exposed children is small and the association could be due to a combination of selection bias, confounding and chance. Corroborating experimental evidence is limited insofar as there is no clear indication of harm at the field levels implicated; however, the aetiology of childhood leukaemia is poorly understood. Taking a precautionary approach suggests that low-cost intervention to reduce exposure is appropriate. This assumes that if the risk is real, its impact is likely to be small. It also recognises the consequential cost of any major intervention. The recommendation is controversial in that other interpretations of the data are possible, and low-cost intervention may not fully alleviate the risk. Conclusions The debate shows how the EC risk management framework can be used to apply the Precautionary Principle to small and uncertain public health risks. However, despite the need for evidence-based policy making, many of the decisions remain value driven and therefore subjective.
Collapse
Affiliation(s)
- Myron Maslanyj
- Health Protection Agency, Chilton, Didcot, Oxfordshire OX110RQ, UK.
| | | | | | | | | |
Collapse
|
19
|
Keune H, Morrens B, Croes K, Colles A, Koppen G, Springael J, Loots I, Van Campenhout K, Chovanova H, Schoeters G, Nelen V, Baeyens W, Van Larebeke N. Opening the research agenda for selection of hot spots for human biomonitoring research in Belgium: a participatory research project. Environ Health 2010; 9:33. [PMID: 20604935 PMCID: PMC2911417 DOI: 10.1186/1476-069x-9-33] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2010] [Accepted: 07/06/2010] [Indexed: 05/12/2023]
Abstract
BACKGROUND In order to select priority hotspots for environment and health research in Flanders (Belgium), an open procedure was organized. Environment and health hotspots are strong polluting point sources with possible health effects for residents living in the vicinity of the hot spot. The selection procedure was part of the work of the Flemish Centre of Expertise for Environment and Health, which investigates the relation between environmental pollution and human health. The project is funded and steered by the Flemish government. METHODS The involvement of other actors than merely experts is inspired by the 'analytical-deliberative' approach of the National Research Council in the United States and the extended peer community approach. These approaches stress the importance of involving different expert- and social perspectives in order to increase the knowledge base on complex issues. In the procedure used in the project a combination of expert and stakeholder input was essential. The final decision was supported by a multi-criteria analysis of expert assessment and stakeholder advice. RESULTS The endeavour was challenging from the start because of the complicated ambition of including a diversity of actors, potential hotspots, concerns and assessment criteria, but nevertheless the procedure proved its value in both structuring and informing the decision-making process. Moreover the process gained the support of most actors participating in the process, even though the final selection could not satisfy all preferences. CONCLUSIONS Opening the research agenda exemplifies the value of inter- and transdisciplinary cooperation as well as the need for a well-structured and negotiated procedure that combines relevant factors and actors with pragmatism. The value of such a process also needs to prove itself in practice after the procedure has been completed: the tension between an ambition of openness on the one hand and a more closed attitude amongst experts on the other will continue to play a role even after closure.
Collapse
Affiliation(s)
- Hans Keune
- Faculty of Political and Social Sciences, University of Antwerp Belgium
| | - Bert Morrens
- Faculty of Political and Social Sciences, University of Antwerp Belgium
| | - Kim Croes
- Vrije Universiteit Brussel, Brussels, Belgium
| | - Ann Colles
- Environmental Toxicology, Flemish Institute of Technological Research, Mol-Belgium
| | - Gudrun Koppen
- Environmental Toxicology, Flemish Institute of Technological Research, Mol-Belgium
| | - Johan Springael
- Faculty of Applied Economics, University of Antwerp, Belgium
| | - Ilse Loots
- Faculty of Political and Social Sciences, University of Antwerp Belgium
| | - Karen Van Campenhout
- Environment and Health, Flemish Government, Environment, Nature and Energy Department, Brussels, Belgium
| | - Hana Chovanova
- Flemish Agency for Care and Health, Division of Public Health Surveillance, Brussels, Belgium
| | - Greet Schoeters
- Environmental Toxicology, Flemish Institute of Technological Research, Mol-Belgium
| | - Vera Nelen
- Provincial Institute of Hygiene, Antwerp, Belgium
| | | | - Nik Van Larebeke
- Study Centre for Carcinogenesis and Primary Prevention of Cancer, Department of Radiotherapy, Nuclear Medicine and Experimental Oncology, University of Ghent, Belgium
| |
Collapse
|
20
|
Brenner GM, Stevens CW. Drug Development and Safety. Pharmacology 2010. [DOI: 10.1016/b978-1-4160-6627-9.00004-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
21
|
Gee D. Late Lessons from Early Warnings: Towards realism and precaution with EMF? PATHOPHYSIOLOGY 2009; 16:217-31. [DOI: 10.1016/j.pathophys.2009.01.004] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2008] [Accepted: 01/30/2009] [Indexed: 11/28/2022] Open
|
22
|
Abstract
This article describes uranium and depleted uranium (DU), their similar isotopic compositions, how DU arises, its use in munitions and armour-proofing, and its pathways for human exposures. Particular attention is paid to the evidence of DU's health effects from cell and animal experiments and from epidemiology studies. It is concluded that a precautionary approach should be adopted to DU and that there should be a moratorium on its use by military forces. International efforts to this end are described.
Collapse
Affiliation(s)
- Ian Fairlie
- Independent Consultant on Radioactivity in the Environment, London, UK.
| |
Collapse
|
23
|
Pullin AS, Knight TM. Data credibility: A perspective from systematic reviews in environmental management. ACTA ACUST UNITED AC 2009. [DOI: 10.1002/ev.296] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|