Introducing Open Science in Teaching Health Economic Modelling

Open Science is gaining ground in all research fields, including health economics and outcomes research (HEOR). However, teaching Open Science is still in its infancy. This paper describes the design, implementation and evaluation of a teaching activity focusing on introducing Open Science during a Master's course during which participants have to develop a discrete event simulation. The teaching activity was organised as a series of lectures introducing different aspects of the Open Science philosophy and practices, such as good software coding practices, version control systems and reproducible research. The participants' increase in Open Science knowledge was elicited through a survey before and after the teaching innovation. After the teaching innovation, participants' knowledge of Open Science increased and they reported an improvement in Open Science-related skills, such as using a script-based statistical software, identifying and re-using open data, and collaborative script development. During the evaluation at the end of the course, the course participants mentioned that the Open Science-related content was interesting but would fit better within a course in which broader research-related content is taught. Based on this feedback, we will most likely narrow the scope of the Open-Science-related content in this course to Open Source Modelling which may better fit the scope of the course. This paper contains links to the teaching activities we developed and other resources which may be used to design teaching activities on Open Science. Herewith, we hope to inspire other teachers in including Open Science into their teaching.

Social science - STEM collaborations in agriculture, food and beyond: an STSFAN manifesto

Interdisciplinary research needs innovation. As an action-oriented intervention, this Manifesto begins from the authors' experiences as social scientists working within interdisciplinary science and technology collaborations in agriculture and food. We draw from these experiences to: 1) explain what social scientists contribute to interdisciplinary agri-food tech collaborations; (2) describe barriers to substantive and meaningful collaboration; and (3) propose ways to overcome these barriers. We encourage funding bodies to develop mechanisms that ensure funded projects respect the integrity of social science expertise and incorporate its insights. We also call for the integration of social scientific questions and methods in interdisciplinary projects from the outset, and for a genuine curiosity on the part of STEM and social science researchers alike about the knowledge and skills each of us has to offer. We contend that cultivating such integration and curiosity within interdisciplinary collaborations will make them more enriching for all researchers involved, and more likely to generate socially beneficial outcomes.

Cascading effects of climate change on recreational marine flats fishes and fisheries

Tropical and subtropical coastal flats are shallow regions of the marine environment at the intersection of land and sea. These regions provide myriad ecological goods and services, including recreational fisheries focused on flats-inhabiting fishes such as bonefish, tarpon, and permit. The cascading effects of climate change have the potential to negatively impact coastal flats around the globe and to reduce their ecological and economic value. In this paper, we consider how the combined effects of climate change, including extremes in temperature and precipitation regimes, sea level rise, and changes in nutrient dynamics, are causing rapid and potentially permanent changes to the structure and function of tropical and subtropical flats ecosystems. We then apply the available science on recreationally targeted fishes to reveal how these changes can cascade through layers of biological organization-from individuals, to populations, to communities-and ultimately impact the coastal systems that depend on them. We identify critical gaps in knowledge related to the extent and severity of these effects, and how such gaps influence the effectiveness of conservation, management, policy, and grassroots stewardship efforts.

Ten simple rules for improving communication among scientists

Communication is a fundamental part of scientific development and methodology. With the advancement of the internet and social networks, communication has become rapid and sometimes overwhelming, especially in science. It is important to provide scientists with useful, effective, and dynamic tools to establish and build a fluid communication framework that allows for scientific advancement. Therefore, in this article, we present advice and recommendations that can help promote and improve science communication while respecting an adequate balance in the degree of commitment toward collaborative work. We have developed 10 rules shown in increasing order of commitment that are grouped into 3 key categories: (1) speak (based on active participation); (2) join (based on joining scientific groups); and (3) assess (based on the analysis and retrospective consideration of the weaknesses and strengths). We include examples and resources that provide actionable strategies for involvement and engagement with science communication, from basic steps to more advanced, introspective, and long-term commitments. Overall, we aim to help spread science from within and encourage and engage scientists to become involved in science communication effectively and dynamically.

Teaching the process of science through COVID-19 pandemic themes

Several predominant themes have emerged during the COVID-19 global pandemic that intersect with the nature and process of science. This paper identifies three such themes and briefly explores how they can be used as case studies and narrative cornerstones in teaching and learning. The themes include: (1) the understanding that science is cumulative and ever-changing, meaning that new findings may cause us to reconsider previous understandings; (2) the importance of citation tracking in the process of science; and (3) the need for accessible and purposeful science communication.

Why citizen review might beat peer review at identifying pursuitworthy scientific research

In response to increasing fatigue with the failings of the peer review system, granting agencies are beginning to consider lotteries as an alternative. I argue that citizen review, in which non-scientists determine funding allocations, has advantages over both. This is particularly true when it comes to identifying which research is most pursuitworthy.

Developing achievable alternate futures for key challenges during the UN Decade of Ocean Science for Sustainable Development

The oceans face a range of complex challenges for which the impacts on society are highly uncertain but mostly negative. Tackling these challenges is testing society's capacity to mobilise transformative action, engendering a sense of powerlessness. Envisaging positive but realistic visions of the future, and considering how current knowledge, resources, and technology could be used to achieve these futures, may lead to greater action to achieve sustainable transformations. Future Seas (www.FutureSeas2030.org) brought together researchers across career stages, Indigenous Peoples and environmental managers to develop scenarios for 12 challenges facing the oceans, leveraging interdisciplinary knowledge to improve society's capacity to purposefully shape the direction of marine social-ecological systems over the UN Decade of Ocean Science for Sustainable Development (2021-2030). We describe and reflect on Future Seas, providing guidance for co-developing scenarios in interdisciplinary teams tasked with exploring ocean futures. We detail the narrative development for two futures: our current trajectory based on published evidence, and a more sustainable future, consistent with the UN's Sustainable Development Goals, which is technically achievable using existing and emerging knowledge. Presentation of Business-as-usual and More Sustainable futures-together-allows communication of both trajectories, whilst also highlighting achievable, sustainable versions of the future. The advantages of the interdisciplinary approach taken include: (1) integrating different perspectives on solutions, (2) capacity to explore interactions between Life Under Water (Goal 14) and other SDGs, and (3) cross-disciplinary learning. This approach allowed participants to conceptualise shared visions of the future and co-design transformative pathways to achieving those futures.

SUPPLEMENTARY INFORMATION SI

The online version contains supplementary material available at (10.1007/s11160-020-09629-5) contains supplementary material, which is available to authorized users.

Developing achievable alternate futures for key challenges during the UN Decade of Ocean Science for Sustainable Development

The oceans face a range of complex challenges for which the impacts on society are highly uncertain but mostly negative. Tackling these challenges is testing society's capacity to mobilise transformative action, engendering a sense of powerlessness. Envisaging positive but realistic visions of the future, and considering how current knowledge, resources, and technology could be used to achieve these futures, may lead to greater action to achieve sustainable transformations. Future Seas (www.FutureSeas2030.org) brought together researchers across career stages, Indigenous Peoples and environmental managers to develop scenarios for 12 challenges facing the oceans, leveraging interdisciplinary knowledge to improve society's capacity to purposefully shape the direction of marine social-ecological systems over the UN Decade of Ocean Science for Sustainable Development (2021-2030). We describe and reflect on Future Seas, providing guidance for co-developing scenarios in interdisciplinary teams tasked with exploring ocean futures. We detail the narrative development for two futures: our current trajectory based on published evidence, and a more sustainable future, consistent with the UN's Sustainable Development Goals, which is technically achievable using existing and emerging knowledge. Presentation of Business-as-usual and More Sustainable futures-together-allows communication of both trajectories, whilst also highlighting achievable, sustainable versions of the future. The advantages of the interdisciplinary approach taken include: (1) integrating different perspectives on solutions, (2) capacity to explore interactions between Life Under Water (Goal 14) and other SDGs, and (3) cross-disciplinary learning. This approach allowed participants to conceptualise shared visions of the future and co-design transformative pathways to achieving those futures.

SUPPLEMENTARY INFORMATION SI

The online version contains supplementary material available at (10.1007/s11160-020-09629-5) contains supplementary material, which is available to authorized users.

Going rogue: what scientists can learn about Twitter communication from "alt" government accounts

The inauguration of President Trump in the United States led to the active restriction of science communication from federal agencies, resulting in the creation of many unofficial "alt" Twitter accounts to maintain communication. Alt accounts had many followers (e.g., 15 accounts had > 100,000) and received a large amount of media attention, making them ideal for better understanding how differences in messaging can affect public engagement with science on microblogging platforms. We analyzed tweets produced by alt and corresponding official agency accounts to compare the two groups and determine if specific features of a tweet made them more likely to be retweeted or liked to help the average scientist potentially reach a broader audience on Twitter. We found adding links, images, hashtags, and mentions, as well as expressing angry and annoying sentiments all increased retweets and likes. Evidence-based terms such as "peer-review" had high retweet rates but linking directly to peer-reviewed publications decreased attention compared to popular science websites. Word choice and attention did not reflect official or alt account types, indicating topic is more important than source. The number of tweets generated and attention received by alt accounts has decreased since their creation, demonstrating the importance of timeliness in science communication on social media. Together our results show potential pathways for scientists to increase efficacy in Twitter communications.

Filming the Historical Geography: Story from the Realm of Maps in Regensburg

Research on a specific topic requires the individualized cartographic methods of work that may be defined as the Realm of Maps. The double dimensionality in the Realm of Maps is understood here as a physical place—a studio workroom—and as a research method. In this study, we focused on the way of presenting a research method designed to study the topic of historico-geographical space in the form of a short film story. The second purpose is to indicate the legitimacy of combining two dimensions of working with maps, the real one and the virtual one, to be able to collect cartographic and descriptive sources in one scientific center. Our research on the Story from the Realm of Maps in Regensburg: ‘People Movement in Southeast Europe’ included a concept adopted by cartographical, historical, and geographical sources; the construction of a studio workroom; a script draft; individual sequences of the story in different types of media; editing the video, along with publishing it on an online video-sharing platform. We used as many different types of geomedia as possible, which, on the one hand, boosts the attractiveness of the film and, on the other, may hamper the proper perception of the main film plot. Finally, we recommend principles of map design for the film, with analog maps and maps created specifically for the short film, published using online video-sharing platforms.

Ethical issues and public communication in the development of cell-based treatments for COVID-19: Lessons from the pandemic

The significant morbidity and mortality of coronavirus disease 19 (COVID-19) prompted a global race to develop new therapies. These include interventions using cell- or cell-derived products, several of which are being tested in well-designed, properly controlled clinical trials. Yet, the search for cell-based COVID-19 treatments has also been fraught with hyperbolic claims; flouting of crucial regulatory, scientific, and ethical norms; and distorted communication of research findings. In this paper, we critically examine ethical issues and public communication challenges related to the development of cell-based therapeutics for COVID-19. Drawing on the lessons learned from this ongoing process, we argue against the rushed development of cell-based interventions. We conclude by outlining ways to improve the ethical conduct of cell-based clinical investigations and public communication of therapeutic claims.

Integrative approaches to dispersing science: A case study of March Mammal Madness

OBJECTIVES

Public engagement is increasingly viewed as an important pillar of scientific scholarship. For early career and established scholars, navigating the mosaic landscape of public education and science communication, noted for rapid "ecological" succession, can be daunting. Moreover, academics are characterized by diverse skills, motivations, values, positionalities, and temperaments that may differentially incline individuals to particular public translation activities.

METHODS

Here we briefly contextualize engagement activities within a scholarly portfolio, describe the use of one public education program-March Mammal Madness (MMM)- to highlight approaches to science communication, and explore essential elements and practical considerations for creating and sustaining outreach pursuits in tandem with other scholarly activities.

RESULTS

MMM, an annual simulated tournament of living and fossil animal taxa, has reached hundreds of thousands of learners since 2013. This program has provided a platform to communicate research findings from biology and anthropology and showcase numerous scholars in these fields. MMM has leveraged tournament devices to intentionally address topics of climate change, capitalist environmental degradation, academic sexism, and racist settler-colonialism. The tournament, however, has also perpetuated implicit biases that need disrupting.

CONCLUSIONS

By embracing reflexive, self-interrogative, and growth attitudes, the tournament organizers iteratively refine and improve this public science education program to better align our activities with our values and goals. Our experiences with MMM suggest that dispersing science is most sustainable when we combine ancestral adaptations for cooperation, community, and storytelling with good-natured competition in the context of shared experiences and shared values.

Online Noise as Illustrated by Pitfalls and Biogeography Associated With Common Names for Puma concolor

Noise is the non-target search results that people encounter when searching for a particular topic of interest; it is also the cloud of distracting data that can obscure or deflect conservation communication. Online noise associated with large carnivores is particularly dense because their defining characteristics make them salient. Mountain lions (Puma concolor) exemplify noise associated with multiple vernaculars for a species in the crosshairs of conservation conundrums. We compared internet search results, Google Trends reflecting topic interest, use in science publications and sentiment in print and online media for P. concolor's most frequent vernacular names, “mountain lion,” “cougar,” “puma” and “Florida panther.” Puma and panther exhibited greater noise and salience than cougar or mountain lion, but, results for mountain lion, followed by cougar, yielded the highest biological relevance. Online sentiment negatively correlated with biological relevance, with positive sentiment highest for the noisiest vernaculars, puma and panther. As conservation practitioners, we must recognize that public outreach is part of our scientific agenda and be conscious of crafting communication that reaches and resonates with our intended audiences.

Public understanding of climate change-related sea-level rise

Sea-level rise resulting from climate change is impacting coasts around the planet. There is strong scientific consensus about the amount of sea-level rise to 2050 (0.24–0.32 m) and a range of projections to 2100, which vary depending on the approach used and the mitigation measures taken to reduce carbon emissions. Despite this strong scientific consensus regarding the reality of climate change-related sea-level rise, and the associated need to engage publics in adaptation and mitigation efforts, there is a lack of empirical evidence regarding people’s understanding of the issue. Here we investigate public understanding of the amount, rate and causes of sea-level rise. Data from a representative sample of New Zealand adults showed a suprising tendency for the public to overestimate the scientifically plausible amount of sea-level rise by 2100 and to identify melting sea ice as its primary causal mechanism. These findings will be valuable for scientists communicating about sea-level rise, communicators seeking to engage publics on the issue of sea-level rise, and media reporting on sea-level rise.

March Mammal Madness and the power of narrative in science outreach

March Mammal Madness is a science outreach project that, over the course of several weeks in March, reaches hundreds of thousands of people in the United States every year. We combine four approaches to science outreach - gamification, social media platforms, community event(s), and creative products - to run a simulated tournament in which 64 animals compete to become the tournament champion. While the encounters between the animals are hypothetical, the outcomes rely on empirical evidence from the scientific literature. Players select their favored combatants beforehand, and during the tournament scientists translate the academic literature into gripping "play-by-play" narration on social media. To date ~1100 scholarly works, covering almost 400 taxa, have been transformed into science stories. March Mammal Madness is most typically used by high-school educators teaching life sciences, and we estimate that our materials reached ~1% of high-school students in the United States in 2019. Here we document the intentional design, public engagement, and magnitude of reach of the project. We further explain how human psychological and cognitive adaptations for shared experiences, social learning, narrative, and imagery contribute to the widespread use of March Mammal Madness.

"You Gotta Choose Your Words Carefully": Findings from Interviews with Environmental Health Scientists about Their Research Translation Perceptions and Training Needs

Scientists are trained to communicate research in a technical manner but often lack the skills to communicate scientific findings to the general public. Effective communication and research translation are increasingly important competencies for researchers and have broader benefits to scientists and society. The aim of the study was to assess the perceptions, experiences, and training needs relative to science communication and research translation of project investigators associated with the Center for Oceans and Human Health and Climate Change Interactions at the University of South Carolina. In-depth, semi-structured interviews were conducted with the Center's investigators. Interview transcripts were coded and analyzed for emergent themes related to science communication and research translation. Investigator descriptions of their research varied in length, and researchers mentioned multiple target audiences. Most investigators preferred in-person and written communication channels and felt "comfortable" communicating uncertain findings to the public despite no formal science communication training. Investigators suggested training focused on plain language development for target communities, assessment of audience needs, and formatting research findings for various groups. Working with multiple target audiences that have preferred communication channels necessitates a comprehensive approach to science communication training to enhance two-way communication between scientists and stakeholders.

A Novel Framework to Protect Animal Data in a World of Ecosurveillance

Surveillance of animal movements using electronic tags (i.e., biotelemetry) has emerged as an essential tool for both basic and applied ecological research and monitoring. Advances in animal tracking are occurring simultaneously with changes to technology, in an evolving global scientific culture that increasingly promotes data sharing and transparency. However, there is a risk that misuse of biotelemetry data could increase the vulnerability of animals to human disturbance or exploitation. For the most part, telemetry data security is not a danger to animals or their ecosystems, but for some high-risk cases, as with species’ with high economic value or at-risk populations, available knowledge of their movements may promote active disturbance or worse, potential poaching. We suggest that when designing animal tracking studies it is incumbent on scientists to consider the vulnerability of their study animals to risks arising from the implementation of the proposed program, and to take preventative measures.

Sentiment analysis as a measure of conservation culture in scientific literature

Culturomics is emerging as an important field within science, as a way to measure attitudes and beliefs and their dynamics across time and space via quantitative analysis of digitized data from literature, news, film, social media, and more. Sentiment analysis is a culturomics tool that, within the last decade, has provided a means to quantify the polarity of attitudes expressed within various media. Conservation science is a crisis discipline; therefore, accurate and effective communication are paramount. We investigated how conservation scientists communicate their findings through scientific journal articles. We analyzed 15,001 abstracts from articles published from 1998 to 2017 in 6 conservation-focused journals selected based on indexing in scientific databases. Articles were categorized by year, focal taxa, and the conservation status of the focal species. We calculated mean sentiment score for each abstract (mean adjusted z score) based on 4 lexicons (Jockers-Rinker, National Research Council, Bing, and AFINN). We found a significant positive annual trend in the sentiment scores of articles. We also observed a significant trend toward increasing negativity along the spectrum of conservation status categories (i.e., from least concern to extinct). There were some clear differences in the sentiments with which research on different taxa was reported, however. For example, abstracts mentioning lobe finned fishes tended to have high sentiment scores, which could be related to the rediscovery of the coelacanth driving a positive narrative. Contrastingly, abstracts mentioning elasmobranchs had low scores, possibly reflecting the negative sentiment score associated with the word shark. Sentiment analysis has applications in science, especially as it pertains to conservation psychology, and we suggest a new science-based lexicon be developed specifically for the field of conservation.

Fundamentals of graphic design—essential tools for effective visual science communication

Guidance on improving the visual aspects of science communication range from “recipe”-style instructions to hyper-focused aspects of data visualization. Currently lacking in the peer-reviewed literature is a primer in graphic design tailored to a high-level overview of basic design principles and associated jargon related to layout, imagery, typeface, and colour. We illustrate why these aspects are important to effective communication. Further, we provide considerations on when to solicit professional assistance and what to expect when working with graphic designers. Having the fundamental principles of good design in your toolbox facilitates the production of effective visual communication related to your research and fruitful scientist–designer collaborations.

Envisioning the scientific paper of the future

Consider for a moment the rate of advancement in the scientific understanding of DNA. It is formidable; from Fredrich Miescher’s nuclein extraction in the 1860s to Rosalind Franklin’s double helix X-ray in the 1950s to revolutionary next-generation sequencing in the late 2000s. Now consider the scientific paper, the medium used to describe and publish these advances. How is the scientific paper advancing to meet the needs of those who generate and use scientific information? We review four essential qualities for the scientific paper of the future: ( i) a robust source of trustworthy information that remains peer reviewed and is ( ii) communicated to diverse users in diverse ways, ( iii) open access, and ( iv) has a measurable impact beyond Impact Factor. Since its inception, scientific literature has proliferated. We discuss the continuation and expansion of practices already in place including: freely accessible data and analytical code, living research and reviews, changes to peer review to improve representation of under-represented groups, plain language summaries, preprint servers, evidence-informed decision-making, and altmetrics.

Translating Marine Animal Tracking Data into Conservation Policy and Management

There have been efforts around the globe to track individuals of many marine species and assess their movements and distribution, with the putative goal of supporting their conservation and management. Determining whether, and how, tracking data have been successfully applied to address real-world conservation issues is, however, difficult. Here, we compile a broad range of case studies from diverse marine taxa to show how tracking data have helped inform conservation policy and management, including reductions in fisheries bycatch and vessel strikes, and the design and administration of marine protected areas and important habitats. Using these examples, we highlight pathways through which the past and future investment in collecting animal tracking data might be better used to achieve tangible conservation benefits.

Towards linking environmental law and science

Gaps between environmental science and environmental law may undermine sound environmental decision-making. We link perspectives and insights from science and law to highlight opportunities and challenges at the environmental science–law interface. The objectives of this paper are to assist scientists who wish to conduct and communicate science that informs environmental statutes, regulations, and associated operational policies (OPs), and to ensure the environmental lawyers (and others) working to ensure that these statutes, regulations, and OPs are appropriately informed by scientific evidence. We provide a conceptual model of how different kinds of science-based activities can feed into legislative and policy cycles, ranging from actionable science that can inform decision-making windows to retrospective analyses that can inform future regulations. We identify a series of major gaps and barriers that challenge the successful linking of environmental science and law. These include (1) the different time frames for science and law, (2) the different standards of proof for scientific and legal (un)certainty, (3) the need for effective scientific communication, (4) the multijurisdictional (federal, provincial, and Indigenous) nature of environmental law, and (5) the different ethical obligations of law and science. Addressing these challenges calls for bidirectional learning among scientists and lawyers and more intentional collaborations at the law–science interface.

Tweet success? Scientific communication correlates with increased citations in Ecology and Conservation

Science communication is seen as critical for the disciplines of ecology and conservation, where research products are often used to shape policy and decision making. Scientists are increasing their online media communication, via social media and news. Such media engagement has been thought to influence or predict traditional metrics of scholarship, such as citation rates. Here, we measure the association between citation rates and the Altmetric Attention Score—an indicator of the amount and reach of the attention an article has received—along with other forms of bibliometric performance (year published, journal impact factor, and article type). We found that Attention Score was positively correlated with citation rates. However, in recent years, we detected increasing media exposure did not relate to the equivalent citations as in earlier years; signalling a diminishing return on investment. Citations correlated with journal impact factors up to ∼13, but then plateaued, demonstrating that maximizing citations does not require publishing in the highest-impact journals. We conclude that ecology and conservation researchers can increase exposure of their research through social media engagement and, simultaneously, enhance their performance under traditional measures of scholarly activity.

Learning Science Communication Skills Using Improvisation, Video Recordings, and Practice, Practice, Practice

Doctoral students in science disciplines spend countless hours learning how to conduct cutting-edge research but very little time learning to communicate the nature and significance of their science to people outside their field. To narrow this disparity, we created an unusual course titled Communicating Science for doctoral science trainees at Rutgers University. Our goal was to help students develop an advanced ability to communicate their research clearly and accurately and to emphasize its value and significance to diverse audiences. Course design included classroom instruction supplemented with improvisation, video recordings, and ample opportunity for students to practice and receive immediate, constructive feedback in a supportive environment. A multidisciplinary faculty with expertise in science, education, communication, and theater arts taught this course. PhD students came from diverse scientific disciplines, ranging from biology and chemistry to civil engineering. Students also completed a capstone project in which they worked with a professional in the academic or private sector to explore a possible career aspiration. Assessment was in the form of feedback on students' oral and poster presentations, and written abstracts about their research. Student evaluations and comments about course format and content were mostly positive and also provided input for ways to improve the course. We discovered that the diversity of scientific backgrounds among our students enhanced their ability to learn how to communicate their science to others outside their disciplines. We are leveraging the success of our initial course offering to reach other student and faculty groups at Rutgers.

Science communication in the field of fundamental biomedical research (editorial)

The aim of this special issue on science communication is to inspire and help scientists who are taking part or want to take part in science communication and engage with the wider public, clinicians, other scientists or policy makers. For this, some articles provide concise and accessible advice to individual scientists, science networks, or learned societies on how to communicate effectively; others share rationales, objectives and aims, experiences, implementation strategies and resources derived from existing long-term science communication initiatives. Although this issue is primarily addressing scientists working in the field of biomedical research, much of it similarly applies to scientists from other disciplines. Furthermore, we hope that this issue will also be used as a helpful resource by academic science communicators and social scientists, as a collection that highlights some of the major communication challenges that the biomedical sciences face, and which provides interesting case studies of initiatives that use a breadth of strategies to address these challenges. In this editorial, we first discuss why we should communicate our science and contemplate some of the different approaches, aspirations and definitions of science communication. We then address the specific challenges that researchers in the biomedical sciences are faced with when engaging with wider audiences. Finally, we explain the rationales and contents of the different articles in this issue and the various science communication initiatives and strategies discussed in each of them, whilst also providing some information on the wide range of further science communication activities in the biomedical sciences that could not all be covered here.