Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers

Quiz-style online training tool helps to learn birdsong identification and support citizen science

Citizen science is an important approach to monitoring for biodiversity conservation because it allows for data acquisition or analysis on a scale that is not possible for researchers alone. In citizen science projects, the use of online training is increasing to improve such skills. However, the effectiveness of quiz-style online training, assumed to be efficient to enhance participants' skills, has not been evaluated adequately on species identification for citizen science biodiversity monitoring projects. Memory mechanisms in adaptive learning were hypothesized to guide the development of quiz-based online training tools for learning birdsong identification and for improving interest in birds and natural environments. To examine the hypothesis, we developed a quiz-style online training tool called TORI-TORE. We experimentally applied TORI-TORE in Fukushima, Japan, and examined its effectiveness for bird identification training using test scores and questionnaires to determine participants' attitudes in a randomized control trial. We obtained the following key results: (1) TORI-TORE had positive effects on test scores and trainees' attitudes toward birds. (2) Adaptive training, in which questions focused preferentially on unmastered bird species based on the answer history of individual trainees inspired by adaptive learning, unexpectedly led to lower scores and satisfaction in TORI-TORE. (3) Focusing on species that are relatively easy to remember, short lag times between training and testing, and long question intervals positively affected scores. While there is room for improvement, we expect TORI-TORE to contribute to online capacity building and to increase interest in natural environments.

Comparing avian species richness estimates from structured and semi-structured citizen science data

Citizen science, including structured and semi-structured forms, has become a powerful tool to collect biodiversity data. However, semi-structured citizen science data have been criticized for higher variability in quality, including less information to adjust for imperfect detection and uneven duration that bias the estimates of species richness. Species richness estimators may quantify bias in estimates. Here, we test the effectiveness of Chao1 estimator in eBird (semi-structured) by comparing it to averaged species richness in Breeding Bird Survey Taiwan, BBS (structured) and quantifying bias. We then fit a power function to compare bias while controlling for differences in count duration. The Chao1 estimator increased the species richness estimates of eBird data from 56 to 69% of the average observed BBS and from 47 to 59% of the average estimated BBS. Effects of incomplete short duration samples and variability in detectability skills of observers can lead to biased estimates. Using the Chao1 estimator improved estimates of species richness from semi-structured and structured data, but the strong effect of singleton species on bias, especially in short duration counts, should be evaluated in advance to reduce the uncertainty of estimation processes.

Citizen science projects in freshwater monitoring. From individual design to clusters?

Citizen science projects spring up in freshwater monitoring, with an increasing number of projects in river, lake, and groundwater monitoring around the globe. Citizen science scholars assume that these projects have different designs, including diverse characteristics of citizens, institutions, and forms of interactions, potentially affecting the outcomes of these projects. Given the strong focus on case studies or reviews in the field, there is, however, little comparative evidence of the different types of citizen science projects in freshwater monitoring. Based on a global survey, this study provides a systematic comparative analysis of the design of 85 citizen science projects in the field of freshwater monitoring. Descriptive statistics reveal how projects differ along 45 literature-based design variables raised in the survey. Factor analysis based on 31 of these variables yields ten key design factors, and cluster analysis, based on these design factors, allows to identify seven distinguished clusters of citizen science projects. While these clusters are rather heterogeneous, they reveal differences between groups of projects regarding institutional motivation, citizen characteristics, and interaction forms. These results significantly add to a systematic comparison of citizen science projects in freshwater monitoring and enable a more effective involvement of citizens in environmental management.

Monitoring biological water quality by volunteers complements professional assessments

Progressively more community initiatives have been undertaken over last decades to monitor water quality. Biological data collected by volunteers has been used for biodiversity and water quality studies. Despite the many citizen science projects collecting and using macroinvertebrates, the number of scientific peer-reviewed publications that use this data, remains limited. In 2018, a citizen science project on biological water quality assessment was launched in the Netherlands. In this project, volunteers collect macroinvertebrates from a nearby waterbody, identify and count the number of specimens, and register the catch through a web portal to instantaneously receive a water quality score based on their data. Water quality monitoring in the Netherlands is traditionally the field of professionals working at water authorities. Here, we compare the data from the citizen science project with the data gathered by professionals. We evaluate information regarding type and distribution of sampled waterbodies and sampling period, and compare general patterns in both datasets with respect to collected animals and calculated water quality scores. The results show that volunteers and professionals seldomly sample the same waterbody, that there is some overlap in sampling period, and that volunteers more frequently sampled urban waters and smaller waterbodies. The citizen science project is thus yielding data about understudied waters and this spatial and temporal complementarity is useful. The character and thoroughness of the assessments by volunteers and professionals are likely to differentiate. Volunteers collected significantly lower numbers of animals per sample and fewer animals from soft sediments like worms and more mobile individuals from the open water column such as boatsmen and beetles. Due to the lack of simultaneous observations at various locations by volunteers and professionals, a direct comparison of water quality scores is impossible. However, the obtained patterns from both datasets show that the water quality scores between volunteers and professionals are dissimilar for the different water types. To bridge these differences, new tools and processes need to be further developed to increase the value of monitoring biological water quality by volunteers for professionals.

Deep learning increases the availability of organism photographs taken by citizens in citizen science programs

Citizen science programs using organism photographs have become popular, but there are two problems related to photographs. One problem is the low quality of photographs. It is laborious to identify species in photographs taken outdoors because they are out of focus, partially invisible, or under different lighting conditions. The other is difficulty for non-experts to identify species. Organisms usually have interspecific similarity and intraspecific variation, which hinder species identification by non-experts. Deep learning solves these problems and increases the availability of organism photographs. We trained a deep convolutional neural network, Xception, to identify bee species using various quality of bee photographs that were taken by citizens. These bees belonged to two honey bee species and 10 bumble bee species with interspecific similarity and intraspecific variation. We investigated the accuracy of species identification by biologists and deep learning. The accuracy of species identification by Xception (83.4%) was much higher than that of biologists (53.7%). When we grouped bee photographs by different colors resulting from intraspecific variation in addition to species, the accuracy of species identification by Xception increased to 84.7%. The collaboration with deep learning and experts will increase the reliability of species identification and their use for scientific researches.

Alientoma, a Dynamic Database for Alien Insects in Greece and Its Use by Citizen Scientists in Mapping Alien Species

Invasive alien species have been increasingly acknowledged as a major threat to native biodiversity and ecosystem services, while their adverse impacts expand to human health, society and the economy on a global scale. Insects represent one of the most numerous alien organismic groups, accounting for about one fifth of their total number. In Greece, a large number of alien insects have been identified, currently reaching 469 species. In recent decades, the contribution of citizen science towards detecting and mapping the distribution of alien insects has been steeply increasing. Addressing the need for up-to-date information on alien species as well as encouraging public participation in scientific research, the Alientoma website-derived from "alien" and the Greek word "entoma", meaning insects, is presented. The website aims towards providing updated information on alien species of insects to the public as well as the scientific community, raising awareness about biological invasions and addressing their distribution and impacts inter alia. By maintaining a dynamic online database alongside a strong social media presence since its launch, Alientoma has attracted individuals mainly from Greece and Cyprus, interacting with the website through a total of 1512 sessions. Alientoma intends to establish a constantly increasing network of citizen scientists and to supplement early detection, monitoring and management efforts to mitigate the adverse impacts of alien insects in Greece.

The Number and Distribution of Introduced and Naturalized Parrots

Parrots have been transported and traded by humans for at least the last 2000 years and this trade continues unabated today. This transport of species has involved the majority of recognized parrot species (300+ of 382 species). Inevitably, some alien species either escape captivity or are released and may establish breeding populations in the novel area. With respect to parrots, established but alien populations are becoming common in many parts of the world. In this review, we attempt to estimate the total number of parrot species introduced into the wild in non-native areas and assess how many of these have self-sustaining breeding populations. Based the public databases GAVIA, eBird, and iNaturalist, 166 species of Psittaciformes have been introduced (seen in the wild) into 120 countries or territories outside of the native range. Of these, 60 species are naturalized, and an additional 11 species are breeding in at least one country outside of their native range (86 countries or territories total). The Rose-ringed Parakeet (Psittacula krameri) and Monk Parakeet (Myiopsitta monachus) are the most widely distributed and successful of the introduced parrots, being naturalized in 47 and 26 countries or territories, respectively. Far and away, the United States and its territories support the greatest number of naturalized parrots, with 28 different species found in either the continental US, or Hawaii or Puerto Rico. Naturalized species as well as urbanized native species of parrots are likely to continue increasing in numbers and geographical range, and detailed studies are needed to both confirm species richness in each area as well mitigate potential ecological impacts and conflicts with humans.

Data Reliability in a Citizen Science Protocol for Monitoring Stingless Bees Flight Activity

Simple Summary

This work aims to validate a citizen science protocol for monitoring the flight activity of stingless bees. The count of flight activity (entrance, exit, and entrance carrying pollen) filmed in 30 s videos was compared among three different groups: “original” citizen scientists (group that filmed and performed the count in their own videos), “replicator” citizen scientists (group of citizen scientists who performed flight activity counts on videos shot by other citizen scientists), and experts (researchers who work with bees and who performed the counts on videos shot by citizen scientists). The analysis was divided into two levels: perception (detection of activity in videos) and counting. The results of this analysis revealed that citizen scientists and experts have similar perception and count of bee entrance and exit activity, as no statistical differences were found in these two items. However, replicator citizen scientists noticed more bees carrying pollen than original citizen scientists and experts. Despite this, considering only the videos in which the groups agreed on the presence of pollen, the count was similar for both. These results enabled the validation of the protocol and indicated high quality of data produced by individuals who participate in scientific practices following a citizen science approach.

Abstract

Although the quality of citizen science (CS) data is often a concern, evidence for high-quality CS data increases in the scientific literature. This study aimed to assess the data reliability of a structured CS protocol for monitoring stingless bees’ flight activity. We tested (1) data accuracy for replication among volunteers and for expert validation and (2) precision, comparing dispersion between citizen scientists and expert data. Two distinct activity dimensions were considered: (a) perception of flight activity and (b) flight activity counts (entrances, exits, and pollen load). No significant differences were found among groups regarding entrances and exits. However, replicator citizen scientists presented a higher chance of perceiving pollen than original data collectors and experts, likely a false positive. For those videos in which there was an agreement about pollen presence, the effective pollen counts were similar (with higher dispersion for citizen scientists), indicating the reliability of CS-collected data. The quality of the videos, a potential source of variance, did not influence the results. Increasing practical training could be an alternative to improve pollen data quality. Our study shows that CS provides reliable data for monitoring bee activity and highlights the relevance of a multi-dimensional approach for assessing CS data quality.

The Native Bees of Texas: Evaluating the Benefits of a Public Engagement Course

Simple Summary

As concerns over bee population declines have entered the public consciousness worldwide, people are eager to learn about bees, the roles they play in our world, and how to conserve them. However, the public’s growing enthusiasm and efforts to conserve bees in North America are not always matched by their scientific knowledge of native bees. To satisfy a growing regional demand for knowledge about native bees, we have developed a public engagement program that aims to provide basic information about the native bees of Texas and their conservation guidelines based on science. At the University of Texas, Austin, we designed an outreach course with the objectives of teaching basic identification, diversity, ecology, and conservation of native bees and we implemented it on university botanical garden grounds. To gauge the course’s impact and quality, we integrated assessment tools into the course design. Evaluation results indicated that the course had a positive impact on participants who acquired specific topic knowledge and skills. The outreach course helped educate the public on native bees and benefitted participants, such as landowners and citizen scientists, who intended to apply their acquired knowledge and skills to specific conservation projects. It is relevant and timely to offer such courses, especially in regions that represent biodiversity hotspots for native bees and whose habitat is being fragmented and altered by rapid urbanization.

Abstract

Declines in native bee communities due to forces of global change have become an increasing public concern. Despite this heightened interest, there are few publicly available courses on native bees, and little understanding of how participants might benefit from such courses. In October of 2018 and 2019, we taught the ‘Native Bees of Texas’ course to the public at The University of Texas at Austin Lady Bird Johnson Wildflower Center botanical gardens in an active learning environment with slide-based presentations, printed photo-illustrated resources, and direct insect observations. In this study, we evaluated course efficacy and learning outcomes with a pre/post-course test, a survey, and open-ended feedback, focused on quality improvement findings. Overall, participants’ test scores increased significantly, from 60% to 87% correct answers in 2018 and from 64% to 87% in 2019, with greater post-course differences in ecological knowledge than in identification skills. Post-course, the mean of participants’ bee knowledge self-ratings was 4.56 on a five-point scale. The mean of participants’ ratings of the degree to which they attained the course learning objectives was 4.43 on a five-point scale. Assessment results provided evidence that the course enriched participants’ knowledge of native bee ecology and conservation and gave participants a basic foundation in bee identification. This highlights the utility of systematic course evaluations in public engagement efforts related to biodiversity conservation.

Studying phenological phenomena in subarctic biomes with international school pupils as citizen scientists

Citizen science can facilitate in-depth learning for pupils and students, contribute to scientific research, and permit civic participation. Here, we describe the development of the transnational school-based citizen science project Phenology of the North Calotte. Its primary goal is to introduce pupils (age 12-15; grades 7-10) in northern Norway, Russia, and Finland to the local and global challenges of climate change resulting in life cycle changes at different trophic and ecosystem levels in their backyards. Partnerships between regional scientists and staff from NIBIO Svanhovd, State nature reserves, national parks, and teachers and pupils from regional schools aim to engage pupils in project-based learning. The project uses standardized protocols, translated into the different languages of participating schools. The phenological observations are centered around documenting clearly defined life cycle phases (e.g., first appearance of species, flowering, ripening, leaf yellowing, snow fall, and melt). The observations are collected either on paper and are subsequently submitted manually to an open-source online database or submitted directly via a newly developed mobile app. In the long term, the database is anticipated to contribute to research studying changes in phenology at different trophic levels. In principle, guided school-based citizen science projects have the potential to contribute to increased environmental awareness and education and thereby to transformative learning at the societal level while contributing to scientific progress of understudied biomes, like the northern taiga and (sub)arctic tundra. However, differences in school systems and funding insecurity for some schools have been major prohibiting factors for long-term retention of pupils/schools in the program. Project-based and multidisciplinary learning, although pedagogically desired, has been partially difficult to implement in participating schools, pointing to the need of structural changes in national school curricula and funding schemes as well as continuous offers for training and networking for teachers.

Democratizing water monitoring: Implementation of a community-based qPCR monitoring program for recreational water hazards

Recreational water monitoring can be challenging due to the highly variable nature of pathogens and indicator concentrations, the myriad of potential biological hazards to measure for, and numerous access points, both official and unofficial, that are used for recreation. The aim of this study was to develop, deploy, and assess the effectiveness of a quantitative polymerase chain reaction (qPCR) community-based monitoring (CBM) program for the assessment of bacterial and parasitic hazards in recreational water. This study developed methodologies for performing qPCR ‘in the field,’ then engaged with water management and monitoring groups and tested the method in a real-world implementation study to evaluate the accuracy of CBM using qPCR both quantitatively and qualitatively. This study found high reproducibility between qPCR results performed by non-expert field users and expert laboratory results, suggesting that qPCR as a methodology could be amenable to a CBM program.

Using verified citizen science as a tool for monitoring the European hornet (Vespa crabro) in the island of Sardinia (Italy)

The European hornet, Vespa crabro L. (Hymenoptera: Vespidae), is a eusocial insect native to Eurasia that was accidentally introduced in the island of Sardinia (Italy) in 2010. This alien generalist predator could impact on native insect species through predation or competition by modifying interspecific relations in sensitive island ecosystems. As part of the Interreg project ALIEM, the present work regarded the first monitoring activity of the European hornet conducted in Sardinia by means of verified citizen science. The main goals of this study were to define the distribution area of the wasp in 2018 and evaluate the profile and the performance of citizen scientists. Our results showed that V. crabro is mainly located in north-eastern Sardinia and that the hotspot was probably the commercial port of Olbia. Furthermore, data provided by contributors were very accurate and none of the parameters considered to define the participants (age, gender, education level, job category) was a true predictor of a volunteer’s ability to recognise the wasp. In conclusion, this small-scale study suggests that citizen scientists could be a valuable aid to monitor already-established alien species and could be part of a surveillance network for early detection of other potentially invasive alien species not yet introduced in Sardinia, such as the yellow-legged hornet, Vespa velutina, which is already present in northern Italy.

Evaluating the ability of citizen scientists to identify bumblebee (Bombus) species

Citizen science is an increasingly popular way of engaging volunteers in the collection of scientific data. Despite this, data quality remains a concern and there is little published evidence about the accuracy of records generated by citizen scientists. Here we compare data generated by two British citizen science projects, Blooms for Bees and BeeWatch, to determine the ability of volunteer recorders to identify bumblebee (Bombus) species. We assessed recorders' identification ability in two ways-as recorder accuracy (the proportion of expert-verified records correctly identified by recorders) and recorder success (the proportion of recorder-submitted identifications confirmed correct by verifiers). Recorder identification ability was low (<50% accuracy; <60% success), despite access to project specific bumblebee identification materials. Identification ability varied significantly depending on bumblebee species, with recorders most able to correctly identify species with distinct appearances. Blooms for Bees recorders (largely recruited from the gardening community) were markedly less able to identify bumblebees than BeeWatch recorders (largely individuals with a more specific interest in bumblebees). Within both projects, recorders demonstrated an improvement in identification ability over time. Here we demonstrate and quantify the essential role of expert verification within citizen science projects, and highlight where resources could be strengthened to improve recorder ability.

How training citizen scientists affects the accuracy and precision of phenological data

Monitoring plant and animal phenology is a critical step to anticipating and predicting changes in species interactions and biodiversity. Because phenology necessarily involves frequent and repeated observations over time, citizen scientists have become a vital part of collecting phenological data. However, there is still concern over the accuracy and precision of citizen science data. It is possible that training citizen scientists can improve data quality though there are few comparisons of trained and untrained citizen scientists in the ability of each to accurately and precisely measure phenology. We assessed how three types of observers-experts, trained citizen scientists that make repeated observations, and untrained citizen scientists making once-per-year observations-differ in quantifying temporal change in flower and fruit abundance of American mountain ash trees (Sorbus americana Marsh.) and arthropods in Acadia National Park, Maine, USA. We found that trained more so than untrained citizen science observers over- or under-estimated abundances leading to precise but inaccurate characterizations of phenological patterns. Our results suggest a new type of bias induced by repeated observations: A type of learning takes place that reduces the independence of observations taken on different trees or different dates. Thus, in this and many other cases, having individuals make one-off observations of marked plants may produce data as good if not better than individuals making repeated observations. For citizen science programs related to phenology, our results underscore the importance of (a) attracting the most number of observers possible even if they only make one observation, (b) producing easy-to-use and informative data sheets, and

Utilization of photographs taken by citizens for estimating bumblebee distributions

Citizen science is a powerful tool for collecting large volumes of observational data on various species. These data are used to estimate distributions using environmental factors with Species Distribution Models (SDM). However, if citizens are inexperienced in recognizing organisms, they may report different species as the subject species. Here we show nation-wide bumblebee distributions using photographs taken by citizens in our project, and estimated distributions for six bumblebee species using land use, climate, and altitude data with SDM. We identified species from photographic images, and took their locations from GPS data of photographs or the text in e-mails. When we compared our data with conventional data for specimens in the Global Biodiversity Information Facility (GBIF), we found that the volume and the number of species were larger, and the bias of spatial range was lower, than those of GBIF. Our estimated distributions were more consistent with bumblebee distributions reported in previous studies than with those of GBIF. Our method was effective for collecting distribution data, and estimating distributions with SDM. The estimated SDM allows us to predict the previous and future species distributions, and to develop conservation policies taking account of future city planning and/or global climate changes.