Assessing Diabetes-Relevant Data Provided by Undergraduate and Crowdsourced Web-Based Survey Participants for Honesty and Accuracy

Response Consistency of Crowdsourced Web-Based Surveys on Type 1 Diabetes

Although Amazon Mechanical Turk facilitates the quick surveying of a large sample from various demographic and socioeconomic backgrounds, it may not be an optimal platform for obtaining reliable diabetes-related information from the online type 1 diabetes population.

Development and Validation of a Comprehensive Well-Being Scale for People in the University Environment (Pitt Wellness Scale) Using a Crowdsourcing Approach: Cross-Sectional Study

Background

Well-being has multiple domains, and these domains are unique to the population being examined. Therefore, to precisely assess the well-being of a population, a scale specifically designed for that population is needed.

Objective

The goal of this study was to design and validate a comprehensive well-being scale for people in a university environment, including students, faculty, and staff.

Methods

A crowdsourcing approach was used to determine relevant domains for the comprehensive well-being scale in this population and identify specific questions to include in each domain. A web-based questionnaire (Q1) was used to collect opinions from a group of university students, faculty, and staff about the domains and subdomains of the scale. A draft of a new well-being scale (Q2) was created in response to the information collected via Q1, and a second group of study participants was invited to evaluate the relevance and clarity of each statement. A newly created well-being scale (Q3) was then used by a third group of university students, faculty, and staff. A psychometric analysis was performed on the data collected via Q3 to determine the validity and reliability of the well-being scale.

Results

In the first step, a group of 518 university community members (students, faculty, and staff) indicated the domains and subdomains that they desired to have in a comprehensive well-being scale. In the second step, a second group of 167 students, faculty, and staff evaluated the relevance and clarity of the proposed statements in each domain. In the third step, a third group of 546 students, faculty, and staff provided their responses to the new well-being scale (Pitt Wellness Scale). The psychometric analysis indicated that the reliability of the well-being scale was high.

Conclusions

Using a crowdsourcing approach, we successfully created a comprehensive and highly reliable well-being scale for people in the university environment. Our new Pitt Wellness Scale may be used to measure the well-being of people in the university environment.

Mapping of Crowdsourcing in Health: Systematic Review

Background

Crowdsourcing involves obtaining ideas, needed services, or content by soliciting Web-based contributions from a crowd. The 4 types of crowdsourced tasks (problem solving, data processing, surveillance or monitoring, and surveying) can be applied in the 3 categories of health (promotion, research, and care).

Objective

This study aimed to map the different applications of crowdsourcing in health to assess the fields of health that are using crowdsourcing and the crowdsourced tasks used. We also describe the logistics of crowdsourcing and the characteristics of crowd workers.

Methods

MEDLINE, EMBASE, and ClinicalTrials.gov were searched for available reports from inception to March 30, 2016, with no restriction on language or publication status.

Results

We identified 202 relevant studies that used crowdsourcing, including 9 randomized controlled trials, of which only one had posted results at ClinicalTrials.gov. Crowdsourcing was used in health promotion (91/202, 45.0%), research (73/202, 36.1%), and care (38/202, 18.8%). The 4 most frequent areas of application were public health (67/202, 33.2%), psychiatry (32/202, 15.8%), surgery (22/202, 10.9%), and oncology (14/202, 6.9%). Half of the reports (99/202, 49.0%) referred to data processing, 34.6% (70/202) referred to surveying, 10.4% (21/202) referred to surveillance or monitoring, and 5.9% (12/202) referred to problem-solving. Labor market platforms (eg, Amazon Mechanical Turk) were used in most studies (190/202, 94%). The crowd workers’ characteristics were poorly reported, and crowdsourcing logistics were missing from two-thirds of the reports. When reported, the median size of the crowd was 424 (first and third quartiles: 167-802); crowd workers’ median age was 34 years (32-36). Crowd workers were mainly recruited nationally, particularly in the United States. For many studies (58.9%, 119/202), previous experience in crowdsourcing was required, and passing a qualification test or training was seldom needed (11.9% of studies; 24/202). For half of the studies, monetary incentives were mentioned, with mainly less than US $1 to perform the task. The time needed to perform the task was mostly less than 10 min (58.9% of studies; 119/202). Data quality validation was used in 54/202 studies (26.7%), mainly by attention check questions or by replicating the task with several crowd workers.

Conclusions

The use of crowdsourcing, which allows access to a large pool of participants as well as saving time in data collection, lowering costs, and speeding up innovations, is increasing in health promotion, research, and care. However, the description of crowdsourcing logistics and crowd workers’ characteristics is frequently missing in study reports and needs to be precisely reported to better interpret the study findings and replicate them.