Assessing digital literacy in web-based physical activity surveillance: the WIN study

Combating Fraudulent Participation in Urban American Indian and Alaska Native Virtual Health Research: Protocol for Increasing Data Integrity in Online Research (PRIOR)

BACKGROUND

While the advantages of using the internet and social media for research recruitment are well documented, the evolving online environment also enhances motivations for misrepresentation to receive incentives or to "troll" research studies. Such fraudulent assaults can compromise data integrity, with substantial losses in project time; money; and especially for vulnerable populations, research trust. With the rapid advent of new technology and ever-evolving social media platforms, it has become easier for misrepresentation to occur within online data collection. This perpetuation can occur by bots or individuals with malintent, but careful planning can help aid in filtering out fraudulent data.

OBJECTIVE

Using an example with urban American Indian and Alaska Native young women, this paper aims to describe PRIOR (Protocol for Increasing Data Integrity in Online Research), which is a 2-step integration protocol for combating fraudulent participation in online survey research.

METHODS

From February 2019 to August 2020, we recruited participants for formative research preparatory to an online randomized control trial of a preconceptual health program. First, we described our initial protocol for preventing fraudulent participation, which proved to be unsuccessful. Then, we described modifications we made in May 2020 to improve the protocol performance and the creation of PRIOR. Changes included transferring data collection platforms, collecting embedded geospatial variables, enabling timing features within the screening survey, creating URL links for each method or platform of data collection, and manually confirming potentially eligible participants' identifying information.

RESULTS

Before the implementation of PRIOR, the project experienced substantial fraudulent attempts at study enrollment, with less than 1% (n=6) of 1300 screened participants being identified as truly eligible. With the modified protocol, of the 461 individuals who completed a screening survey, 381 did not meet the eligibility criteria assessed on the survey. Of the 80 that did, 25 (31%) were identified as ineligible via PRIOR. A total of 55 (69%) were identified as eligible and verified in the protocol and were enrolled in the formative study.

CONCLUSIONS

Fraudulent surveys compromise study integrity, validity of the data, and trust among participant populations. They also deplete scarce research resources including respondent compensation and personnel time. Our approach of PRIOR to prevent online misrepresentation in data was successful. This paper reviews key elements regarding fraudulent data participation in online research and demonstrates why enhanced protocols to prevent fraudulent data collection are crucial for building trust with vulnerable populations.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04376346; https://www.clinicaltrials.gov/study/NCT04376346.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/52281.

Adherence to Blended or Face-to-Face Smoking Cessation Treatment and Predictors of Adherence: Randomized Controlled Trial

BACKGROUND

Blended face-to-face and web-based treatment is a promising way to deliver smoking cessation treatment. Since adherence has been shown to be an indicator of treatment acceptability and a determinant for effectiveness, we explored and compared adherence and predictors of adherence to blended and face-to-face alone smoking cessation treatments with similar content and intensity.

OBJECTIVE

The objectives of this study were (1) to compare adherence to a blended smoking cessation treatment with adherence to a face-to-face treatment; (2) to compare adherence within the blended treatment to its face-to-face mode and web mode; and (3) to determine baseline predictors of adherence to both treatments as well as (4) the predictors to both modes of the blended treatment.

METHODS

We calculated the total duration of treatment exposure for patients (N=292) of a Dutch outpatient smoking cessation clinic who were randomly assigned either to the blended smoking cessation treatment (n=130) or to a face-to-face treatment with identical components (n=162). For both treatments (blended and face-to-face) and for the two modes of delivery within the blended treatment (face-to-face vs web mode), adherence levels (ie, treatment time) were compared and the predictors of adherence were identified within 33 demographic, smoking-related, and health-related patient characteristics.

RESULTS

We found no significant difference in adherence between the blended and the face-to-face treatments. Participants in the blended treatment group spent an average of 246 minutes in treatment (median 106.7% of intended treatment time, IQR 150%-355%) and participants in the face-to-face group spent 238 minutes (median 103.3% of intended treatment time, IQR 150%-330%). Within the blended group, adherence to the face-to-face mode was twice as high as that to the web mode. Participants in the blended group spent an average of 198 minutes (SD 120) in face-to-face mode (152% of the intended treatment time) and 75 minutes (SD 53) in web mode (75% of the intended treatment time). Higher age was the only characteristic consistently found to uniquely predict higher adherence in both the blended and face-to-face groups. For the face-to-face group, more social support for smoking cessation was also predictive of higher adherence. The variability in adherence explained by these predictors was rather low (blended R2=0.049; face-to-face R2=0.076). Within the blended group, living without children predicted higher adherence to the face-to-face mode (R2=0.034), independent of age. Higher adherence to the web mode of the blended treatment was predicted by a combination of an extrinsic motivation to quit, a less negative attitude toward quitting, and less health complaints (R2=0.164).

CONCLUSIONS

This study represents one of the first attempts to thoroughly compare adherence and predictors of adherence of a blended smoking cessation treatment to an equivalent face-to-face treatment. Interestingly, although the overall adherence to both treatments appeared to be high, adherence within the blended treatment was much higher for the face-to-face mode than for the web mode. This supports the idea that in blended treatment, one mode of delivery can compensate for the weaknesses of the other. Higher age was found to be a common predictor of adherence to the treatments. The low variance in adherence predicted by the characteristics examined in this study suggests that other variables such as provider-related health system factors and time-varying patient characteristics should be explored in future research.

TRIAL REGISTRATION

Netherlands Trial Register NTR5113; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=5113.

Methodological Challenges in Web-Based Trials: Update and Insights From the Relatives Education and Coping Toolkit Trial

RR2-10.1136/bmjopen-2017-016965.

Attrition in an Online Loneliness Intervention for Adults Aged 50 Years and Older: Survival Analysis

BACKGROUND

Online interventions can be as effective as in-person interventions. However, attrition in online intervention is high and potentially biases the results. More importantly, high attrition rates might reduce the effectiveness of online interventions. Therefore, it is important to discover the extent to which factors affect adherence to online interventions. The setting for this study is the online Friendship Enrichment Program, a loneliness intervention for adults aged 50 years and older.

OBJECTIVE

This study examined the contribution of severity of loneliness, coping preference, activating content, and engagement in attrition within an online intervention.

METHODS

Data were collected from 352 participants in an online loneliness intervention for Dutch people aged 50 years and older. Attrition was defined as not completing all 10 intervention lessons. The number of completed lessons was assessed through the management system of the intervention. We tested 4 hypotheses on attrition by applying survival analysis (Cox regression).

RESULTS

Of the 352 participants who subscribed to the intervention, 46 never started the introduction. The remaining 306 participants were divided into 2 categories: 73 participants who did not start the lessons of the intervention and 233 who started the lessons of the intervention. Results of the survival analysis (n=233) showed that active coping preference (hazard ratio [HR]=0.73), activating content (HR=0.71), and 2 indicators of engagement (HR=0.94 and HR=0.79) lowered attrition. Severity of loneliness was not related to attrition.

CONCLUSIONS

To reduce attrition, developers of online (loneliness) interventions may focus on stimulating active behavior within the intervention.

Blended Smoking Cessation Treatment: Exploring Measurement, Levels, and Predictors of Adherence

Background

Blended face-to-face and Web-based treatment is a promising way to deliver cognitive behavioral therapy. Since adherence has been shown to be a measure for treatment’s acceptability and a determinant for treatment’s effectiveness, in this study, we explored adherence to a new blended smoking cessation treatment (BSCT).

Objective

The objective of our study was to (1) develop an adequate method to measure adherence to BSCT; (2) define an adequate degree of adherence to be used as a threshold for being adherent; (3) estimate adherence to BSCT; and (4) explore the possible predictors of adherence to BSCT.

Methods

The data of patients (N=75) were analyzed to trace adherence to BSCT delivered at an outpatient smoking cessation clinic. In total, 18 patient activities (eg, using a Web-based smoking diary tool or responding to counselors’ messages) were selected to measure adherence; the degree of adherence per patient was compared with quitting success. The minimum degree of adherence of patients who reported abstinence was examined to define a threshold for the detection of adherent patients. The number of adherent patients was calculated for each of the 18 selected activities; the degree of adherence over the course of the treatment was displayed; and the number of patients who were adherent was analyzed. The relationship between adherence and 33 person-, smoking-, and health-related characteristics was examined.

Results

The method for measuring adherence was found to be adequate as adherence to BSCT correlated with self-reported abstinence (P=.03). Patients reporting abstinence adhered to at least 61% of BSCT. Adherence declined over the course of the treatment; the percentage of adherent patients per treatment activity ranged from 82% at the start of the treatment to 11%-19% at the final-third of BSCT; applying a 61% threshold, 18% of the patients were classified as adherent. Marital status and social modeling were the best independent predictors of adherence. Patients having a partner had 11-times higher odds of being adherent (OR [odds ratio]=11.3; CI: 1.33-98.99; P=.03). For social modeling, graded from 0 (=partner and friends are not smoking) to 8 (=both partner and nearly all friends are smoking), each unit increase was associated with 28% lower odds of being adherent (OR=0.72; CI: 0.55-0.94; P=.02).

Conclusions

The current study is the first to explore adherence to a blended face-to-face and Web-based treatment (BSCT) based on a substantial group of patients. It revealed a rather low adherence rate to BSCT. The method for measuring adherence to BSCT could be considered adequate because the expected dose-response relationship between adherence and quitting could be verified. Furthermore, this study revealed that marital status and social modeling were independent predictors of adherence.

Trial Registration

Netherlands Trial Registry NTR5113; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=5113 (Archived by WebCite at http://www.webcitation.org/71BAPwER8).

Impact of Baseline Assessment Modality on Enrollment and Retention in a Facebook Smoking Cessation Study

Background

Few studies have addressed enrollment and retention methods in online smoking cessation interventions. Fully automated Web-based trials can yield large numbers of participants rapidly but suffer from high rates of attrition. Personal contact with participants can increase recruitment of smokers into cessation trials and improve participant retention.

Objective

To compare the impact of Web-based (WEB) and phone (PH) baseline assessments on enrollment and retention metrics in the context of a Facebook smoking cessation study.

Methods

Participants were recruited via Facebook and Google ads which were randomly displayed to adult smokers in the United States over 27 days from August to September 2013. On each platform, two identical ads were randomly displayed to users who fit the advertising parameters. Clicking on one of the ads resulted in randomization to WEB, and clicking on the other ad resulted in randomization to PH. Following online eligibility screening and informed consent, participants in the WEB arm completed the baseline survey online whereas PH participants completed the baseline survey by phone with a research assistant. All participants were contacted at 30 days to complete a follow-up survey that assessed use of the cessation intervention and smoking outcomes. Participants were paid $15 for follow-up survey completion.

Results

A total of 4445 people clicked on the WEB ad and 4001 clicked on the PH ad: 12.04% (n=535) of WEB participants and 8.30% (n=332) of PH participants accepted the online study invitation (P<.001). Among the 726 participants who completed online eligibility screening, an equivalent proportion in both arms was eligible and an equivalent proportion of the eligible participants in both arms provided informed consent. There was significant drop-off between consent and completion of the baseline survey in the PH arm, resulting in enrollment rates of 32.7% (35/107) for the PH arm and 67.9% (114/168) for the WEB arm (P<.001). The overall enrollment rate among everyone who clicked on a study ad was 2%. There were no between group differences in the proportion that installed the Facebook app (66/114, 57.9% WEB vs 17/35, 49% PH) or that completed the 30-day follow-up survey (49/114, 43.0% WEB vs 16/35, 46% PH). A total of $6074 was spent on ads, generating 3,834,289 impressions and resulting in 8446 clicks (average cost $0.72 per click). Per participant enrollment costs for advertising alone were $27 WEB and $87 PH.

Conclusions

A more intensive phone baseline assessment protocol yielded a lower rate of enrollment, equivalent follow-up rates, and higher enrollment costs compared to a Web-based assessment protocol. Future research should focus on honing mixed-mode assessment protocols to further optimize enrollment and retention.

How computer literacy and socioeconomic status affect attitudes toward a Web-based cohort: results from the NutriNet-Santé study

BACKGROUND

In spite of the growing literature in the field of e-epidemiology, clear evidence about computer literacy or attitudes toward respondent burden among e-cohort participants is largely lacking.

OBJECTIVE

We assessed the computer and Internet skills of participants in the NutriNet-Santé Web-based cohort. We then explored attitudes toward the study demands/respondent burden according to levels of computer literacy and sociodemographic status.

METHODS

Self-reported data from 43,028 e-cohort participants were collected in 2013 via a Web-based questionnaire. We employed unconditional logistic and linear regression analyses.

RESULTS

Approximately one-quarter of participants (23.79%, 10,235/43,028) reported being inexperienced in terms of computer use. Regarding attitudes toward participant burden, women tended to be more favorable (eg, "The overall website use is easy") than were men (OR 0.65, 95% CI 0.59-0.71, P<.001), whereas better educated participants (>12 years of schooling) were less likely to accept the demands associated with participation (eg, "I receive questionnaires too often") compared to their less educated counterparts (OR 1.62, 95% CI 1.48-1.76, P<.001).

CONCLUSIONS

A substantial proportion of participants had low computer/Internet skills, suggesting that this does not represent a barrier to participation in Web-based cohorts. Our study also suggests that several subgroups of participants with lower computer skills (eg, women or those with lower educational level) might more readily accept the demands associated with participation in the Web cohort. These findings can help guide future Web-based research strategies.

Patients report worse MS symptoms after menopause: findings from an online cohort

BACKGROUND

Many women with multiple sclerosis (MS) are postmenopausal, yet the impact of menopause on MS symptoms is unknown.

OBJECTIVE

To investigate patient-reported impact of menopause in a large online research platform, PatientsLikeMe (PLM).

METHODS

A detailed reproductive history survey was deployed to PLM members, and responses were linked to PLM׳s prospectively collected patient-reported severity score (MS Rating Scale, MSRS). The MSRS has previously shown good correlation with physician-derived EDSS scores.

RESULTS

Of the 513 respondents, 55% were postmenopausal; 54% of these reported induced menopause. Median age at natural menopause was 51. Surgical menopause occurred at an earlier age (p<0.001) and was associated with more hormone replacement therapy use (p=0.02) than natural menopause. Postmenopausal status, surgical menopause, and earlier age at menopause were all associated with worse MSRS scores (p≤0.01) in regressions adjusting for age, disease type and duration.

CONCLUSION

Postmenopausal patients in this study reported worse MS disease severity. Further, this study highlights a utility for online research platforms, which allow for rapid generation of hypotheses that then require validation in clinical settings.

Attrition revisited: adherence and retention in a web-based alcohol trial

Background

Attrition is a noted feature of eHealth interventions and trials. In 2005, Eysenbach published a landmark paper calling for a “science of attrition,” suggesting that the 2 forms of attrition—nonusage attrition (low adherence to the intervention) and dropout attrition (poor retention to follow-up)—may be related and that this potential relationship deserved further study.

Objective

The aim of this paper was to use data from an online alcohol trial to explore Eysenbach’s hypothesis, and to answer 3 research questions: (1) Are adherence and retention related? If so, how, and under which circumstances? (2) Do adherence and retention have similar predictors? Can these predictors adequately explain any relationship between adherence and retention or are there additional, unmeasured predictors impacting on the relationship? (3) If there are additional unmeasured predictors impacting on the relationship, are there data to support Eysenbach’s hypothesis that these are related to overall levels of interest?

Methods

Secondary analysis of data from an online trial of an online intervention to reduce alcohol consumption among heavy drinkers. The 2 outcomes were adherence to the intervention measured by number of log-ins, and retention to the trial measured by provision of follow-up data at 3 months (the primary outcome point). Dependent variables were demographic and alcohol-related data collected at baseline. Predictors of adherence and retention were modeled using logistic regression models.

Results

Data were available on 7932 participants. Adherence and retention were related in a complex fashion. Participants in the intervention group were more likely than those in the control group to log in more than once (42% vs 28%, P<.001) and less likely than those in the control group to respond at 3 months (40% vs 49%, P<.001). Within each randomized group, participants who logged in more frequently were more likely to respond than those who logged in less frequently. Response rates in the intervention group for those who logged in once, twice, or ≥3 times were 34%, 46%, and 51%, respectively (P<.001); response rates in the control group for those who logged in once, twice, or ≥3 times were 44%, 60%, and 67%, respectively (P<.001). Relationships between baseline characteristics and adherence and retention were also complex. Where demographic characteristics predicted adherence, they tended also to predict retention. However, characteristics related to alcohol consumption and intention or confidence in reducing alcohol consumption tended to have opposite effects on adherence and retention, with factors that predicted improved adherence tending to predict reduced retention. The complexity of these relationships suggested the existence of an unmeasured confounder.

Conclusions

In this dataset, adherence and retention were related in a complex fashion. We propose a possible explanatory model for these data.

Trial Registration

International Standard Randomized Controlled Trial Number (ISRCTN): 31070347; http://www.controlled-trials.com/ISRCTN31070347 (Archived by WebCite at http://www.webcitation.org/6IEmNnlCn).

Evaluation of an online platform for multiple sclerosis research: patient description, validation of severity scale, and exploration of BMI effects on disease course

Objectives

To assess the potential of an online platform, PatientsLikeMe.com (PLM), for research in multiple sclerosis (MS). An investigation of the role of body mass index (BMI) on MS disease course was conducted to illustrate the utility of the platform.

Methods

First, we compared the demographic characteristics of subjects from PLM and from a regional MS center. Second, we validated PLM’s patient-reported outcome measure (MS Rating Scale, MSRS) against standard physician-rated tools. Finally, we analyzed the relation of BMI to the MSRS measure.

Results

Compared with 4,039 MS Center patients, the 10,255 PLM members were younger, more educated, and less often male and white. Disease course was more often relapsing remitting, with younger symptom onset and shorter disease duration. Differences were significant because of large sample sizes but small in absolute terms. MSRS scores for 121 MS Center patients revealed acceptable agreement between patient-derived and physician-derived composite scores (weighted kappa = 0.46). The Walking domain showed the highest weighted kappa (0.73) and correlation (rs = 0.86) between patient and physician scores. Additionally, there were good correlations between the patient-reported MSRS composite and walking scores and physician-derived measures: Expanded Disability Status Scale (composite rs = 0.61, walking rs = 0.74), Timed 25 Foot Walk (composite rs = 0.70, walking rs = 0.69), and Ambulation Index (composite rs = 0.81, walking rs = 0.84). Finally, using PLM data, we found a modest correlation between BMI and cross-sectional MSRS (rho = 0.17) and no association between BMI and disease course.

Conclusions

The PLM population is comparable to a clinic population, and its patient-reported MSRS is correlated with existing clinical instruments. Thus, this online platform may provide a venue for MS investigations with unique strengths (frequent data collection, large sample sizes). To illustrate its applicability, we assessed the role of BMI in MS disease course but did not find a clinically meaningful role for BMI in this setting.