1
|
Joyal-Desmarais K, Stojanovic J, Kennedy EB, Enticott JC, Boucher VG, Vo H, Košir U, Lavoie KL, Bacon SL, Granana N, Losada AV, Boyle J, Shawon SR, Dawadi S, Teede H, Kautzky-Willer A, Dash A, Cornelio ME, Karsten M, Matte DL, Reichert F, Abou-Setta A, Aaron S, Alberga A, Barnett T, Barone S, Bélanger-Gravel A, Bernard S, Birch LM, Bondy S, Booij L, Da Silva RB, Bourbeau J, Burns R, Campbell T, Carlson L, Charbonneau É, Corace K, Drouin O, Ducharme F, Farhadloo M, Falk C, Fleet R, Fournier M, Garber G, Gauvin L, Gordon J, Grad R, Gupta S, Hellemans K, Herba C, Hwang H, Jedwab J, Kakinami L, Kim S, Liu J, Norris C, Pelaez S, Pilote L, Poirier P, Presseau J, Puterman E, Rash J, Ribeiro PAB, Sadatsafavi M, Chaudhuri PS, Suarthana E, Tse S, Vallis M, Caceres NB, Ortiz M, Repetto PB, Lemos-Hoyos M, Kassianos A, Rod NH, Beraneck M, Ninot G, Ditzen B, Kubiak T, Codjoe S, Kpobi L, Laar A, Skoura T, Francis DL, Devi NK, Meitei S, Nethan ST, Pinto L, Saraswathy KN, Tumu D, Lestari S, Wangge G, Byrne M, Durand H, McSharry J, Meade O, Molloy G, Noone C, Levine H, Zaidman-Zait A, Boccia S, Hoxhaj I, Paduano S, Raparelli V, Zaçe D, Aburub A, Akunga D, Ayah R, Barasa C, Godia PM, Kimani-Murage EW, Mutuku N, Mwoma T, Naanyu V, Nyamari J, Oburu H, Olenja J, Ongore D, Ziraba A, Bandawe C, Yim L, Ajuwon A, Shar NA, Usmani BA, Martínez RMB, Creed-Kanashiro H, Simão P, Rutayisire PC, Bari AZ, Vojvodic K, Nagyova I, Bantjes J, Barnes B, Coetzee B, Khagee A, Mothiba T, Roomaney R, Swartz L, Cho J, Lee MG, Berman A, Stattin NS, Fischer S, Hu D, Kara Y, Şimşek C, Üzmezoğlu B, Isunju JB, Mugisha J, Byrne-Davis L, Griffiths P, Hart J, Johnson W, Michie S, Paine N, Petherick E, Sherar L, Bilder RM, Burg M, Czajkowski S, Freedland K, Gorin SS, Holman A, Lee J, Lopez G, Naar S, Okun M, Powell L, Pressman S, Revenson T, Ruiz J, Sivaram S, Thrul J, Trudel-Fitzgerald C, Yohannes A, Navani R, Ranakombu K, Neto DH, Ben-Porat T, Dragomir A, Gagnon-Hébert A, Gemme C, Jamil M, Käfer LM, Vieira AM, Tasbih T, Woods R, Yousefi R, Roslyakova T, Priesterroth L, Edelstein S, Snir R, Uri Y, Alyami M, Sanuade C, Crescenzi O, Warkentin K, Grinko K, Angne L, Jain J, Mathur N, Mithe A, Nethan S. How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses. Eur J Epidemiol 2022; 37:1233-1250. [PMID: 36335560 PMCID: PMC9638233 DOI: 10.1007/s10654-022-00932-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Accepted: 10/06/2022] [Indexed: 11/07/2022]
Abstract
COVID-19 research has relied heavily on convenience-based samples, which-though often necessary-are susceptible to important sampling biases. We begin with a theoretical overview and introduction to the dynamics that underlie sampling bias. We then empirically examine sampling bias in online COVID-19 surveys and evaluate the degree to which common statistical adjustments for demographic covariates successfully attenuate such bias. This registered study analysed responses to identical questions from three convenience and three largely representative samples (total N = 13,731) collected online in Canada within the International COVID-19 Awareness and Responses Evaluation Study ( www.icarestudy.com ). We compared samples on 11 behavioural and psychological outcomes (e.g., adherence to COVID-19 prevention measures, vaccine intentions) across three time points and employed multiverse-style analyses to examine how 512 combinations of demographic covariates (e.g., sex, age, education, income, ethnicity) impacted sampling discrepancies on these outcomes. Significant discrepancies emerged between samples on 73% of outcomes. Participants in the convenience samples held more positive thoughts towards and engaged in more COVID-19 prevention behaviours. Covariates attenuated sampling differences in only 55% of cases and increased differences in 45%. No covariate performed reliably well. Our results suggest that online convenience samples may display more positive dispositions towards COVID-19 prevention behaviours being studied than would samples drawn using more representative means. Adjusting results for demographic covariates frequently increased rather than decreased bias, suggesting that researchers should be cautious when interpreting adjusted findings. Using multiverse-style analyses as extended sensitivity analyses is recommended.
Collapse
Affiliation(s)
- Keven Joyal-Desmarais
- Department of Health, Kinesiology and Applied Physiology, Concordia University, 7141 Sherbrooke Street West, Montreal, QC H4B 1R6 Canada ,Montreal Behavioural Medicine Centre, CIUSSS-NIM, Montreal, Canada
| | - Jovana Stojanovic
- Montreal Behavioural Medicine Centre, CIUSSS-NIM, Montreal, Canada ,Canadian Agency for Drugs and Technologies in Health, Ottawa, Canada
| | - Eric B. Kennedy
- Disaster and Emergency Management, York University, Toronto, Canada
| | - Joanne C. Enticott
- Department of General Practice, Monash University, Melbourne, Australia ,Monash Partners, Advanced Health Research and Translation Centre, Melbourne, Australia
| | | | - Hung Vo
- Austin Health, Victoria, Australia
| | - Urška Košir
- Department of Health, Kinesiology and Applied Physiology, Concordia University, 7141 Sherbrooke Street West, Montreal, QC H4B 1R6 Canada ,Montreal Behavioural Medicine Centre, CIUSSS-NIM, Montreal, Canada
| | - Kim L. Lavoie
- Montreal Behavioural Medicine Centre, CIUSSS-NIM, Montreal, Canada ,Département de Psychologie, Université du Québec à Montréal, Montreal, Canada
| | - Simon L. Bacon
- Department of Health, Kinesiology and Applied Physiology, Concordia University, 7141 Sherbrooke Street West, Montreal, QC H4B 1R6 Canada ,Montreal Behavioural Medicine Centre, CIUSSS-NIM, Montreal, Canada
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|