Communicating Uncertainty in Written Consumer Health Information to the Public: Parallel-Group, Web-Based Randomized Controlled Trial

Alternative Presentations of Overall and Statistical Uncertainty for Adults' Understanding of the Results of a Randomized Trial of a Public Health Intervention: Parallel Web-Based Randomized Trials

BACKGROUND

Well-designed public health messages can help people make informed choices, while poorly designed messages or persuasive messages can confuse, lead to poorly informed decisions, and diminish trust in health authorities and research. Communicating uncertainties to the public about the results of health research is challenging, necessitating research on effective ways to disseminate this important aspect of randomized trials.

OBJECTIVE

This study aimed to evaluate people's understanding of overall and statistical uncertainty when presented with alternative ways of expressing randomized trial results.

METHODS

Two parallel, web-based, individually randomized trials (3×2 factorial designs) were conducted in the United States and Norway. Participants were randomized to 1 of 6 versions of a text (summary) communicating results from a study examining the effects of wearing glasses to prevent COVID-19 infection. The summaries varied in how overall uncertainty ("Grading of Recommendations Assessment, Development and Evaluation [GRADE] language," "plain language," or "no explicit language") and statistical uncertainty (whether a margin of error was shown or not) were presented. Participants completed a web-based questionnaire exploring 4 coprimary outcomes: 3 to measure understanding of overall uncertainty (benefits, harms, and sufficiency of evidence), and one to measure statistical uncertainty. Participants were adults who do not wear glasses recruited from web-based research panels in the United States and Norway. Results of the trials were analyzed separately and combined in a meta-analysis.

RESULTS

In the US and Norwegian trials, 730 and 497 individuals were randomized, respectively; data for 543 (74.4%) and 452 (90.9%) were analyzed. More participants had a correct understanding of uncertainty when presented with plain language (United States: 37/99, 37% and Norway: 40/76, 53%) than no explicit language (United States: 18/86, 21% and Norway: 34/80, 42%). Similar positive effect was seen for the GRADE language in the United States (26/79, 33%) but not in Norway (30/71, 42%). There were only small differences between groups for understanding the uncertainty of harms. Plain language improved correct understanding of evidence sufficiency (odds ratio 2.05, 95% CI 1.17-3.57), compared to no explicit language. The effect of GRADE language was inconclusive (odds ratio 1.34, 95% CI 0.79-2.28). The understanding of statistical uncertainty was improved when the participants were shown the margin of error compared to not being shown: Norway: 16/75, 21% to 24/71, 34% vs 1/71, 1% to 2/76, 3% and the United States: 21/101, 21% to 32/90, 36% vs 0/86, 0% to 3/79, 4%).

CONCLUSIONS

Plain language, but not GRADE language, was better than no explicit language in helping people understand overall uncertainty of benefits and harms. Reporting margin of error improved understanding of statistical uncertainty around the effect of wearing glasses, but only for a minority of participants.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05642754; https://tinyurl.com/4mhjsm7s.

Optimizing Communication on HPV Vaccination to Parents of 11- to 14-Year-Old Adolescents in France: A Discrete Choice Experiment

BACKGROUND

With the aim to optimize communication during HPV vaccination campaigns in France, we elicited parental preferences around HPV vaccination.

METHODS

We conducted a single-profile discrete choice experiment (DCE) among parents of 11- to 14-year-old middle-school pupils, who completed an anonymous, self-administered, internet-based questionnaire during 2020-2021. The DCE comprised five attributes (vaccine-preventable disease, justification of optimal age, information on safety, indirect protection and coverage) of vaccination against an unnamed disease that were presented to respondents in ten choice tasks, or scenarios. We use fixed effect logit models to estimate attribute weights on theoretical vaccine acceptance, and random effect linear regression to estimate attribute coefficients on vaccine eagerness (decision and decision certainty). We estimated marginal effects of attributes on expected vaccine acceptance.

RESULTS

Vaccination scenarios were accepted by 55.6-89.2% of the 1291 participants. The largest marginal effects on expected vaccine acceptance in the full sample arose from prevention of cancer versus genital warts (+ 11.3 percentage points); from a "severe side effect suspicion that was not scientifically confirmed" versus a statement about "more benefits than risks" (+ 8.9 percentage points), and information on 80% vaccine coverage in neighbouring countries versus on "insufficient coverage" (+ 4.2 percentage points). Explaining the early age of vaccination by sexual debut had a strong negative impact among French monolingual parents with lower education level (vs age-independent, OR 0.48, 95% CI 0.27-0.86), but not other socio-economic groups. After removing low-quality responses (unvaried certainty and short questionnaire completion), among serial non-demanders with children not vaccinated against HPV, only disease elimination impacted vaccine eagerness positively (coefficient 0.54, 0.06-1.02).

DISCUSSION

Using DCEs to elicit parents' preferences around communication messages, notably on cancer prevention, vaccine coverage and information about vaccine safety, could help to optimize HPV vaccination promotion efforts.

Understanding the Impact of Communicating Uncertainty About COVID-19 in the News: Randomized Between-Subjects Factorial Experiment

BACKGROUND

Whether and how the uncertainty about a public health crisis should be communicated to the general public have been important and yet unanswered questions arising over the past few years. As the most threatening contemporary public health crisis, the COVID-19 pandemic has renewed interest in these unresolved issues by both academic scholars and public health practitioners.

OBJECTIVE

The aim of this study was to investigate the impact of communicating uncertainty about COVID-19-related threats and solutions on individuals' risk perceptions and misinformation vulnerability, as well as the sequential impact of these effects on health information processing and preventative behavioral intentions.

METHODS

A 2×2 (threat uncertainty [presence vs absence]×solution uncertainty [presence vs absence]) full-fractional between-subjects online experiment was conducted with 371 Chinese adults. Focusing on the discussion of whether the asymptomatic cases detected during the COVID-19 pandemic would further lead to an uncontrolled pandemic, news articles were manipulated in terms of whether the infectiousness of asymptomatic cases and the means to control the transmission are presented in terms of their certainty or uncertainty. Participants were randomly assigned to one of the four experimental conditions, being instructed to read one news article. After reading the news article assigned, participants were asked to respond to a series of questions to assess their cognitive and behavioral responses.

RESULTS

Individuals were more susceptible to believing false COVID-19-related information when a certain threat and uncertain solution were framed in the news article. Moreover, individuals' perceptions of crisis severity increased when exposed to news information containing uncertain solutions. Both misinformation vulnerability and perceived severity were positively associated with information processing. Information seeking was positively associated with protective behavioral intention, whereas information avoidance was negatively associated with protective behavioral intention.

CONCLUSIONS

Our findings imply that uncertainty, depending on its aspect, can be effectively communicated to the public during an emerging public health crisis. These results have theoretical and practical implications for health communicators and journalists. Given its limited influence on individuals' cognitive and behavioral responses, uncertainty related to a health threat should be disseminated to meet the public's expectation of information transparency. However, caution is advised when communicating uncertainty related to potential solutions, as this factor exhibited a mixed impact on individual responses during a crisis.

Strategies for communicating scientific evidence on healthcare to managers and the population: a scoping review

BACKGROUND

Health evidence needs to be communicated and disseminated in a manner that is clearly understood by decision-makers. As an inherent component of health knowledge translation, communicating results of scientific studies, effects of interventions and health risk estimates, in addition to understanding key concepts of clinical epidemiology and interpreting evidence, represent a set of essential instruments to reduce the gap between science and practice. The advancement of digital and social media has reshaped the concept of health communication, introducing new, direct and powerful communication platforms and gateways between researchers and the public. The objective of this scoping review was to identify strategies for communicating scientific evidence in healthcare to managers and/or population.

METHODS

We searched Cochrane Library, Embase®, MEDLINE® and other six electronic databases, in addition to grey literature, relevant websites from related organizations for studies, documents or reports published from 2000, addressing any strategy for communicating scientific evidence on healthcare to managers and/or population.

RESULTS

Our search identified 24 598 unique records, of which 80 met the inclusion criteria and addressed 78 strategies. Most strategies focused on risk and benefit communication in health, were presented by textual format and had been implemented and somehow evaluated. Among the strategies evaluated and appearing to yield some benefit are (i) risk/benefit communication: natural frequencies instead of percentages, absolute risk instead relative risk and number needed to treat, numerical instead nominal communication, mortality instead survival; negative or loss content appear to be more effective than positive or gain content; (ii) evidence synthesis: plain languages summaries to communicate the results of Cochrane reviews to the community were perceived as more reliable, easier to find and understand, and better to support decisions than the original summaries; (iii) teaching/learning: the Informed Health Choices resources seem to be effective for improving critical thinking skills.

CONCLUSION

Our findings contribute to both the knowledge translation process by identifying communication strategies with potential for immediate implementation and to future research by recognizing the need to evaluate the clinical and social impact of other strategies to support evidence-informed policies. Trial registration protocol is prospectively available in MedArxiv (doi.org/10.1101/2021.11.04.21265922).

A scattered landscape: assessment of the evidence base for 71 patient decision aids developed in a hospital setting

Background

Recent publications reveal shortcomings in evidence review and summarization methods for patient decision aids. In the large-scale “Share to Care (S2C)” Shared Decision Making (SDM) project at the University Hospital Kiel, Germany, one of 4 SDM interventions was to develop up to 80 decision aids for patients. Best available evidence on the treatments’ impact on patient-relevant outcomes was systematically appraised to feed this information into the decision aids. Aims of this paper were to (1) describe how PtDAs are developed and how S2C evidence reviews for each PtDA are conducted, (2) appraise the quality of the best available evidence identified and (3) identify challenges associated with identified evidence.

Methods

The quality of the identified evidence was assessed based on GRADE quality criteria and categorized into high-, moderate-, low-, very low-quality evidence. Evidence appraisal was conducted across all outcomes assessed in an evidence review and for specific groups of outcomes, namely mortality, morbidity, quality of life, and treatment harms. Challenges in evidence interpretation and summarization resulting from the characteristics of decision aids and the type and quality of evidence are identified and discussed.

Results

Evidence reviews assessed on average 25 systematic reviews/guidelines/studies and took about 3 months to be completed. Despite rigorous review processes, nearly 70% of outcome-specific information derived for decision aids was based on low-quality and mostly on non-directly comparative evidence. Evidence on quality of life and harms was often not provided or not in sufficient form/detail. Challenges in evidence interpretation for use in decision aids resulted from, e.g., a lack of directly comparative evidence or the existence of very heterogeneous evidence for the diverse treatments being compared.

Conclusions

Evidence reviews in this project were carefully conducted and summarized. However, the evidence identified for our decision aids was indeed a “scattered landscape” and often poor quality. Facing a high prevalence of low-quality, non-directly comparative evidence for treatment alternatives doesn’t mean it is not necessary to choose an evidence-based approach to inform patients. While there is an urgent need for high quality comparative trials, best available evidence nevertheless has to be appraised and transparently communicated to patients.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01777-x.

The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials

BACKGROUND

The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public.

METHODS

In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured.

FINDINGS

Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was 'low', rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was 'high', and in one of the two studies, those shown 'low' quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was 'high' showed no statistically significant differences on these measures compared to those given no information about evidence quality.

CONCLUSIONS

Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding ineligible criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the review, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

Background Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032). Methods We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding inappropriate criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies.  Results We included 44 primary studies in the qualitative summary, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty).  Discussion There is room for improving health news, but it is logically more important to improve the public's ability to critically appraise health information and make judgements for themselves.