Educational interventions to improve people's understanding of key concepts in assessing the effects of health interventions: a systematic review protocol

Educational Video Intervention to Improve Health Misinformation Identification on WhatsApp Among Saudi Arabian Population: Pre-Post Intervention Study

BACKGROUND

Health misinformation can adversely affect individuals' quality of life and increase the risk of mortality. People often fail to assess the content of messages before sharing them on the internet, increasing the spread of misinformation. The problem is exacerbated by the growing variety of digital information environments, especially social media, which presents as an effective platform for spreading misinformation due to its rapid information-sharing capabilities. Educational interventions have been developed to help consumers verify the validity of digital health information. However, tools designed to detect health misinformation on social media content have not been validated. Given the increased use of social media platforms, particularly WhatsApp, it is crucial to develop tools to help consumers assess the credibility of messages and detect misinformation.

OBJECTIVE

The main objective of this study is to develop and assess an educational tool aimed at educating consumers about detecting health misinformation on WhatsApp. The secondary objective is to assess the association between demographic factors and knowledge levels.

METHODS

The study used a single-arm, pre-post intervention design to evaluate the effectiveness of an educational video in improving participants' ability to detect health-related misinformation in WhatsApp messages. In the first phase, an educational video intervention was developed and validated. In the second phase, participants were invited to complete a web-based survey that consisted of pre-evaluation questions, followed by the educational video intervention. Subsequently, they were asked to answer the same questions as the postevaluation questions.

RESULTS

The web-based survey received 485 responses. The completion rate was 99.6% (n=483). Statistically significant associations existed between knowledge level and age, gender, employment, and region of residence (P<.05). The video intervention did elicit a statistically significant change in the participants' abilities to identify misinformation in WhatsApp messages (z=-6.887; P<.001). Viewing the video was associated with increased knowledge about the following concepts: checking the "forwarded" label (P<.001), looking for spelling and grammatical errors (P<.001), analyzing the facts (P=.03), checking links (P=.002, P=.001), and assessing the photos and videos (P<.001). There was a statistically significant difference in knowledge level before and after the intervention (P<.001).

CONCLUSIONS

This study developed and evaluated the effectiveness of an educational video intervention to improve health misinformation identification on WhatsApp among the Saudi Arabian population. The results indicate that educational videos can be valuable tools for improving participants' abilities to identify misinformation. The outcomes of this research can contribute to our understanding of what constitutes an effective tool for enhancing health misinformation awareness. Such interventions may be particularly useful in combating misinformation among Arabic-speaking populations on WhatsApp, which may ultimately improve eHealth literacy. Limiting the prevalence and impact of misinformation allows people to make better-informed health decisions.

The Norwegian public's ability to assess treatment claims: results of a cross-sectional study of critical health literacy

Background: Few studies have evaluated the ability of the general public to assess the trustworthiness of claims about the effects of healthcare. For the most part, those studies have used self-reported measures of critical health literacy. Methods: We mailed 4500 invitations to Norwegian adults. Respondents were randomly assigned to one of four online questionnaires that included multiple-choice questions that test understanding of Key Concepts people need to understand to assess healthcare claims. They also included questions about intended behaviours and self-efficacy. One of the four questionnaires was identical to one previously used in two randomised trials of educational interventions in Uganda, facilitating comparisons to Ugandan children, parents, and teachers. We adjusted the results using demographic data to reflect the population. Results: A total of 771 people responded. The adjusted proportion of Norwegian adults who answered correctly was < 50% for 17 of the 30 Key Concepts. On the other hand, less than half answered correctly for 13 concepts. The results for Norwegian adults were better than the results for Ugandan children in the intervention arm of the trial and parents, and similar to those of Ugandan teachers in the intervention arm of the trial. Based on self-report, most Norwegians are likely to find out the basis of treatment claims, but few consider it easy to assess whether claims are based on research and to assess the trustworthiness of research. Conclusions: Norwegian adults do not understand many concepts that are essential for assessing healthcare claims and making informed choices. Future interventions should be tailored to address Key Concepts for which there appears to be a lack of understanding.

The James Lind Initiative: books, websites and databases to promote critical thinking about treatment claims, 2003 to 2018

BACKGROUND

The James Lind Initiative (JLI) was a work programme inaugurated by Iain Chalmers and Patricia Atkinson to press for better research for better health care. It ran between 2003 and 2018, when Iain Chalmers retired. During the 15 years of its existence, the JLI developed three strands of work in collaboration with the authors of this paper, and with others.

WORK THEMES

The first work strand involved developing a process for use by patients, carers and clinicians to identify shared priorities for research - the James Lind Alliance. The second strand was a series of articles, meetings, prizes and other developments to raise awareness of the massive amounts of avoidable waste in research, and of ways of reducing it. The third strand involved using a variety of approaches to promote better public and professional understanding of the importance of research in clinical practice and public health. JLI work on the first two themes has been addressed in previously published reports. This paper summarises JLI involvement during the 15 years of its existence in giving talks, convening workshops, writing books, and creating websites and databases to promote critical thinking about treatment claims.

CONCLUSION

During its 15-year life, the James Lind Initiative worked collaboratively with others to create free teaching and learning resources to help children and adults learn how to recognise untrustworthy claims about the effects of treatments. These resources have been translated in more than twenty languages, but much more could be done to support their uptake and wider use.

Educational interventions to improve people's understanding of key concepts in assessing the effects of health interventions: a systematic review

BACKGROUND

Health information is readily accessible but is of variable quality. General knowledge about how to assess whether claims about health interventions are trustworthy is not common, so people's health decisions can be ill-informed, unnecessarily costly and even unsafe. This review aims to identify and evaluate studies of educational interventions designed to improve people's understanding of key concepts for evaluating claims about the effects of health interventions.

METHODS/DESIGN

We searched multiple electronic databases and sources of grey literature. Inclusion criteria included all study types that included a comparison, any participants (except health professionals or health professional students) and educational interventions aimed at improving people's understanding of one or more of the key concepts considered necessary for assessing health intervention claims. Knowledge and/or understanding of concepts or skills relevant to evaluating health information were our primary outcome measures. Secondary outcomes included behaviour, confidence, attitude and satisfaction with the educational interventions. Two authors independently screened search results, assessed study eligibility and risk of bias and extracted data. Results were summarised using descriptive synthesis.

RESULTS

Among 24 eligible studies, 14 were randomised trials and 10 used other study designs. There was heterogeneity across study participants, settings and educational intervention type, content and delivery. The risk of bias was high in at least one domain for all randomised studies. Most studies measured outcomes immediately after the educational intervention, with few measuring later. In most of the comparisons, measures of knowledge and skills were better among those who had received educational interventions than among controls, and some of these differences were statistically significant. The effects on secondary outcomes were inconsistent.

CONCLUSIONS

Educational interventions to improve people's understanding of key concepts for evaluating health intervention claims can improve people's knowledge and skills, at least in the short term. Effects on confidence, attitude and behaviour are uncertain. Many of the studies were at moderate or greater risk of bias. Improvements in study quality, consistency of outcome measures and measures of longer-term effects are needed to improve confidence in estimates of the effects of educational interventions to improve people's understanding of key concepts for evaluating health intervention claims.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016033103.

Establishing a library of resources to help people understand key concepts in assessing treatment claims-The "Critical thinking and Appraisal Resource Library" (CARL)

BACKGROUND

People are frequently confronted with untrustworthy claims about the effects of treatments. Uncritical acceptance of these claims can lead to poor, and sometimes dangerous, treatment decisions, and wasted time and money. Resources to help people learn to think critically about treatment claims are scarce, and they are widely scattered. Furthermore, very few learning-resources have been assessed to see if they improve knowledge and behavior.

OBJECTIVES

Our objectives were to develop the Critical thinking and Appraisal Resource Library (CARL). This library was to be in the form of a database containing learning resources for those who are responsible for encouraging critical thinking about treatment claims, and was to be made available online. We wished to include resources for groups we identified as 'intermediaries' of knowledge, i.e. teachers of schoolchildren, undergraduates and graduates, for example those teaching evidence-based medicine, or those communicating treatment claims to the public. In selecting resources, we wished to draw particular attention to those resources that had been formally evaluated, for example, by the creators of the resource or independent research groups.

METHODS

CARL was populated with learning-resources identified from a variety of sources-two previously developed but unmaintained inventories; systematic reviews of learning-interventions; online and database searches; and recommendations by members of the project group and its advisors. The learning-resources in CARL were organised by 'Key Concepts' needed to judge the trustworthiness of treatment claims, and were made available online by the James Lind Initiative in Testing Treatments interactive (TTi) English (www.testingtreatments.org/category/learning-resources).TTi English also incorporated the database of Key Concepts and the Claim Evaluation Tools developed through the Informed Healthcare Choices (IHC) project (informedhealthchoices.org).

RESULTS

We have created a database of resources called CARL, which currently contains over 500 open-access learning-resources in a variety of formats: text, audio, video, webpages, cartoons, and lesson materials. These are aimed primarily at 'Intermediaries', that is, 'teachers', 'communicators', 'advisors', 'researchers', as well as for independent 'learners'. The resources included in CARL are currently accessible at www.testingtreatments.org/category/learning-resources.

CONCLUSIONS

We hope that ready access to CARL will help to promote the critical thinking about treatment claims, needed to help improve healthcare choices.

Does the use of the Informed Healthcare Choices (IHC) primary school resources improve the ability of grade-5 children in Uganda to assess the trustworthiness of claims about the effects of treatments: protocol for a cluster-randomised trial

BACKGROUND

The ability to appraise claims about the benefits and harms of treatments is crucial for informed health care decision-making. This research aims to enable children in East African primary schools (the clusters) to acquire and retain skills that can help them make informed health care choices by improving their ability to obtain, process and understand health information. The trial will evaluate (at the individual participant level) whether specially designed learning resources can teach children some of the key concepts relevant to appraising claims about the benefits and harms of health care interventions (treatments).

METHODS

This is a two-arm, cluster-randomised trial with stratified random allocation. We will recruit 120 primary schools (the clusters) between April and May 2016 in the central region of Uganda. We will stratify participating schools by geographical setting (rural, semi-urban, or urban) and ownership (public or private). The Informed Healthcare Choices (IHC) primary school resources consist of a textbook and a teachers' guide. Each of the students in the intervention arm will receive a textbook and attend nine lessons delivered by their teachers during a school term, with each lesson lasting 80 min. The lessons cover 12 key concepts that are relevant to assessing claims about treatments and making informed health care choices. The second arm will carry on with the current primary school curriculum. We have designed the Claim Evaluation Tools to measure people's ability to apply key concepts related to assessing claims about the effects of treatments and making informed health care choices. The Claim Evaluation Tools use multiple choice questions addressing each of the 12 concepts covered by the IHC school resources. Using the Claim Evaluation Tools we will measure two primary outcomes: (1) the proportion of children who 'pass', based on an absolute standard and (2) their average scores.

DISCUSSION

As far as we are aware this is the first randomised trial to assess whether key concepts needed to judge claims about the effects of treatment can be taught to primary school children. Whatever the results, they will be relevant to learning how to promote critical thinking about treatment claims. Trial status: the recruitment of study participants was ongoing at the time of manuscript submission.

TRIAL REGISTRATION

Pan African Clinical Trial Registry, trial identifier: PACTR201606001679337 . Registered on 13 June 2016.

A qualitative study exploring high school students' understanding of, and attitudes towards, health information and claims

Background

Exposure to health claims, particularly in the media and social media, is pervasive, and the information conveyed is often inaccurate, incomplete or misleading. Some young people of high school ages are already making decisions about using readily available health interventions (such as sports drinks and beauty products).Although previous research has assessed adults’ understanding of health claims, no research has examined this issue in young adults who are attending high school.

Objective

To explore high school students’ understanding of, and attitudes towards, concepts relevant to assessing health information and claims.

Design

A qualitative study involving semi‐structured interviews with 27 Australian high school students. Responses were recorded, transcribed and a thematic analysis performed. Three themes emerged as follows: (i) Variability in sources of health information and claims, and general understanding of their creation and accuracy of content, (ii) The use of substitute indicators to assess health information and claims and make judgements about their trustworthiness, (iii) Uncertainty about, and literal interpretation of, the language of health claims. Despite general scepticism of health claims and admitted uncertainty of research terminology, many students were generally convinced. Students had poor understanding about how health claims are generated and tended to rely on substitute indicators, such as endorsements, when evaluating the believability of claims.

Conclusion

School students’ lack of awareness of basic health research processes and methods of assessing the accuracy of health information and claims makes them vulnerable to distorted and misleading health information. This restricts their ability to make informed health decisions – a skill that increases in importance as they become adults.

Interventions and assessment tools addressing key concepts people need to know to appraise claims about treatment effects: a systematic mapping review

BACKGROUND

People's ability to appraise claims about treatment effects is crucial for informed decision-making. Our objective was to systematically map this area of research in order to (a) provide an overview of interventions targeting key concepts that people need to understand to assess treatment claims and (b) to identify assessment tools used to evaluate people's understanding of these concepts. The findings of this review provide a starting point for decisions about which key concepts to address when developing new interventions, and which assessment tools should be considered.

METHODS

We conducted a systematic mapping review of interventions and assessment tools addressing key concepts important for people to be able to assess treatment claims. A systematic literature search was done by a reserach librarian in relevant databases. Judgement about inclusion of studies and data collection was done by at least two researchers. We included all quantitative study designs targeting one or more of the key concepts, and targeting patients, healthy members of the public, and health professionals. The studies were divided into four categories: risk communication and decision aids, evidence-based medicine and critical appraisal, understanding of controlled trials, and science education. Findings were summarised descriptively.

RESULTS

We included 415 studies, of which the interventions and assessment tools we identified included only a handful of the key concepts. The most common key concepts in interventions were "Treatments usually have beneficial and harmful effects," "Treatment comparisons should be fair," "Compare like with like," and "Single studies can be misleading." A variety of assessment tools were identified, but only four assessment tools included 10 or more key concepts.

CONCLUSIONS

There is great potential for developing learning and assessment tools targeting key concepts that people need to understand to assess claims about treatment effects. There is currently no instrument covering assessment of all these key concepts.

Effects of School-Based Educational Interventions for Enhancing Adolescents Abilities in Critical Appraisal of Health Claims: A Systematic Review

Background and Objective

Adolescents are frequent media users who access health claims from various sources. The plethora of conflicting, pseudo-scientific, and often misleading health claims in popular media makes critical appraisal of health claims an essential ability. Schools play an important role in educating youth to critically appraise health claims. The objective of this systematic review was to evaluate the effects of school-based educational interventions for enhancing adolescents’ abilities in critically appraising health claims.

Methods

We searched MEDLINE, Embase, PsycINFO, AMED, Cinahl, Teachers Reference Centre, LISTA, ERIC, Sociological Abstracts, Social Services Abstracts, The Cochrane Library, Science Citation Index Expanded, Social Sciences Citation Index, and sources of grey literature. Studies that evaluated school-based educational interventions to improve adolescents’ critical appraisal ability for health claims through advancing the students’ knowledge about science were included. Eligible study designs were randomised and non-randomised controlled trials, and interrupted time series. Two authors independently selected studies, extracted data, and assessed risk of bias in included studies. Due to heterogeneity in interventions and inadequate reporting of results, we performed a descriptive synthesis of studies. We used GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) to assess the certainty of the evidence.

Results

Eight studies were included: two compared different teaching modalities, while the others compared educational interventions to instruction as usual. Studies mostly reported positive short-term effects on critical appraisal-related knowledge and skills in favour of the educational interventions. However, the certainty of the evidence for all comparisons and outcomes was very low.

Conclusion

Educational interventions in schools may have beneficial short-term effects on knowledge and skills relevant to the critical appraisal of health claims. The small number of studies, their heterogeneity, and the predominantly high risk of bias inhibit any firm conclusions about their effects. None of the studies evaluated any long-term effects of interventions. Future intervention studies should adhere to high methodological standards, target a wider variety of school-based settings, and include a process evaluation.

Systematic Review Registration

PROSPERO no. CRD42015017936.