Teaching critical thinking about health using digital technology in lower secondary schools in Rwanda: A qualitative context analysis

What is the effect of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically about health choices after one-year follow-up? A cluster-randomized trial

AIM

The aim of this study was to evaluate the effects of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically and make informed health choices after 1 year.

METHODS

This was a two-arm cluster-randomized trial conducted in 84 lower secondary schools from 10 districts representing five provinces of Rwanda. We used stratified randomization to allocate schools 1:1 to the intervention or control arm. One class in each intervention school had ten 40-min lessons taught by a trained teacher in addition to the usual curriculum. Control schools followed the usual curriculum. The primary outcome was a passing score (≥ 9 out of 18 questions answered correctly) for students on the Critical Thinking about Health Test completed 1 year after the intervention. We conducted an intention to treat analysis using generalized linear mixed models, accounting for the cluster design using random intercepts.

RESULTS

After 1 year, 35 of 42 teachers (83.3%) and 1181 of 1556 students (75.9%) in the control arm completed the test. In the intervention arm, 35 of 42 teachers (83.3%) and 1238 of 1572 students (78.8%) completed the test. The proportion of students who had a passing score in the intervention arm was 625/1238 (50.5%) compared to 230/1181 (19.5%) in the control arm (adjusted odds ratio 7.6 [95% CI: 4.6-12.6], p < 0.0001). The adjusted difference in the proportion of students with a passing score was 32.2% (95% CI 24.5-39.8%).

CONCLUSIONS

The IHC secondary school intervention was effective after 1 year. However, the size of the effect was smaller than immediately after the intervention (adjusted difference 32.2% vs 37.2%) due to decay in the proportion of students in intervention schools with a passing score (50.5% vs 58.2%).

TRIAL REGISTRATION

Pan African Clinical Trial Registry (PCTR), trial identifier: PACTR202203880375077. Registered on February 15, 2022.

The effects of teaching strategies on learning to think critically in primary and secondary schools: an overview of systematic reviews

Background

We conducted an overview of systematic reviews about the effects of teaching strategies that can be used to teach primary and secondary school students to think critically. Our objective was to inform decisions about what teaching strategies to use in resources that we developed to teach critical thinking about health in secondary schools.

Methods

We mapped characteristics of systematic reviews of teaching strategies and summarised findings from the most relevant reviews to teaching students to think critically about health. We included reviews that assessed the effects of teaching strategies that could potentially be used in primary or secondary schools to teach students to think critically, had a Methods section with explicit selection criteria, reported at least one outcome measure of the ability to undertake one of four basic types of cognitive tasks (memory, procedural, comprehension, or opinion), and were published after 1999.

Results

We included 326 systematic reviews. The reviews evaluated a wide range of teaching strategies for a variety of purposes. Important limitations of the reviews included not considering adverse effects (99% of the reviews), not assessing the risk of bias for included studies (93% of the reviews), and not assessing the credibility of subgroup effects (100% of the reviews). We summarised the findings for 37 teaching strategies that we considered most relevant. The certainty of the evidence of the effects varied from very low to moderate. We used 12 of the strategies in resources that we developed to teach secondary students to think critically about health.

Conclusions

A tremendous amount of work has gone into evaluating the effects of a wide range of teaching strategies. The results of this research can inform decisions about how to teach critical thinking and future research. However, well-designed, up-to-date systematic reviews are still needed for many teaching strategies.

Process Evaluation of Teaching Critical Thinking About Health Using the Informed Health Choices Intervention in Rwanda: A Mixed Methods Study

INTRODUCTION

We evaluated the Informed Health Choices secondary school intervention in a cluster randomized trial in Rwanda. The intervention was effective in helping students to think critically about health. In parallel to the trial, we conducted a process evaluation to assess factors affecting the implementation, impacts, and scale-up of the intervention.

METHODS

We used a mixed methods approach that included quantitative and qualitative methods. We collected quantitative data from teachers to evaluate the teacher training and each lesson. We conducted focus group discussions with students (n=10) and their parents/guardians (n=5). We conducted lesson observations (n=16) and key informant interviews with teachers (n=10) and school administrators (n=10) from intervention schools and policymakers (n=2). We analyzed the quantitative data using descriptive statistics. We used framework analysis and thematic content analysis to analyze the qualitative data.

RESULTS

Teachers noted that the teacher training supported their delivery of the intervention and that they made only small adaptations to fit student, teacher, or contextual needs. Students reported obtaining important skills, including recognizing health claims, understanding the need for research, and "thinking twice" before deciding. Participants saw the design of the intervention, students' and teachers' motivation, and school and home support as key facilitators for the implementation and impact of the intervention. Implementation barriers identified included the content of the lessons not being included in national examinations, competing priorities, and time constraints. Participants identified several factors that could facilitate intervention scale-up, including the need for the skills taught in the lessons and compatibility of the intervention with the national curriculum.

CONCLUSION

We found that it was feasible to implement the intervention in Rwandan secondary schools and that students benefited from the intervention. Scaling up the intervention will likely require addressing the barriers identified in this study.

Effects of the informed health choices secondary school intervention after 1 year: a prospective meta-analysis using individual participant data

BACKGROUND

Critical thinking about health choices is essential to avoid being misled by unreliable information and to use reliable information appropriately. The aim of this prospective meta-analysis was to synthesize the results of 1-year follow-up data from three cluster-randomized trials of an intervention designed to teach lower secondary school students to think critically about health choices. Only one other randomized trial has evaluated a school-based intervention to teach adolescents to think critically about health choices. That trial compared two teaching strategies to teach statistical reasoning. It did not assess long-term learning-retention.

METHODS

We conducted the trials in Kenya, Rwanda, and Uganda. The intervention included providing a 2-3-day teacher training workshop and digital resources for ten lessons. The intervention focused on nine key concepts. We did not intervene in control schools. The primary outcome was a passing score on a test (≥ 9 of 18 multiple-choice questions answered correctly). We performed random effects meta-analyses to estimate the overall intervention effects. We calculated learning retention as the test results in the intervention schools after 1 year relative to just after the intervention, adjusted for chance.

RESULTS

Altogether, 244 schools (11,344 students) took part in the three trials. Follow-up data was collected for 8298 students (73%). The overall odds ratio for the primary outcome after 1 year was 3.6 (95% CI: 1.9-7.1; p = 0.0001) in favor of the intervention, whereas it was 5.5 (95% CI: 3.0-10.2) just after the intervention. This corresponds to 25.6% (95% CI: 21.1-30.0%) more students in the intervention schools passing the test after 1 year versus 33.3% (95% CI: 28.7-37.8%) just after the intervention. Overall, 2273 (52.6%) of 4324 students in intervention schools had a passing score after 1 year compared to 3397 (58.1%) of 5846 students just after the intervention, indicating 88.3% learning retention.

CONCLUSIONS

One year after the intervention, we still found a positive effect on the ability of students to think critically about health choices, but 5.5% fewer students in the intervention schools had a passing score. The certainty of the evidence was also lower due to 27% of students being lost to follow-up.

TRIAL REGISTRATION

The protocol for this prospective meta-analysis was registered with PROSPERO May 31, 2022, ID 336580. The three randomized trials were registered in the Pan African Clinical Trial Registry February 15, 2022, PACTR202203880375077; April 5, 2022, PACTR20220488391731; and April 14, 2022, PACTR202204861458660.

Teaching critical thinking about health information and choices in secondary schools: human-centred design of digital resources

Background

Learning to thinking critically about health information and choices can protect people from unnecessary suffering, harm, and resource waste. Earlier work revealed that children can learn these skills, but printing costs and curricula compatibility remain important barriers to school implementation. We aimed to develop a set of digital learning resources for students to think critically about health that were suitable for use in Kenyan, Rwandan, and Ugandan secondary schools.

Methods

We conducted work in two phases collaborating with teachers, students, schools, and national curriculum development offices using a human-centred design approach. First, we conducted context analyses and an overview of teaching strategies, prioritised content and collected examples. Next, we developed lessons and guidance iteratively, informed by data from user-testing, individual and group interviews, and school pilots.

Results

Final resources include online lesson plans, teachers' guide, and extra resources, with lesson plans in two modes, for use in a classroom equipped with a blackboard/flip-chart and a projector. The resources are accessible offline for use when electricity or Internet is lacking. Teachers preferred the projector mode, as it provided structure and a focal point for class attention. Feedback was largely positive, with teachers and students appreciating the learning and experiencing it as relevant. Four main challenges included time to teach lessons; incorrect comprehension; identifying suitable examples; and technical, logistical, and behavioural challenges with a student-computer mode that we piloted. We resolved challenges by simplifying and combining lessons; increasing opportunities for review and assessment; developing teacher training materials, creating a searchable set of examples; and deactivating the student-computer mode.

Conclusion

Using a human-centred design approach, we created digital resources for teaching secondary school students to think critically about health actions and for training teachers. Be smart about your health resources are open access and can be translated or adapted to other settings.

Key concepts for informed health choices: Where's the evidence?

Background

The Informed Health Choices (IHC) Key Concepts is a framework that provides a basis for developing educational resources and evaluating people's ability to think critically about health actions. We developed the original Key Concepts framework by reviewing texts and checklists for the public, journalists, and health professionals and collecting structured feedback from an international advisory group. We revised the original 2015 framework yearly from 2016 to 2018 based on feedback and experience using the framework. The objectives of this paper are to describe the development of the framework since 2018 and summarise their basis.

Methods

For the 2019 version, we responded to feedback on the 2018 version. For the current 2022 version, in addition to responding to feedback on the 2019 version, we reviewed the evidence base for each of the concepts. Whenever possible, we referenced systematic reviews that provide a basis for a concept. We screened all Cochrane methodology reviews and searched Epistemonikos, PubMed, and Google Scholar for methodology reviews and meta-epidemiological studies.

Results

The original framework included 32 concepts in six groups. The 2019 version and the current 2022 version include 49 concepts in the same three main groups that we have used since 2016. There are now 10 subgroups or higher-level concepts. For each concept, there is an explanation including one or more examples, the basis for the concept, and implications. Over 600 references are cited that support the concepts, and over half of the references are systematic reviews.

Conclusions

There is a large body of evidence that supports the IHC key concepts and we have received few suggestions for changes since 2019.

Effects of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically about health choices: A cluster-randomized trial

AIM

The aim of this trial was to evaluate the effects of the Informed Health Choices intervention on the ability of students in Rwandan to think critically and make Informed Health Choices.

METHODS

We conducted a two-arm cluster-randomized trial in 84 lower secondary schools from 10 districts representing five provinces of Rwanda. We used stratified randomization to allocate schools to the intervention or control. One class in each intervention school had ten 40-min lessons taught by a trained teacher in addition to the usual curriculum. Control schools followed the usual curriculum. The primary outcome was a passing score (≥ 9 out of 18 questions answered correctly) for students on the Critical Thinking about Health Test completed within 2 weeks after the intervention. We conducted an intention-to-treat analysis using generalized linear mixed models, accounting for the cluster design using random intercepts.

RESULTS

Between February 25 and March 29, 2022, we recruited 3,212 participants. We assigned 1,572 students and 42 teachers to the intervention arm and 1,556 students and 42 teachers to the control arm. The proportion of students who passed the test in the intervention arm was 915/1,572 (58.2%) compared to 302/1,556 (19.4%) in the control arm, adjusted odds ratio 10.6 (95% CI: 6.3-17.8), p < 0.0001, adjusted difference 37.2% (95% CI: 29.5%-45.0%).

CONCLUSIONS

The intervention is effective in helping students think critically about health choices. It was possible to improve students' ability to think critically about health in the context of a competence-based curriculum in Rwanda, despite challenging postpandemic conditions.

Effects of the Informed Health Choices secondary school intervention: A prospective meta-analysis

AIM

The aim of this prospective meta-analysis was to synthesize the results of three cluster-randomized trials of an intervention designed to teach lower-secondary school students (age 14-16) to think critically about health choices.

METHODS

We conducted the trials in Kenya, Rwanda, and Uganda. The intervention included a 2- to 3-day teacher training workshop, digital resources, and ten 40-min lessons. The lessons focused on nine key concepts. We did not intervene in control schools. The primary outcome was a passing score on a test (≥9 of 18 multiple-choice questions answered correctly). We performed random effects meta-analyses to estimate the overall adjusted odds ratios. Secondary outcomes included effects of the intervention on teachers.

RESULTS

Altogether, 244 schools (11,344 students) took part in the three trials. The overall adjusted odds ratio was 5.5 (95% CI: 3.0-10.2; p < 0.0001) in favor of the intervention (high certainty evidence). This corresponds to 33% (95% CI: 25-40%) more students in the intervention schools passing the test. Overall, 3397 (58%) of 5846 students in intervention schools had a passing score. The overall adjusted odds ratio for teachers was 13.7(95% CI: 4.6-40.4; p < 0.0001), corresponding to 32% (95% CI: 6%-57%) more teachers in the intervention schools passing the test (moderate certainty evidence). Overall, 118 (97%) of 122 teachers in intervention schools had a passing score.

CONCLUSIONS

The intervention led to a large improvement in the ability of students and teachers to think critically about health choices, but 42% of students in the intervention schools did not achieve a passing score.

Effects of the informed health choices secondary school intervention on the ability of students in Kenya to think critically about health choices: A cluster-randomized trial

AIM

There is an overabundance of claims about the advantages and disadvantages of health interventions. People need to be able to appraise the reliability of these claims. The aim of this two-arm cluster-randomized trial was to evaluate the Informed Health Choices secondary school intervention designed to teach students to assess claims about the effects of health actions and make informed decisions.

METHODS

We conducted the trial among students from 80 secondary schools in five subcounties in Kenya. We used stratified randomization to allocate schools to the intervention or control arm. The intervention included a 2-day teacher training workshop and 10 lessons that addressed nine prioritized key concepts for assessing claims about treatment effects. We did not intervene in the control schools. The primary outcome was the proportion of students with a passing score (≥ 9/18 correct answers) on the Critical Thinking about Health test, which included two multiple-choice questions for each concept.

RESULTS

Between May 11, 2022, and July 8, 2022, we recruited 3362 students and 80 teachers. We allocated 1863 students and 40 teachers to the intervention and 1499 students and 40 teachers to the control arm. In the intervention schools, 1149/1863 (61.7%) of students achieved a passing score compared to 511/1499 (34.1%) in the control schools (odds ratio 3.6 (95% CI 2.5-5.2), p < 0.0001).

CONCLUSIONS

The intervention had a large effect on students' ability to think critically about health interventions. It is possible to integrate the learning of critical thinking about health within Kenya secondary school curriculum.

Factors influencing in-service teachers' technology integration model: Innovative strategies for educational technology

Technology invention and implementation have resulted in long-term educational progress. This study aimed to identify the innovative strategies in determining the factors influencing in-service teachers' technology integration in China, focusing on the information and communication technology (ICT) integration. The sample consists of 685 in-service teachers. Four factors are found to influence in-service teachers' ICT integration: attitudes, self-efficacy, digital competence, and digital tools use. The results of the study utilizing the PLS-SEM approach demonstrated that all four factors have a substantial impact on in-service teachers' ICT integration and interact with each other. Meanwhile, attitudes, digital competence, and digital tools use have a mediating effect in this research. In addition, the study discussed the effects of gender, age, and teaching experience on influencing factors and ICT integration. This research provided beneficial insights for a successful design of instruction with ICT and contributed to innovative strategies for educational technology.

Prioritisation of Informed Health Choices (IHC) key concepts to be included in lower secondary school resources: A consensus study

BACKGROUND

The Informed Health Choices Key Concepts are principles for thinking critically about healthcare claims and deciding what to do. The Key Concepts provide a framework for designing curricula, learning resources, and evaluation tools.

OBJECTIVES

To prioritise which of the 49 Key Concepts to include in resources for lower secondary schools in East Africa.

METHODS

Twelve judges used an iterative process to reach a consensus. The judges were curriculum specialists, teachers, and researchers from Kenya, Uganda, and Rwanda. After familiarising themselves with the concepts, they pilot-tested draft criteria for selecting and ordering the concepts. After agreeing on the criteria, nine judges independently assessed all 49 concepts and reached an initial consensus. We sought feedback on the draft consensus from other stakeholders, including teachers. After considering the feedback, nine judges independently reassessed the prioritised concepts and reached a consensus. The final set of concepts was determined after user-testing prototypes and pilot-testing the resources.

RESULTS

The first panel of judges prioritised 29 concepts. Based on feedback from teachers, students, curriculum specialists, and members of the research team, two concepts were dropped. A second panel of nine judges prioritised 17 of the 27 concepts that emerged from the initial prioritisation and feedback. Based on feedback on prototypes of lessons and pilot-testing a set of 10 lessons, we determined that it was possible to introduce nine concepts in 10 single-period (40-minute) lessons. We included eight of the 17 prioritised concepts and one additional concept.

CONCLUSION

Using an iterative process with explicit criteria, we prioritised nine concepts as a starting point for students to learn to think critically about healthcare claims and choices.

Secondary teachers' competencies and attitude: A mediated multigroup model based on usefulness and enjoyment to examine the differences between key dimensions of STEM teaching practice

This research explores the mediating role of perceived usefulness and enjoyment of science, technology, engineering, and mathematics (STEM) teaching practice between secondary teachers' competencies and attitudes in the formal educational context. Also, the research aimed to examine if the relationships between model constructs differ by STEM teaching practice dimension (e.g., Inquiry-based learning (IBL) and Integration of STEM content (INT)). We synthesized the will, skill, tool model (WST), technology acceptance model (TAM) and flow theory (FLT) to develop a theoretical model predicting teacher attitude under the influence of Competencies, Perceived Usefulness and Perceived Enjoyment. Therefore, a mediated multigroup model with validated data from three hundred Romanian secondary teachers who completed questionnaires related to their competencies, perceived usefulness, enjoyment, and attitude toward STEM teaching practices was used. Two comparative teacher survey studies were carried out: one for IBL and one for INT. There are direct and positive relationships between Competencies and Attitude, Competencies and Enjoyment, Competencies and Usefulness, and Enjoyment and Attitude for both IBL and INT teaching practices. The partial least squares path modeling (PLS-SEM) results showed that the control variables had no significant impact on attitude. This research supports evidence for the belief that teachers' competency is a key predictor of attitude. Precisely, the positive strong direct effect of Competencies on Attitude is similar for IBL (β = 0.49, t = 7.46, p < 0.001; f 2 (Effect size) = 0.29) with for INT teaching practice (β = 0.46, t = 6.46, p < 0.001; f2 (Effect size) = 0.22). Interestingly, this research showed that the perceived Usefulness and Enjoyment partially mediated the association between Competencies and Attitude in both case studies. Understanding the mediating role of perceived usefulness and enjoyment for each STEM practice would help teachers successfully implement STEM education.

Contextualizing critical thinking about health using digital technology in secondary schools in Kenya: a qualitative analysis

BACKGROUND

Good health decisions depend on one's ability to think critically about health claims and make informed health choices. Young people can learn these skills through school-based interventions, but learning resources need to be low-cost and built around lessons that can fit into existing curricula. As a first step to developing and evaluating digital learning resources that are feasible to use in Kenyan secondary schools, we conducted a context analysis to explore interest in critical thinking for health, map where critical thinking about health best fits in the curriculum, explore conditions for introducing new learning resources, and describe the information and communication technology (ICT) infrastructure available for teaching and learning.

METHODS

We employed a qualitative descriptive approach. We interviewed 15 key informants, carried out two focus group discussions, observed ICT conditions in five secondary schools, reviewed seven documents, and conducted an online catalog of ICT infrastructure in all schools (n=250) in Kisumu County. Participants included national curriculum developers, national ICT officers, teachers, and national examiners. We used a framework analysis approach to analyze data and report findings.

FINDINGS

Although critical thinking is a core competence in the curriculum, critical thinking about health is not currently taught in Kenyan secondary schools. Teachers, health officials, and curriculum developers recognized the importance of teaching critical thinking about health in secondary schools. Stakeholders agreed that Informed Health Choices learning resources could be embedded in nine subjects. The National Institute of Curriculum Development regulates resources for learning; the development of new resources requires collaboration and approval from this body. Most schools do not use ICT for teaching, and for those few that do, the use is limited. Implementation of Kenya's ICT policy framework for schools faces several challenges which include inadequate ICT infrastructure, poor internet connectivity, and teachers' lack of training and experience.

CONCLUSION

Teaching critical thinking about health is possible within the current Kenyan lower secondary school curriculum, but the learning resources will need to be designed for inclusion in and across existing subjects. The National ICT Plan and Vision for 2030 provides an opportunity for scale-up and integration of technology in teaching and learning environments, which can enable future use of digital resources in schools. However, given the current ICT condition in schools in the country, digital learning resources should be designed to function with limited ICT infrastructure, unstable Internet access, and for use by teachers with low levels of experience using digital technology.

Learning to think critically about health using digital technology in Ugandan lower secondary schools: A contextual analysis

INTRODUCTION

The world is awash with claims about the effects of health interventions. Many of these claims are untrustworthy because the bases are unreliable. Acting on unreliable claims can lead to waste of resources and poor health outcomes. Yet, most people lack the necessary skills to appraise the reliability of health claims. The Informed Health Choices (IHC) project aims to equip young people in Ugandan lower secondary schools with skills to think critically about health claims and to make good health choices by developing and evaluating digital learning resources. To ensure that we create resources that are suitable for use in Uganda's secondary schools and can be scaled up if found effective, we conducted a context analysis. We aimed to better understand opportunities and barriers related to demand for the resources, how the learning content overlaps with existing curriculum and conditions in secondary schools for accessing and using digital resources, in order to inform resource development.

METHODS

We used a mixed methods approach and collected both qualitative and quantitative data. We conducted document analyses, key informant interviews, focus group discussions, school visits, and a telephone survey regarding information communication and technology (ICT). We used a nominal group technique to obtain consensus on the appropriate number and length of IHC lessons that should be planned in a school term. We developed and used a framework from the objectives to code the transcripts and generated summaries of query reports in Atlas.ti version 7.

FINDINGS

Critical thinking is a key competency in the lower secondary school curriculum. However, the curriculum does not explicitly make provision to teach critical thinking about health, despite a need acknowledged by curriculum developers, teachers and students. Exam oriented teaching and a lack of learning resources are additional important barriers to teaching critical thinking about health. School closures and the subsequent introduction of online learning during the COVID-19 pandemic has accelerated teachers' use of digital equipment and learning resources for teaching. Although the government is committed to improving access to ICT in schools and teachers are open to using ICT, access to digital equipment, unreliable power and internet connections remain important hinderances to use of digital learning resources.

CONCLUSIONS

There is a recognized need for learning resources to teach critical thinking about health in Ugandan lower secondary schools. Digital learning resources should be designed to be usable even in schools with limited access and equipment. Teacher training on use of ICT for teaching is needed.