Implementation research training for learners in low- and middle-income countries: evaluating behaviour change after participating in a massive open online course

Strengthening healthcare delivery in Rwanda: Implementation science training for reproductive health researchers

BACKGROUND

Addressing the gap between research and practice is crucial for enhancing reproductive healthcare outcomes. In Rwanda and other low- and middle-income countries, bolstering health researchers' implementation science (IS) capacity is essential. We present a pre-post-intervention study assessing the influence of an intensive IS training program on Rwandan reproductive health researchers' perceived IS knowledge and self-efficacy in applying IS in their own research.

METHODS

To introduce IS principles, we held a one-day training for a diverse cohort of 25 sexual and reproductive health researchers in Rwanda. The training encompassed modules on IS concepts, methodologies, and practical applications. Pre- and post-training assessments gauged changes in participants' perceived IS knowledge and self-efficacy in applying IS in their own work.

RESULTS

The study revealed a significant improvement in self-efficacy related to performing IS related tasks. Researchers reported heightened confidence in designing and implementing evidence-based interventions. In terms of perceived knowledge, participants retained what they learned at 4 months. The training fostered a collaborative learning environment, encouraging participants to exchange ideas and experiences.

CONCLUSION

Targeted training in IS appears to enhance reproductive health researchers' capacity to translate research into practice, potentially leading to improved healthcare outcomes in Rwanda. Moving forward, we advocate for the Ministry of Health to establish structures for IS research agenda-setting, particularly for sexual and reproductive health and rights. Ideally, universities, health systems, and research institutions will incorporate IS capacity strengthening into their routine activities. Ongoing training is crucial to reinforce and expand IS knowledge. Our findings are expected to inform future interventions and guide policy development.

Does diversity beget diversity? A scientometric analysis of over 150,000 studies and 49,000 authors published in high-impact medical journals between 2007 and 2022

Background

Health research that significantly impacts global clinical practice and policy is often published in high-impact factor (IF) medical journals. These outlets play a pivotal role in the worldwide dissemination of novel medical knowledge. However, researchers identifying as women and those affiliated with institutions in low- and middle-income countries (LMIC) have been largely underrepresented in high-IF journals across multiple fields of medicine. To evaluate disparities in gender and geographical representation among authors who have published in any of five top general medical journals, we conducted scientometric analyses using a large-scale dataset extracted from the New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), The British Medical Journal (BMJ), The Lancet, and Nature Medicine.

Methods

Author metadata from all articles published in the selected journals between 2007 and 2022 were collected using the DimensionsAI platform. The Genderize.io API was then utilized to infer each author's likely gender based on their extracted first name. The World Bank country classification was used to map countries associated with researcher affiliations to the LMIC or the high-income country (HIC) category. We characterized the overall gender and country income category representation across the medical journals. In addition, we computed article-level diversity metrics and contrasted their distributions across the journals.

Findings

We studied 151,536 authors across 49,764 articles published in five top medical journals, over a long period spanning 15 years. On average, approximately one-third (33.1%) of the authors of a given paper were inferred to be women; this result was consistent across the journals we studied. Further, 86.6% of the teams were exclusively composed of HIC authors; in contrast, only 3.9% were exclusively composed of LMIC authors. The probability of serving as the first or last author was significantly higher if the author was inferred to be a man (18.1% vs 16.8%, P < .01) or was affiliated with an institution in a HIC (16.9% vs 15.5%, P < .01). Our primary finding reveals that having a diverse team promotes further diversity, within the same dimension (i.e., gender or geography) and across dimensions. Notably, papers with at least one woman among the authors were more likely to also involve at least two LMIC authors (11.7% versus 10.4% in baseline, P < .001; based on inferred gender); conversely, papers with at least one LMIC author were more likely to also involve at least two women (49.4% versus 37.6%, P < .001; based on inferred gender).

Conclusion

We provide a scientometric framework to assess authorship diversity. Our research suggests that the inclusiveness of high-impact medical journals is limited in terms of both gender and geography. We advocate for medical journals to adopt policies and practices that promote greater diversity and collaborative research. In addition, our findings offer a first step towards understanding the composition of teams conducting medical research globally and an opportunity for individual authors to reflect on their own collaborative research practices and possibilities to cultivate more diverse partnerships in their work.

MOOC-based blended learning for knowledge translation capacity-building: A qualitative evaluative study

This qualitative study investigated the effectiveness of blended learning using MOOCs (massive open online courses) for capacity-building in knowledge translation (KT). The evaluation followed Kirkpatrick's updated model. A total of 23 semi-directed interviews were conducted with participants working at a research centre in Côte d'Ivoire, with a first wave of interviews immediately post-training and a second wave after five months. Results showed that the training met learners' needs, with both the content and teaching format being deemed appropriate. Learners reacted positively to face-to-face activities and affirmed the importance of coaching for putting learning into practice. Specific KT skills and principles appeared to have been acquired, such as a procedure for structuring the KT process and improved skills for communicating and presenting scientific knowledge. Five months after the training, encouraging changes were reported, but the sustainability of the new KT practices remained uncertain. KT capacity-building initiatives in low- and middle-income countries struggle to meet demand. Little is known about effective KT training in that context, and even less in non-anglophone countries. The study presented here contributes to the understanding of success factors from the learners' standpoint.

Training in eight low-and middle-income countries: lessons learned from a pilot study using the WHO-TDR dissemination and implementation massive open online course

Introduction

Non-communicable diseases (NCDs) are a leading cause of morbidity and mortality in low-and middle- income countries (LMICs). Despite this, a lack of funding, training and mentorship for NCD investigators in LMICs exists. In an effort to gain knowledge and skills to address these gaps, participants from the Global Research on Implementation and Translation Science (GRIT), a consortium of studies in eight LMICs and their networks, attended the dissemination and implementation (D&I) massive open online course (MOOC) developed by the Special Programme for Research and Training in Tropical Diseases at the World Health Organization to strengthen D&I capacity building. Here, we report on the pilot of this MOOC, which was implemented during the SARS COVID-19 pandemic from April- November 2020.

Methods

Participants completed pre-and post-training questionnaires to assess self-reported D&I competencies, general research skills, and research mentor access and quality. D&I competencies were measured by use of a scale developed for a US-based training program, with change in competency scores assessed by paired t test. We used univariate statistics to analyze the data for all other outcomes.

Results

Of the 247 participants enrolled, 32 (13%) completed all course requirements, 21 (9%) completed the pre-and post-surveys and are included in the analysis. D&I competency scores suggest improvement for those who had complete pre- and post-assessments. Trainee's average score on the full competency scale improved 1.45 points (0-5 scale) from pre- to post-test; all four subscales also showed evidence of improvements. There were small but not significant increases in competencies for grant writing, proposal/ manuscript writing and presentations from pre- to post-test assessment. 40% of trainees reported access to a research mentor and 12% reported access to a D&I specific mentor. Participants reported barriers (e.g., unstable internet access and challenges due to COVID-19) and facilitators (e.g., topical interests, collaboration with colleagues) to completing the MOOC.

Conclusions

Although COVID-19 affected program usage and completion, the MOOC was feasible. We also had signals of effectiveness, meaning among LMIC participants completing the course, there was improvement in self-report D&I competency scores. Recommendations for future D&I trainings in LMICs include (1) adding more topic specific modules (i.e., NCD research, general research skills) for scalability; (2) fostering more collaboration with participants across LMICs; and (3) establishing partnerships with D&I mentors for course participants.

A perspective of massive open online courses (MOOCs) and public health

Massive open online courses (MOOCs) have emerged as an innovative educational technology relevant to and affecting higher education, professional development, and lifelong learning. This paper introduces the principles of MOOCs and reviews the development of these platforms over time. We reflect upon the considerable investment by institutions to develop, deliver and promote such courses, particularly in public health. While open to interpretation, the inherent power, influence, and effectiveness of MOOCs is unquestionable. The potential contribution of MOOCs to public health education is immense, with almost universal reach and access. However, apart from research into participant engagement and knowledge, MOOC-related research and evaluation continue to lag with the rapid proliferation of these courses in response to emerging challenges, as seen with the Coronavirus Disease 19 (COVID-19) pandemic. This makes analyzing the contribution of MOOCs to public health education, health promotion and community programs challenging. This perspective article provides a robust rationale for the necessity of MOOCs and their utility in upskilling health professionals and the general public. It builds on current knowledge to comprehensively explore the factors influencing the development, and application of MOOCs.

Strengthening Capacity for Implementation Research Amid COVID-19 Pandemic: Learnings From the Global Alliance for Chronic Diseases Implementation Science School

Objective: To describe the design, delivery and evaluation of the 3rd Global Alliance for Chronic Diseases (GACD) Implementation Science School (ISS), delivered virtually in 2020 for the first time.

Methods: Since 2014, GACD has supported the delivery of more than ten Implementation Science Workshops for more than 500 international participants. It has also been conducting an annual ISS since 2018. In this study, we described the design, delivery and evaluation of the third ISS.

Results: Forty-six participants from 23 countries in five WHO regions attended the program. The virtual delivery was well-received and found to be efficient in program delivery, networking and for providing collaborative opportunities for trainees from many different countries. The recently developed GACD Implementation Science e-Hub was found to be an instrumental platform to support the program by providing a stand-alone, comprehensive online learning space for knowledge and skill development in implementation research.

Conclusion: The delivery of the virtual GACD ISS proved to be feasible, acceptable and effective and offers greater scalability and sustainability as part of a future strategy for capacity strengthening in implementation research globally.

Evaluating results of the implementation research MOOC using Kirkpatrick's four-level model: a cross-sectional mixed-methods study

INTRODUCTION

An implementation research (IR) massive open online course (MOOC) was developed by the Special Programme for Research and Training in Tropical Diseases, to address the scarcity of training in low-income and middle-income countries in the field of IR. The Kirkpatrick model was used to evaluate the IR MOOC as it is widely applied for evaluation of training and educational programmes. The Kirkpatrick model evaluates training programmes on four levels: reaction, learning, behaviour and results. This paper addresses the impact of the IR MOOC on participants' professional practice.

METHODS

Findings are based on analysis of survey and interview data collected 1.5-2 years after the conclusion of the two 2018 IR MOOC offerings. Of the 3858 MOC participants, 748 responded to the anonymous online survey and seven of these respondents were interviewed. All data are self-reported.

RESULTS

The IR MOOC was successful in enhancing the professional practice of participants and for their organisations. Over 40% reported modifying or implementing changes in their professional work. Respondents reported that participation in the MOOC had improved their ability to conduct IR, enhanced their professional profiles and increased their opportunities for collaboration, research and job promotion. Respondents stated that the MOOC had improved their work quality and productivity, and allowed them to contribute to research, initiate and develop professional collaborations and train others in IR. Respondents reported an increase in applying for grants and scholarships and presenting and publishing work on IR after participating in the MOOC. Barriers applying the knowledge gained from the IR MOOC were experienced, for example, due to a lack of funding and lack of support from colleagues, managers and organisations.

CONCLUSION

Participants perceived that the IR MOOC was successful in its aims of delivering medium-term and long-term results in relation to their own and their organisations' professional outcomes.

Learning From a Massive Open Online COVID-19 Vaccination Training Experience: Survey Study

Background

To prepare key stakeholders for the global COVID-19 vaccination rollout, the World Health Organization and partners developed online vaccination training packages. The online course was launched in December 2020 on the OpenWHO learning platform. This paper presents the findings of an evaluation of this course.

Objective

The aim of this evaluation was to provide insights into user experiences and challenges, measure the impact of the course in terms of knowledge gained, and anticipate potential interest in future online vaccination courses.

Methods

The primary source of data was the anonymized information on course participants, enrollment, completion, and scores from the OpenWHO platform’s statistical data and metric reporting system. Data from the OpenWHO platform were analyzed from the opening of the courses in mid-December 2020 to mid-April 2021. In addition, a learner feedback survey was sent by email to all course participants to complete within a 3-week period (March 19 to April 9, 2021). The survey was designed to determine the perceived strengths and weaknesses of the training packages and to understand barriers to access.

Results

During the study period, 53,593 learners enrolled in the course. Of them, 30,034 (56.0%) completed the course, which is substantially higher than the industry benchmark of 5%-10% for a massive open online course (MOOC). Overall, learners averaged 76.5% on the prequiz compared to 85% on the postquiz, resulting in an increase in average score of 9%. A total of 2019 learners from the course participated in the survey. Nearly 98% (n=1647 fully agree, n=308 somewhat agree; N=1986 survey respondents excluding missing values) of respondents fully or somewhat agreed that they had more confidence in their ability to support COVID-19 vaccination following completion of this course.

Conclusions

The online vaccine training was well received by the target audience, with a measurable impact on knowledge gained. The key benefits of online training were the convenience, self-paced nature, access to downloadable material, and ability to replay material, as well as an increased ability to concentrate. Online training was identified as a timely, cost-effective way of delivering essential training to a large number of people to prepare for the COVID-19 vaccination rollout.

Increasing Knowledge Translation Capacity in Low- and Middle-Income Countries: A Model for Implementation Research Training

Most health professionals lack the training and expertise to translate clinical innovations into actionable programs. Even though some public health expert communities understand that even widely proven solutions need to be adapted to the demands and characteristics of diverse health systems and societies to be successful, such knowledge has yet to inform routine public health approaches and practices. Therefore, it should not be a surprise that the “know-do” gap between clinical innovations and their on-the-ground application that implementation research seeks to bridge is pervasive and enduring, particularly in low- and middle-income countries. This article draws on a study of implementation research training courses to highlight the various competencies needed to translate different types of knowledge into action, many of which are not adequately addressed in existing curricula. We utilized a four-phase modified Delphi methodology that included a review of the academic and grey literature, one-on-one interviews with experts, virtual dialogue series with key stakeholders, and peer review of the synthesized results. The resulting areas in need of further development include the ability of learners to work as part of a multidisciplinary team, engage various stakeholders, and communicate research findings to decision-makers. Based on these insights, it is argued that knowledge translation in implementation research is a multi-faceted, multi-level sensemaking and communication activity that takes place throughout the research and research-to policy-processes.