MANTRA: development and localization of a mobile educational health game targeting low literacy players in low and middle income countries

Consolidated guidance for behavioral intervention pilot and feasibility studies

BACKGROUND

In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences.

METHODS

To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of a well-known PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences.

RESULTS

A total of 496 authors were invited to take part in the three-round Delphi survey (round 1, N = 46; round 2, N = 24; round 3, N = 22). A set of twenty considerations, broadly categorized into six themes (intervention design, study design, conduct of trial, implementation of intervention, statistical analysis, and reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading.

CONCLUSION

We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

Assessing the Importance of Content Versus Design for Successful Crowdfunding of Health Education Games: Online Survey Study

Background

Health education games make health-related tasks enjoyable and interactive, thereby encouraging user participation. Entrepreneurs and health educators can leverage online crowdfunding platforms, such as Kickstarter, to transform their innovative ideas into funded projects.

Objective

This research focuses on health education game initiatives on Kickstarter. Through an online user survey, it aims to understand user perceptions and evaluate the significance of 8 distinct components that may influence the success of such crowdfunding initiatives.

Methods

A total of 75 participants evaluated games using 8 dimensions: game rules, learning objectives, narrative, content organization, motivation, interactivity, skill building, and assessment and feedback. The survey data were analyzed using descriptive statistical analysis, exploratory factor analysis, the Wilcoxon-Mann-Whitney test, and multivariate analysis.

Results

Exploratory data analysis showed that, among the 8 dimensions, skill building, content organization, and interactivity were the top-ranking dimensions most closely associated with crowdfunding health education game. The 8 dimensions can be grouped into 3 categories from the exploratory factor analysis: game content-, instruction-, and game design-related components. Further statistical analysis confirmed the correlation between these dimensions with the successful crowdfunding of health education games.

Conclusions

This empirical analysis identified critical factors for game proposal design that can increase the likelihood of securing crowdfunding support.

Exploring Maternal and Infant Health App Development and Effectiveness Research: Scoping Review

BACKGROUND

Globally, high rates of maternal and infant mortality call for interventions during the perinatal period to engage pregnant people as well as their loved ones in care. Mobile health technologies have become ubiquitous in our lives and in health care settings. However, there is a need to further explore their safety and effectiveness to support and improve health outcomes locally and globally.

OBJECTIVE

The aim of this study was to review and synthesize published literature that described the development process or effectiveness evaluations of maternal and infant apps.

METHODS

We applied a methodological framework for scoping reviews as well as the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines; in addition, the systematic review platform Covidence (Veritas Health Innovation Ltd) was used to facilitate the review of included studies. Search terms were developed collaboratively, and health sciences-associated databases were searched for studies conducted between January 1, 2000, and February 4, 2022. We excluded studies about apps that only gathered or tracked data or targeted care providers.

RESULTS

A total of 1027 articles were included for title and abstract screening, of which 87 (8.47%) were chosen for full-text screening. Of these 87 articles, 74 (85%) were excluded with reasons, and 19 (22%) were included. Four articles were added at data extraction from hand searching and 2 others were excluded. Thus, we reviewed and synthesized data from 11 unique studies reported in 21 articles published between 2017 and 2021. The included studies represented 8 different countries. Most of the apps (8/11, 73%) were in English, although apps were also developed in Arabic, Bahasa Indonesia, and Nepali. The articles reviewed revealed the early stage of development of the field of maternal and infant health apps, with modest evidence of app use and achievement of study outcomes. Only 1 (9%) of the 11 apps was endorsed by an independent health care provider society. App development and evaluation processes emerged, and specific app features were identified as vital for well-functioning apps. End-user engagement occurred in some, but not all, parts of app research and development.

CONCLUSIONS

Apps to improve maternal and infant health are being developed and launched in enormous numbers, with many of them not developed with mothers' needs in mind. There are concerns about privacy, safety, and the standardization of current apps as well as a need for professional or institution-specific guidelines or best practices. Despite challenges inherent in currently available apps and their design processes, maternal and infant app technology holds promise for achieving health equity goals and improving maternal and child health outcomes. Finally, we propose recommendations for advancing the knowledge base for maternal and infant apps.

Expert Perspectives on Pilot and Feasibility Studies: A Delphi Study and Consolidation of Considerations for Behavioral Interventions

Background

In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences.

Methods

To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of well-know PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences.

Results

A total of 496 authors were invited to take part in the Delphi survey, 50 (10.1%) of which completed all three rounds, representing 60 (37.3%) of the 161 identified PFS-related guidelines, checklists, frameworks, and recommendations. A set of twenty considerations, broadly categorized into six themes (Intervention Design, Study Design, Conduct of Trial, Implementation of Intervention, Statistical Analysis and Reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading.

Conclusion

We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

Gamified antimicrobial decision support app (GADSA) changes antibiotics prescription behaviour in surgeons in Nigeria: a hospital-based pilot study

AIMS

Surgical Antibiotic Prophylaxis (SAP) in Nigeria is often not evidence based. The aim of this study is to test if the GADSA application can change prescription behaviour of surgeons in Nigeria. In addition, the study aims to identify AMS strategies and policies for the future.

METHODS

The GADSA gamified decision support app uses WHO and Sanford prescribing guidelines to deliver real-time persuasive technology feedback to surgeons through an interactive mentor. The app can advise on whether clinician's decisions align with SAP recommendations and provides the opportunity for clinicians to make adjustments. Twenty surgeons actively participated in a 6-month pilot study in three hospitals in Nigeria. The surgeons determined the risk of infection of a surgical procedure, and the need, type and duration of SAP. The study used a longitudinal approach to test whether the GADSA app significantly changed prescribing behaviour of participating surgeons by analysing the reported prescription decisions within the app.

RESULTS

321 SAP prescriptions were recorded. Concerning the surgical risk decision, 12% of surgeons changed their decision to be in line with guidelines after app feedback (p < 0.001) and 10% of surgeons changed their decision about the need for SAP (p = 0.0035) to align with guidelines. The change in decision making for SAP use in terms of "type" and "duration" to align with guidelines was similar with 6% and 5% respectively (both p-values < 0.001).

CONCLUSION

This study suggests that the GADSA app, with its game based and feedback feature, could significantly change prescribing behaviour at the point of care in an African setting, which could help tackle the global challenge of antibiotic resistance.

Tailoring an evidence-based clinical intervention and training package for the treatment and prevention of comorbid heavy drinking and depression in middle-income country settings: the development of the SCALA toolkit in Latin America

BACKGROUND

Effective interventions exist for heavy drinking and depression but to date there has been limited translation into routine practice in global health systems. This evidence-to-practice gap is particularly evident in low- and middle-income countries. The international SCALA project (Scale-up of Prevention and Management of Alcohol Use Disorders and Comorbid Depression in Latin America) sought to test the impact of multilevel implementation strategies on rates of primary health care-based measurement of alcohol consumption and identification of depression in Colombia, Mexico, and Peru.

OBJECTIVE

To describe the process of development and cultural adaptation of the clinical intervention and training package.

METHODS

We drew on Barrero and Castro's four-stage cultural adaption model: 1) information gathering, 2) preliminary adaption, 3) preliminary adaption tests, and 4) adaption refinement. The Tailored Implementation in Chronic Diseases checklist helped us identify potential factors that could affect implementation, with local stakeholder groups established to support the tailoring process, as per the Institute for Healthcare Improvement's Going to Scale Framework.

RESULTS

In Stage 1, international best practice guidelines for preventing heavy drinking and depression, and intelligence on the local implementation context, were synthesised to provide an outline clinical intervention and training package. In Stage 2, feedback was gathered from local stakeholders and materials refined accordingly. These materials were piloted with local trainers in Stage 3, leading to further refinements including developing additional tools to support delivery in busy primary care settings. Stage 4 comprised further adaptions in response to real-world implementation, a period that coincided with the onset of the COVID-19 pandemic, including translating the intervention and training package for online delivery, and higher priority for depression screening in the clinical pathway.

CONCLUSION

Our experience highlights the importance of meaningful engagement with local communities, alongside the need for continuous tailoring and adaptation, and collaborative decision-making.

OUP accepted manuscript

Background

In Nigeria, the prescription of surgical antibiotic prophylaxis for prevention of surgical site infection tends to be driven by local policy rather than by published guidelines (e.g. WHO and Sanford).

Objectives

To triangulate three datasets and understand key barriers to implementation using a behavioural science framework.

Methods

Surgeons (N = 94) from three teaching hospitals in Nigeria participated in an online survey and in focus group discussions about barriers to implementation. The theoretical domains framework (TDF) was used to structure question items and interview schedules. A subgroup (N = 20) piloted a gamified decision support app over the course of 6 months and reported barriers at the point of care.

Results

Knowledge of guidelines and intention to implement them in practice was high. Key barriers to implementation were related to environmental context and resources and concern over potential consequences of implementing recommendations within the Nigerian context applicable for similar settings in low-to-middle-income countries.

Conclusions

The environmental context and limited resource setting of Nigerian hospitals currently presents a significant barrier to implementation of WHO and Sanford guidelines. Research and data collected from the local context must directly inform the writing of future international guidelines to increase rates of implementation.

Addressing and evaluating health literacy in mHealth: a scoping review

BACKGROUND

Recent surveys have revealed many adults have basic or below basic health literacy, which is linked to medical errors, increased illness, and compromised public health. Health literacy as a concept is multi-faceted extending beyond the individual to include social structures and the context in which health information is being accessed. Delivering health information via mobile devices (mHealth) expands the amount of information available while presenting challenges to ensuring these materials are suitable for a variety of literacy needs. The aims of this study are to discover how health literacy is addressed and evaluated in mHealth app development.

METHODS

A scoping review of 5 peer-reviewed databases was conducted. Eligible articles were written in English, addressed general literacy or mHealth/digital/eHealth literacy, and collected literacy information in order to incorporate literacy into the design and/or modification of an app or collected literacy information to describe the population being studied. The "Health Literacy Online" (HLO) United States (U.S.) government guide was used as a framework.

RESULTS

Thirty-two articles were reviewed. Articles included health literacy recommendations for all HLO categories and some recommendations not aligned with these categories. Most articles addressed health literacy using specific HLO categories though none incorporated every HLO category. The most common categories addressed engagement and testing of mHealth content. Though several studies addressed health literacy through a formal assessment tool, most did not. Evaluation of health literacy in mHealth was end-user focused and did not extensively evaluate content for fit to a variety of individuals with limited health literacy.

CONCLUSIONS

The recommendations seen consistently in our results in conjunction with formal HLO categories can act as beginning steps towards development of a health literacy evaluation tool for mHealth apps themselves. It is clear efforts are being made to reduce barriers to using mHealth for those with literacy deficits, however, it was also clear that this space has room to be more pragmatic in evaluation of mHealth tools for literacy. End user engagement in design and testing is necessary in future mHealth literacy tool development.

Do Women in Nepal Like Playing a Mobile Game? MANTRA: A Mobile Gamified App for Improving Healthcare Seeking Behavior in Rural Nepal

In Low and Middle Income Countries (LMIC), one of the causes of maternal and child mortality is a lack of medical knowledge and consequently the inability to seek timely healthcare. Mobile health (mHealth) technology is gradually becoming a universal intervention platform across the globe due to ubiquity of mobile phones and network coverage. MANTRA is a novel mHealth intervention developed to tackle maternal and child health issues through a serious mobile game app in rural Nepal, which demonstrated a statistically significant knowledge improvement in rural women. This paper explores the perceptions and usability of the MANTRA app amongst rural women and Female Community Health Volunteers (FCHVs) in Nepal. Despite the challenges of a target user group with limited educational levels and low smartphone experience, all participants viewed the MANTRA app with approval and enthusiasm. They were willing to engage further with the mHealth intervention and to share their experience and knowledge with fellow community members. Participants also showed an increase in awareness of danger signs enabling them to make better informed health decisions in the future. FCHVs viewed the app as a validation tool providing and support for greater impact of their efforts in rural Nepal. Growing mobile ownership, network coverage and availability of smartphones along with acceptance of the prototype MANTRA app in rural communities suggest encouraging prospects for mHealth interventions to be incorporated in the national health infrastructure in Nepal and other LMICs.

Following Guidelines for Drug-Resistant Tuberculosis: “Yes, it’s a challenge”

Background

Drug-resistant tuberculosis (DR-TB) is a major contributor to antimicrobial resistance (AMR) globally and is projected to be responsible for up to a quarter of AMR-associated deaths in the future. Management of DR-TB is increasingly decentralised to primary healthcare settings, and simultaneously becoming more complex due to a growing range of treatment options (e.g. novel agents, shorter regimens). This is reflected in the numerous recent updates to international guidelines and as such understanding the barriers and enablers to how healthcare workers access and use guidelines is vital.

Materials and Methods

We used an established psychological framework – the theoretical domains framework (TDF) – to construct and analyse an online survey and focus groups to explore healthcare workers current use of DR-TB guidelines in South Africa. We aimed to identify barriers and enablers with which to direct future attempts at improving guideline use.

Results

There were 19 responses to the online survey and 14 participants in two focus groups. 28% used the most up-to-date national guidelines, 79% accessed guidelines primarily on electronic devices. The TDF domains of ‘Social Influences’ (mean Likert score = 4.3) and ‘Beliefs about Consequences’ (4.2) were key enablers, with healthcare workers encouraged to use guidelines and also recognising the value in doing so. ‘Environmental Resources’ (3.7) and ‘Knowledge’ (3.3) were key barriers with limited, or variable access to guidelines and lack of confidence using them being notable issues. This was most noted for certain subgroups: children, HIV co-infected, pregnant women (2.7).

Discussion

Current use of DR-TB guidelines in South Africa is suboptimal. Planned interventions should focus on overcoming the identified key barriers and might include an increased use of digital tools.

MANTRA: Improving Knowledge of Maternal Health, Neonatal Health, and Geohazards in Women in Rural Nepal Using a Mobile Serious Game

Serious games, conveying educational knowledge rather than merely entertainment, are a rapidly expanding research domain for cutting-edge educational technology. Digital interventions like serious games are great opportunities to overcome challenges in low-and-middle-income countries that limit access to health information, such as social barriers like low-literacy and gender. MANTRA: Increasing maternal and child health resilience before, during and after disasters using mobile technology in Nepal takes on these challenges with a novel digital health intervention; a serious mobile game aimed at vulnerable low-literacy female audiences in rural Nepal. The serious game teaches 28 learning objectives of danger signs in geohazards, maternal, and neonatal health to improve knowledge and self-assessment of common conditions and risks to inform healthcare-seeking behavior. Evaluations consisted of recruiting 35 end users to participate in a pre-test assessment, playing the game, post-test assessment, and focus groups to elicit qualitative feedback. Assessments analyzed knowledge gain in two ways; by learning objective with McNemar tests for each learning objective, and by participant scores with paired t-tests of overall scores and by module. Results of assessments of knowledge gain by learning objective (McNemar tests) indicate participants had sufficient prior knowledge to correctly interpret and respond to 26% of pictograms (coded AA), which is a desirable result although without the possibility of improvement through the intervention. The geohazard module had greatest impact as 16% of responses showed knowledge gain (coded BA). The two most successful learning objectives showing statistically significant positive change were evidence of rockfalls and small cracks in the ground (p = < 0.05). Assessment of knowledge gain by participant scores (paired t-tests) showed the 35 participants averaged a 7.7 point improvement (p < 0.001) in the assessment (28 learning objectives). Average change in knowledge of subdivided module scores (each module normalized to 100 points for comparison) was greatest in the geohazard module (9.5 points, p < 0.001), then maternal health (7.4 points, p = 0.0067), and neonatal health (6.0 points, p = 0.013). This evaluation demonstrated that carefully designed digital health interventions with pictograms co-authored by experts and users can teach complex health and geohazard situations. Significant knowledge gain was demonstrated for several learning objectives while those with non-significant or negative change will be re-designed to effectively convey information.