Establishing Standards to Evaluate the Impact of Integrating Digital Health into Health Systems

Strengthening primary health care in low- and middle-income countries: furthering structural changes in the post-pandemic era

Strengthening primary health care (PHC) is the most cost-effective approach in low- and middle-income countries (LMICs) to achieve sustainable universal health coverage (UHC), protect against health shocks, and promote health and wellbeing for all people. It has been 45 years since PHC was put on the global agenda followed by multiple efforts to advocate for more funding and improved performance of PHC. Yet, investment in PHC is still insufficient and overall performance of PHC systems is weak in LMICs, resulting in increased vulnerability and poor health outcomes especially among marginalized populations. As countries recover from the COVID-19 pandemic, which exposed the fragility of PHC platforms, it is imperative to go beyond advocacy for PHC investments and make systemic changes to strengthen PHC as the foundation of resilient and equitable health systems. We propose five gamechangers to facilitate structural changes for strengthening PHC through a focused health systems approach: (i) integration of client-centered health services at PHC level; (ii) digitization of PHC services; (iii) efficiency gains invested in essential health services; (iv) strengthening management practices for PHC at district and facility levels; and (v) advancing community engagement for PHC. To be successful, the implementation of the gamechangers must be contextualized and focus on achieving sustainable health outcomes, and therefore use implementation approaches that link essential health services to health outcomes. Through this way countries will maximize the possibility of achieving UHC and attaining the ambitious health targets of the Sustainable Development Goals (SDGs) by 2030.

Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review

BACKGROUND

The increasing prevalence of DH applications has outpaced research and practice in digital health (DH) evaluations. Patient experience (PEx) was reported as one of the challenges facing the health system by the World Health Organization. To generate evidence on DH and promote the appropriate integration and use of technologies, a standard evaluation of PEx in DH is required.

OBJECTIVE

This study aims to systematically identify evaluation timing considerations (ie, when to measure), evaluation indicators (ie, what to measure), and evaluation approaches (ie, how to measure) with regard to digital PEx. The overall aim of this study is to generate an evaluation guide for further improving digital PEx evaluation.

METHODS

This is a 2-phase study parallel to our previous study. In phase 1, literature reviews related to PEx in DH were systematically searched from Scopus, PubMed, and Web of Science databases. Two independent raters conducted 2 rounds of paper screening, including title and abstract screening and full-text screening, and assessed the interrater reliability for 20% (round 1: 23/115 and round 2: 12/58) random samples using the Fleiss-Cohen coefficient (round 1: k1=0.88 and round 2: k2=0.80). When reaching interrater reliability (k>0.60), TW conducted the rest of the screening process, leaving any uncertainties for group discussions. Overall, 38% (45/119) of the articles were considered eligible for further thematic analysis. In phase 2, to check if there were any meaningful novel insights that would change our conclusions, we performed an updated literature search in which we collected 294 newly published reviews, of which 102 (34.7%) were identified as eligible articles. We considered them to have no important changes to our original results on the research objectives. Therefore, they were not integrated into the synthesis of this review and were used as supplementary materials.

RESULTS

Our review highlights 5 typical evaluation objectives that serve 5 stakeholder groups separately. We identified a set of key evaluation timing considerations and classified them into 3 categories: intervention maturity stages, timing of the evaluation, and timing of data collection. Information on evaluation indicators of digital PEx was identified and summarized into 3 categories (intervention outputs, patient outcomes, and health care system impact), 9 themes, and 22 subthemes. A set of evaluation theories, common study designs, data collection methods and instruments, and data analysis approaches was captured, which can be used or adapted to evaluate digital PEx.

CONCLUSIONS

Our findings enabled us to generate an evaluation guide to help DH intervention researchers, designers, developers, and program evaluators evaluate digital PEx. Finally, we propose 6 directions for encouraging further digital PEx evaluation research and practice to address the challenge of poor PEx.

Optimising the implementation of digital-supported interventions for the secondary prevention of heart disease: a systematic review using the RE-AIM planning and evaluation framework

BACKGROUND

mHealth technologies are now widely utilised to support the delivery of secondary prevention programs in heart disease. Interventions with mHealth included have shown a similar efficacy and safety to conventional programs with improvements in access and adherence. However, questions remain regarding the successful wider implementation of digital-supported programs. By applying the Reach-Effectiveness-Adoption-Implementation-Maintenance (RE-AIM) framework to a systematic review and meta-analysis, this review aims to evaluate the extent to which these programs report on RE-AIM dimensions and associated indicators.

METHODS

This review extends our previous systematic review and meta-analysis that investigated the effectiveness of digital-supported programs for patients with coronary artery disease. Citation searches were performed on the 27 studies of the systematic review to identify linked publications that reported data for RE-AIM dimensions. All included studies and, where relevant, any additional publications, were coded using an adapted RE-AIM extraction tool. Discrepant codes were discussed amongst reviewers to gain consensus. Data were analysed to assess reporting on indicators related to each of the RE-AIM dimensions, and average overall reporting rates for each dimension were calculated.

RESULTS

Searches found an additional nine publications. Across 36 publications that were linked to the 27 studies, 24 (89%) of the studies were interventions solely delivered at home. The average reporting rates for RE-AIM dimensions were highest for effectiveness (75%) and reach (67%), followed by adoption (54%), implementation (36%) and maintenance (11%). Eleven (46%) studies did not describe relevant characteristics of their participants or of staff involved in the intervention; most studies did not describe unanticipated consequences of the intervention; the ongoing cost of intervention implementation and maintenance; information on intervention fidelity; long-term follow-up outcomes, or program adaptation in other settings.

CONCLUSIONS

Through the application of the RE-AIM framework to a systematic review we found most studies failed to report on key indicators. Failing to report these indicators inhibits the ability to address the enablers and barriers required to achieve optimal intervention implementation in wider settings and populations. Future studies should consider alternative hybrid trial designs to enable reporting of implementation indicators to improve the translation of research evidence into routine practice, with special consideration given to the long-term sustainability of program effects as well as corresponding ongoing costs.

REGISTRATION

PROSPERO-CRD42022343030.

How digital health translational research is prioritised: a qualitative stakeholder-driven approach to decision support evaluation

OBJECTIVES

Digital health is now routinely being applied in clinical care, and with a variety of clinician-facing systems available, healthcare organisations are increasingly required to make decisions about technology implementation and evaluation. However, few studies have examined how digital health research is prioritised, particularly research focused on clinician-facing decision support systems. This study aimed to identify criteria for prioritising digital health research, examine how these differ from criteria for prioritising traditional health research and determine priority decision support use cases for a collaborative implementation research programme.

METHODS

Drawing on an interpretive listening model for priority setting and a stakeholder-driven approach, our prioritisation process involved stakeholder identification, eliciting decision support use case priorities from stakeholders, generating initial use case priorities and finalising preferred use cases based on consultations. In this qualitative study, online focus group session(s) were held with stakeholders, audiorecorded, transcribed and analysed thematically.

RESULTS

Fifteen participants attended the online priority setting sessions. Criteria for prioritising digital health research fell into three themes, namely: public health benefit, health system-level factors and research process and feasibility. We identified criteria unique to digital health research as the availability of suitable governance frameworks, candidate technology's alignment with other technologies in use,and the possibility of data-driven insights from health technology data. The final selected use cases were remote monitoring of patients with pulmonary conditions, sepsis detection and automated breast screening.

CONCLUSION

The criteria for determining digital health research priority areas are more nuanced than that of traditional health condition focused research and can neither be viewed solely through a clinical lens nor technological lens. As digital health research relies heavily on health technology implementation, digital health prioritisation criteria comprised enablers of successful technology implementation. Our prioritisation process could be applied to other settings and collaborative projects where research institutions partner with healthcare delivery organisations.

Personal data governance and privacy in digital reproductive, maternal, newborn, and child health initiatives in Palestine and Jordan: a mapping exercise

Introduction

There is a rapid increase in using digital technology for strengthening delivery of reproductive, maternal, newborn, and child health (RMNCH) services. Although digital health has potentially many benefits, utilizing it without taking into consideration the possible risks related to the security and privacy of patients' data, and consequently their rights, would yield negative consequences for potential beneficiaries. Mitigating these risks requires effective governance, especially in humanitarian and low-resourced settings. The issue of governing digital personal data in RMNCH services has to date been inadequately considered in the context of low-and-middle-income countries (LMICs). This paper aimed to understand the ecosystem of digital technology for RMNCH services in Palestine and Jordan, the levels of maturity of them, and the implementation challenges experienced, particularly concerning data governance and human rights.

Methods

A mapping exercise was conducted to identify digital RMNCH initiatives in Palestine and Jordan and mapping relevant information from identified initiatives. Information was collected from several resources, including relevant available documents and personal communications with stakeholders.

Results

A total of 11 digital health initiatives in Palestine and 9 in Jordan were identified, including: 6 health information systems, 4 registries, 4 health surveillance systems, 3 websites, and 3 mobile-based applications. Most of these initiatives were fully developed and implemented. The initiatives collect patients' personal data, which are managed and controlled by the main owner of the initiative. Privacy policy was not available for many of the initiatives.

Discussion

Digital health is becoming a part of the health system in Palestine and Jordan, and there is an increasing use of digital technology in the field of RMNCH services in both countries, particularly expanding in recent years. This increase, however, is not accompanied by clear regulatory policies especially when it comes to privacy and security of personal data, and how this data is governed. Digital RMNCH initiatives have the potential to promote effective and equitable access to services, but stronger regulatory mechanisms are required to ensure the effective realization of this potential in practice.

Expanding the Evidence on the Safety and Efficiency of 2-Way Text Messaging-Based Telehealth for Voluntary Medical Male Circumcision Follow-up Compared With In-Person Reviews: Randomized Controlled Trial in Rural and Urban South Africa

BACKGROUND

There is a dearth of high-quality evidence from digital health interventions in routine program settings in low- and middle-income countries. We previously conducted a randomized controlled trial (RCT) in Zimbabwe, demonstrating that 2-way texting (2wT) was safe and effective for follow-up after adult voluntary medical male circumcision (VMMC).

OBJECTIVE

To demonstrate the replicability of 2wT, we conducted a larger RCT in both urban and rural VMMC settings in South Africa to determine whether 2wT improves adverse event (AE) ascertainment and, therefore, the quality of follow-up after VMMC while reducing health care workers' workload.

METHODS

A prospective, unblinded, noninferiority RCT was conducted among adult participants who underwent VMMC with cell phones randomized in a 1:1 ratio between 2wT and control (routine care) in North West and Gauteng provinces. The 2wT participants responded to a daily SMS text message with in-person follow-up only if desired or an AE was suspected. The control group was requested to make in-person visits on postoperative days 2 and 7 as per national VMMC guidelines. All participants were asked to return on postoperative day 14 for study-specific review. Safety (cumulative AEs ≤day 14 visit) and workload (number of in-person follow-up visits) were compared. Differences in cumulative AEs were calculated between groups. Noninferiority was prespecified with a margin of -0.25%. The Manning score method was used to calculate 95% CIs.

RESULTS

The study was conducted between June 7, 2021, and February 21, 2022. In total, 1084 men were enrolled (2wT: n=547, 50.5%, control: n=537, 49.5%), with near-equal proportions of rural and urban participants. Cumulative AEs were identified in 2.3% (95% CI 1.3-4.1) of 2wT participants and 1.0% (95% CI 0.4-2.3) of control participants, demonstrating noninferiority (1-sided 95% CI -0.09 to ∞). Among the 2wT participants, 11 AEs (9 moderate and 2 severe) were identified, compared with 5 AEs (all moderate) among the control participants-a nonsignificant difference in AE rates (P=.13). The 2wT participants attended 0.22 visits, and the control participants attended 1.34 visits-a significant reduction in follow-up visit workload (P<.001). The 2wT approach reduced unnecessary postoperative visits by 84.8%. Daily response rates ranged from 86% on day 3 to 74% on day 13. Among the 2wT participants, 94% (514/547) responded to ≥1 daily SMS text messages over 13 days.

CONCLUSIONS

Across rural and urban contexts in South Africa, 2wT was noninferior to routine in-person visits for AE ascertainment, demonstrating 2wT safety. The 2wT approach also significantly reduced the follow-up visit workload, improving efficiency. These results strongly suggest that 2wT provides quality VMMC follow-up and should be adopted at scale. Adaptation of the 2wT telehealth approach to other acute follow-up care contexts could extend these gains beyond VMMC.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04327271; https://www.clinicaltrials.gov/ct2/show/NCT04327271.

Design, Development, and Deployment of an Electronic Immunization Registry: Experiences From Vietnam, Tanzania, and Zambia

INTRODUCTION

There is growing interest among low- and middle-income countries to introduce electronic immunization registries (EIRs) that capture individual-level vaccine data. We compare the design, development, and deployment of EIRs in Vietnam, Tanzania, and Zambia. Through desk review and the authors' firsthand implementation experiences, we describe experiences related to timeline, partnerships, financial costs, and technology and infrastructure.

IMPLEMENTATION EXPERIENCE

The country cases highlight the multi-year timeline required to implement an EIR at scale and the benefit of multiple iterative cycles to pilot and redesign the system before achieving scale. Of the 3 countries, only Vietnam has achieved nationwide scale of the EIR, which took 7 years. In all 3 countries, national government leadership as part of an interdisciplinary team (with experience in leadership, technology, and immunization) was important to ensure country ownership and sustainability. Where international software developers were contracted, partnering with a local software company helped improve responsiveness and sustainability. Across all 3 countries, governments contributed significant in-kind time in addition to investments from donors. Cost savings were observed in Tanzania and Zambia, largely driven by health worker time savings from using the EIR. All 3 case countries underscore the need to understand the local technology and infrastructure context and design the EIR to fit the context. In Vietnam, an initial landscape assessment was conducted to assess technology and infrastructure, whereas in Tanzania and Zambia, user advisory groups provided insights. Existing infrastructure informed EIR design decisions, such as choosing a system with offline functionality in Tanzania and Zambia. All 3 countries have a local partner to provide ongoing technical support.

CONCLUSION

Comparing implementation factors across these cases highlights practical experience and recommendations that complement existing EIR guidance documents. The findings and recommendations from this study can inform other countries considering or in the process of implementing an EIR.

What Factors Explain Low Adoption of Digital Technologies for Health Financing in an Insurance Setting? Novel Evidence From a Quantitative Panel Study on IMIS in Tanzania

BACKGROUND

Digital information management systems for health financing are implemented on the assumption thatdigitalization, among other things, enables strategic purchasing. However, little is known about the extent to which thesesystems are adopted as planned to achieve desired results. This study assesses the levels of, and the factors associated withthe adoption of the Insurance Management Information System (IMIS) by healthcare providers in Tanzania.

METHODS

Combining multiple data sources, we estimated IMIS adoption levels for 365 first-line health facilities in2017 by comparing IMIS claim data (verified claims) with the number of expected claims. We defined adoption as abinary outcome capturing underreporting (verified

RESULTS

We found a median (interquartile range [IQR]) difference of 77.8% (32.7-100) between expected and verifiedclaims, showing a consistent pattern of underreporting across districts, regions, and months. Levels of underreportingvaried across regions (ANOVA: F=7.24, P<.001) and districts (ANOVA: F=4.65, P<.001). Logistic regression resultsshowed that higher service volume, share of people insured, and greater distance to district headquarter were associatedwith a higher probability of underreporting.

CONCLUSION

Our study shows that the adoption of IMIS in Tanzania may be sub-optimal and far from policy-makers'expectations, limiting its capacity to provide the necessary information to enhance strategic purchasing in the healthsector. Countries and agencies adopting digital interventions such as openIMIS to foster health financing reform areadvised to closely track their implementation efforts to make sure the data they rely on is accurate. Further, our studysuggests organizational and infrastructural barriers beyond the software itself hamper effective adoption.

Collapse

Key Words

• Adoption
• Digital Health Intervention
• Health Financing
• Health Insurance
• Strategic Purchasing
• Tanzania

Collapse

MESH Headings

• Humans
• Tanzania
• Healthcare Financing
• Digital Technology
• Insurance, Health
• Universal Health Insurance

Collapse

Grants Collapse

Affiliation(s)

Leon Schuetze Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Heidelberg, Germany
Siddharth Srivastava Swiss Tropical and Public Health Institute (Swiss TPH), Basel, Switzerland University of Basel, Basel, Switzerland
Naasegnibe Kuunibe Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Heidelberg, Germany Faculty of Integrated Development Studies, University for Development Studies, Wa, Ghana
Elizeus Josephat Rwezaula Health Promotion and System Strengthening Project (HPSS), Dodoma, Tanzania
Abdallah Missenye Kongwa District Council, Dodoma, Tanzania
Manfred Stoermer Swiss Tropical and Public Health Institute (Swiss TPH), Basel, Switzerland University of Basel, Basel, Switzerland
Manuela De Allegri Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Heidelberg, Germany

Collapse

Factors Affecting the Successful Implementation of a Digital Intervention for Health Financing in a Low-Resource Setting at Scale: Semistructured Interview Study With Health Care Workers and Management Staff

BACKGROUND

Digital interventions for health financing, if implemented at scale, have the potential to improve health system performance by reducing transaction costs and improving data-driven decision-making. However, many interventions never reach sustainability, and evidence on success factors for scale is scarce. The Insurance Management Information System (IMIS) is a digital intervention for health financing, designed to manage an insurance scheme and already implemented on a national scale in Tanzania. A previous study found that the IMIS claim function was poorly adopted by health care workers (HCWs), questioning its potential to enable strategic purchasing and succeed at scale.

OBJECTIVE

This study aimed to understand why the adoption of the IMIS claim function by HCWs remained low in Tanzania and to assess implications for use at scale.

METHODS

We conducted 21 semistructured interviews with HCWs and management staff in 4 districts where IMIS was first implemented. We sampled respondents by using a maximum variation strategy. We used the framework method for data analysis, applying a combination of inductive and deductive coding to organize codes in a socioecological model. Finally, we related emerging themes to a framework for digital health interventions for scale.

RESULTS

Respondents appreciated IMIS's intrinsic software characteristics and technical factors and acknowledged IMIS as a valuable tool to simplify claim management. Human factors, extrinsic ecosystem, and health care ecosystem were considered as barriers to widespread adoption.

CONCLUSIONS

Digital interventions for health financing, such as IMIS, may have the potential for scale if careful consideration is given to the environment in which they are placed. Without a sustainable health financing environment, sufficient infrastructure, and human capacity, they cannot unfold their full potential to improve health financing functions and ultimately contribute to universal health coverage.

Feasibility, usability and acceptability of a novel digital hybrid-system for reporting of routine maternal health information in Southern Tanzania: A mixed-methods study

Health information systems are important for health planning and progress monitoring. Still, data from health facilities are often of limited quality in Low-and-Middle-Income Countries. Quality deficits are partially rooted in the fact that paper-based documentation is still the norm at facility level, leading to mistakes in summarizing and manual copying. Digitization of data at facility level would allow automatization of these procedural steps. Here we aimed to evaluate the feasibility, usability and acceptability of a scanning innovation called Smart Paper Technology for digital data processing. We used a mixed-methods design to understand users' engagement with Smart Paper Technology and identify potential positive and negative effects of this innovation in three health facilities in Southern Tanzania. Eight focus group discussions and 11 in-depth interviews with users were conducted. We quantified time used by health care providers for documentation and patient care using time-motion methods. Thematic analysis was used to analyze qualitative data. Descriptive statistics and multivariable linear models were generated to compare the difference before and after introduction and adjust for confounders. Health care providers and health care managers appreciated the forms' simple design features and perceived Smart Paper Technology as time-saving and easy to use. The time-motion study with 273.3 and 224.0 hours of observations before and after introduction of Smart Paper Technology, respectively, confirmed that working time spent on documentation did not increase (27.0% at baseline and 26.4% post-introduction; adjusted p = 0.763). Time spent on patient care was not negatively impacted (26.9% at baseline and 37.1% at post-intervention; adjusted p = 0.001). Health care providers described positive effects on their accountability for data and service provision relating to the fact that individually signed forms were filled. Health care providers perceived Smart Paper Technology as feasible, easy to integrate and acceptable in their setting, particularly as it did not add time to documentation.

Digital Health Technologies Applied by the Pharmaceutical Industry to Improve Access to Noncommunicable Disease Care in Low- and Middle-Income Countries

BACKGROUND

There is limited research on how digital health technologies (DHTs) are used to promote access to care for patients with noncommunicable diseases (NCDs), particularly in low- and middle-income countries (LMICs). We describe the use of DHTs in pharmaceutical industry-led access programs aimed at improving access to NCD care in LMICs.

METHODS

The Access Observatory is the largest publicly available repository containing detailed information about pharmaceutical industry-led access programs targeting NCDs. The repository includes 101 access program reports submitted by 19 pharmaceutical companies. From each report, we extracted data relating to geographic location, disease area, beneficiary population, use of DHTs, partnerships, strategies, and activities. Data were analyzed descriptively using SAS Statistical Software and categorized according to the World Health Organization Digital Health Classification Framework.

RESULTS

A total of 43 access programs (42.6%) included DHTs. The majority of programs using DHTs were clustered across sub-Saharan Africa (72.1%) and targeted cancer (60.5%) followed by metabolic disorders (39.5%). The applied DHTs mostly related to program strategies on health service strengthening (74.4%) and community awareness (41.9%) and were largely directed toward health providers, followed by data services and clients. Only a few DHTs were used for health system management. To promote access, most DHTs focused on improving data collection, management, and use (51.1%); building health provider capacity through training (37.2%); and providing targeted patient information (34.8%).

CONCLUSION

The range of DHTs applied by the pharmaceutical industry offers opportunities for more effective access to NCD care. Transparent reporting on DHT use and its contributions to access programs' achievements may reduce duplicative and redundant efforts and provide learnings for private and public stakeholders that may contribute to greater access to NCD care in LMICs.

Changes in on-time vaccination following the introduction of an electronic immunization registry, Tanzania 2016-2018: interrupted time-series analysis

BACKGROUND

Digital health interventions (DHI) have the potential to improve the management and utilization of health information to optimize health care worker performance and provision of care. Despite the proliferation of DHI projects in low-and middle-income countries, few have been evaluated in an effort to understand their impact on health systems and health-related outcomes. Although more evidence is needed on their impact and effectiveness, the use of DHIs among immunization programs has become more widespread and shows promise for improving vaccination uptake and adherence to immunization schedules.

METHODS

Our aim was to assess the impact of an electronic immunization registry (EIR) using an interrupted time-series analysis to analyze the effect on proportion of on-time vaccinations following introduction of an EIR in Tanzania. We hypothesized that the introduction of the EIR would lead to statistically significant changes in vaccination timeliness at 3, 6, and > 6 months post-introduction.

RESULTS

For our primary analysis, we observed a decrease in the proportion of on-time vaccinations following EIR introduction. In contrast, our sensitivity analysis estimated improvements in timeliness among those children with complete vaccination records. However, we must emphasize caution interpreting these findings as they are likely affected by implementation challenges.

CONCLUSIONS

This study highlights the complexities of using digitized individual-level routine health information system data for evaluation and research purposes. EIRs have the potential to improve vaccination timeliness, but analyses using EIR data can be complicated by data quality issues and inconsistent data entry leading to difficulties interpreting findings.

Mobile-Social Learning for Continuing Professional Development in Low- and Middle-Income Countries: Integrative Review

BACKGROUND

Access to continuing professional development (CPD) for health care workers in low- and middle-income countries (LMICs) is severely limited. Digital technology serves as a promising platform for supporting CPD for health care workers by providing educational content virtually and enabling virtual peer-to-peer and mentor interaction for enhanced learning. Digital strategies for CPD that foster virtual interaction can increase workforce retention and bolster the health workforce in LMICs.

OBJECTIVE

The objective of this integrative review was to evaluate the evidence on which digital platforms were used to provide CPD to health care workers and clinical students in LMICs, which was complemented with virtual peer-to-peer or mentor interaction. We phrased this intersection of virtual learning and virtual interaction as mobile-social learning.

METHODS

A comprehensive database and gray literature search was conducted to identify qualitative, quantitative, and mixed methods studies, along with empirical evidence, that used digital technology to provide CPD and virtual interaction with peers or mentors. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed. Eligible articles were written in English, conducted in an LMIC, and used a mobile device to provide CPD and facilitate virtual peer-to-peer or mentor interaction. Titles, abstracts, and full texts were screened, followed by an assessment of the quality of evidence and an appraisal of the articles. A content analysis was then used to deductively code the data into emerging themes.

RESULTS

A total of 750 articles were identified, and 31 (4.1%) were included in the review. SMS text messaging and mobile instant messaging were the most common methods used to provide continuing education and virtual interaction between peers and mentors (25/31, 81%). Across the included articles, participants had high acceptability for using digital platforms for learning and interaction. Virtual peer interaction and mentorship were found to contribute to positive learning outcomes in most studies (27/31, 87%) through increased knowledge sharing, knowledge gains, improved clinical skills, and improved service delivery. Peer-to-peer and mentor interaction were found to improve social support and reduce feelings of isolation (9/31, 29%). There were several challenges in the implementation and use of digital technology for mobile-social learning, including limited access to resources (eg, internet coverage and stable electricity), flexibility in scheduling to participate in CPD, and sociobehavioral challenges among students.

CONCLUSIONS

The summary suggests that mobile-social learning is a useful modality for curriculum dissemination and skill training and that the interface of mobile and social learning serves as a catalyst for improved learning outcomes coupled with increased social capital.

The Transition to an Entirely Digital Immunization Registry in Ha Noi Province and Son La Province, Vietnam: Readiness Assessment Study

Background

Vietnam is one of the first low- to middle-income countries to develop and implement a national-scale electronic immunization registry. This system was finalized into the National Immunization Information System (NIIS) and scaled up to a national-level system in 2017. As a result, immunization coverage and the timeliness of vaccinations have drastically improved. The time spent on planning and reporting vaccinations has drastically reduced; as a result, vaccination planning and reporting has become more accurate and effective. However, to date, end users have been tasked with managing both the NIIS and paper-based systems in parallel until a formal assessment of the readiness to fully transition to the NIIS is conducted.

Objective

This study aims to evaluate the readiness to move to an entirely digital NIIS in 2 provinces of Vietnam—Ha Noi and Son La.

Methods

All health facilities were surveyed to assess their infrastructure, capacity, and need for human resources. NIIS end users were observed and interviewed to evaluate their NIIS knowledge and skill sets. Data from immunization cards and facility paper-based logbooks were compared with data from the NIIS, and vaccine stocks at selected facilities were tallied and compared with data from the NIIS.

Results

Of the 990 health facilities evaluated, most used the NIIS to enter and track immunizations (987/990, 99.7%) and vaccine stocks (889/990, 90.8%). Most had stable electricity (971/990, 98.1%), at least 1 computer (986/990, 99.6%), and ≥2 trained NIIS end users (825/990, 83.3%). End users reported that the NIIS supported them in managing and reporting immunization data and saving them time (725/767, 94.5%). Although many end users were able to perform basic skills, almost half struggled with performing more complex tasks. Immunization data were compiled from the NIIS and immunization cards (338/378, 89.4%) and paper-based logbooks (254/269, 94.4%). However, only 54.5% (206/378) of immunization IDs matched, 57% (13/23) of Bacillus Calmette-Guérin vaccination records were accurate, and 70% (21/30) of the facilities had consistent physical vaccine stock balances. The feedback received from NIIS end users suggests that more supportive supervision, frequent refresher training for strengthening their skill sets, and detailed standardized guides for improving data quality are needed.

Conclusions

The readiness to transition to a digital system is promising; however, additional resources are required to address the timeliness, completeness, and accuracy of the data.

Overview of Technologies Implemented During the First Wave of the COVID-19 Pandemic: Scoping Review

BACKGROUND

Technologies have been extensively implemented to provide health care services for all types of clinical conditions during the COVID-19 pandemic. While several reviews have been conducted regarding technologies used during the COVID-19 pandemic, they were limited by focusing either on a specific technology (or features) or proposed rather than implemented technologies.

OBJECTIVE

This review aims to provide an overview of technologies, as reported in the literature, implemented during the first wave of the COVID-19 pandemic.

METHODS

We conducted a scoping review using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) Extension for Scoping Reviews. Studies were retrieved by searching 8 electronic databases, checking the reference lists of included studies and relevant reviews (backward reference list checking), and checking studies that cited included studies (forward reference list checking). The search terms were chosen based on the target intervention (ie, technologies) and the target disease (ie, COVID-19). We included English publications that focused on technologies or digital tools implemented during the COVID-19 pandemic to provide health-related services regardless of target health condition, user, or setting. Two reviewers independently assessed the eligibility of studies and extracted data from eligible papers. We used a narrative approach to synthesize extracted data.

RESULTS

Of 7374 retrieved papers, 126 were deemed eligible. Telemedicine was the most common type of technology (107/126, 84.9%) implemented in the first wave of the COVID-19 pandemic, and the most common mode of telemedicine was synchronous (100/108, 92.6%). The most common purpose of the technologies was providing consultation (75/126, 59.5%), followed by following up with patients (45/126, 35.7%), and monitoring their health status (22/126, 17.4%). Zoom (22/126, 17.5%) and WhatsApp (12/126, 9.5%) were the most commonly used videoconferencing and social media platforms, respectively. Both health care professionals and health consumers were the most common target users (103/126, 81.7%). The health condition most frequently targeted was COVID-19 (38/126, 30.2%), followed by any physical health conditions (21/126, 16.7%), and mental health conditions (13/126, 10.3%). Technologies were web-based in 84.1% of the studies (106/126). Technologies could be used through 11 modes, and the most common were mobile apps (86/126, 68.3%), desktop apps (73/126, 57.9%), telephone calls (49/126, 38.9%), and websites (45/126, 35.7%).

CONCLUSIONS

Technologies played a crucial role in mitigating the challenges faced during the COVID-19 pandemic. We did not find papers describing the implementation of other technologies (eg, contact-tracing apps, drones, blockchain) during the first wave. Furthermore, technologies in this review were used for other purposes (eg, drugs and vaccines discovery, social distancing, and immunity passport). Future research on studies on these technologies and purposes is recommended, and further reviews are required to investigate technologies implemented in subsequent waves of the pandemic.

The complexity of scaling up an mHealth intervention: the case of SMS for Life in Tanzania from a health systems integration perspective

BACKGROUND

SMS for Life was one of the earliest large-scale implementations of mHealth innovations worldwide. Its goal was to increase visibility to antimalarial stock-outs through the use of SMS technology. The objective of this case study was to show the multiple innovations that SMS for Life brought to the Tanzanian public health sector and to discuss the challenges of scaling up that led to its discontinuation from a health systems perspective.

METHODS

A qualitative case-study approach was used. This included a literature review, a document review of 61 project documents, a timeline of key events and the collection and analysis of 28 interviews with key stakeholders involved in or affected by the SMS for Life programme. Data collection was informed by the health system building blocks. We then carried out a thematic analysis using the WHO mHealth Assessment and Planning for Scale (MAPS) Toolkit as a framework. This served to identify the key reasons for the discontinuation of the programme.

RESULTS

SMS for Life was reliable at scale and raised awareness of stock-outs with real-time monitoring. However, it was discontinued in 2015 after 4 years of a national rollout. The main reasons identified for the discontinuation were the programme's failure to adapt to the continuous changes in Tanzania's health system, the focus on stock-outs rather than ensuring appropriate stock management, and that it was perceived as costly by policy-makers. Despite its discontinuation, SMS for Life, together with co-existing technologies, triggered the development of the capacity to accommodate and integrate future technologies in the health system.

CONCLUSION

This study shows the importance of engaging appropriate stakeholders from the outset, understanding and designing system-responsive interventions appropriately when scaling up and ensuring value to a broad range of health system actors. These shortcomings are common among digital health solutions and need to be better addressed in future implementations.

Overview of Technologies Implemented During the First Wave of the COVID-19 Pandemic: Scoping Review (Preprint).. [DOI: 10.2196/preprints.29136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Technologies have been extensively implemented to provide health care services for all types of clinical conditions during the COVID-19 pandemic. While several reviews have been conducted regarding technologies used during the COVID-19 pandemic, they were limited by focusing either on a specific technology (or features) or proposed rather than implemented technologies.

OBJECTIVE

This review aims to provide an overview of technologies, as reported in the literature, implemented during the first wave of the COVID-19 pandemic.

METHODS

We conducted a scoping review using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) Extension for Scoping Reviews. Studies were retrieved by searching 8 electronic databases, checking the reference lists of included studies and relevant reviews (backward reference list checking), and checking studies that cited included studies (forward reference list checking). The search terms were chosen based on the target intervention (ie, technologies) and the target disease (ie, COVID-19). We included English publications that focused on technologies or digital tools implemented during the COVID-19 pandemic to provide health-related services regardless of target health condition, user, or setting. Two reviewers independently assessed the eligibility of studies and extracted data from eligible papers. We used a narrative approach to synthesize extracted data.

RESULTS

Of 7374 retrieved papers, 126 were deemed eligible. Telemedicine was the most common type of technology (107/126, 84.9%) implemented in the first wave of the COVID-19 pandemic, and the most common mode of telemedicine was synchronous (100/108, 92.6%). The most common purpose of the technologies was providing consultation (75/126, 59.5%), followed by following up with patients (45/126, 35.7%), and monitoring their health status (22/126, 17.4%). Zoom (22/126, 17.5%) and WhatsApp (12/126, 9.5%) were the most commonly used videoconferencing and social media platforms, respectively. Both health care professionals and health consumers were the most common target users (103/126, 81.7%). The health condition most frequently targeted was COVID-19 (38/126, 30.2%), followed by any physical health conditions (21/126, 16.7%), and mental health conditions (13/126, 10.3%). Technologies were web-based in 84.1% of the studies (106/126). Technologies could be used through 11 modes, and the most common were mobile apps (86/126, 68.3%), desktop apps (73/126, 57.9%), telephone calls (49/126, 38.9%), and websites (45/126, 35.7%).

CONCLUSIONS

Technologies played a crucial role in mitigating the challenges faced during the COVID-19 pandemic. We did not find papers describing the implementation of other technologies (eg, contact-tracing apps, drones, blockchain) during the first wave. Furthermore, technologies in this review were used for other purposes (eg, drugs and vaccines discovery, social distancing, and immunity passport). Future research on studies on these technologies and purposes is recommended, and further reviews are required to investigate technologies implemented in subsequent waves of the pandemic.

mCARE, a digital health intervention package on pregnancy surveillance and care-seeking reminders from 2018 to 2027 in Bangladesh: a model-based cost-effectiveness analysis

OBJECTIVE

We estimated the cost-effectiveness of a digital health intervention package (mCARE) for community health workers, on pregnancy surveillance and care-seeking reminders compared with the existing paper-based status quo, from 2018 to 2027, in Bangladesh.

INTERVENTIONS

The mCARE programme involved digitally enhanced pregnancy surveillance, individually targeted text messages and in-person home-visit to pregnant women for care-seeking reminders for antenatal care, child delivery and postnatal care.

STUDY DESIGN

We developed a model to project population and service coverage increases with annual geographical expansion (from 1 million to 10 million population over 10 years) of the mCARE programme and the status quo.

MAJOR OUTCOMES

For this modelling study, we used Lives Saved Tool to estimate the number of deaths and disability-adjusted life years (DALYs) that would be averted by 2027, if the coverage of health interventions was increased in mCARE programme and the status quo, respectively. Economic costs were captured from a societal perspective using an ingredients approach and expressed in 2018 US dollars. Probabilistic sensitivity analysis was undertaken to account for parameter uncertainties.

RESULTS

We estimated the mCARE programme to avert 3076 deaths by 2027 at an incremental cost of $43 million relative to the status quo, which is translated to $462 per DALY averted. The societal costs were estimated to be $115 million for mCARE programme (48% of which are programme costs, 35% user costs and 17% provider costs). With the continued implementation and geographical scaling-up, the mCARE programme improved its cost-effectiveness from $1152 to $462 per DALY averted from 5 to 10 years.

CONCLUSION

Mobile phone-based pregnancy surveillance systems with individually scheduled text messages and home-visit reminder strategies can be highly cost-effective in Bangladesh. The cost-effectiveness may improve as it promotes facility-based child delivery and achieves greater programme cost efficiency with programme scale and sustainability.

Access to chronic medicines: patients' preferences for a last kilometre medicine delivery service in Cape Town, South Africa

Background

Chronic patients are required to access their chronic medicines on a regular basis, often only to refill their repeat prescriptions. Adherence to chronic medicines is challenging and has stimulated health care providers to devise differentiated service delivery models of care to decentralise chronic medicine distribution to decrease the frequency of medicine collection at health care facilities. One such option includes a last kilometre medicine delivery service. This study investigated chronic patients’ preferences for a last kilometre medicine delivery service model.

Methods

An exploratory non-randomised quantitative study was conducted over 4 weeks at four public sector primary health care facilities in Cape Town, South Africa. Data was collected on a structured questionnaire from chronic patients queuing to receive medication at each facility’s pharmacy waiting area. Patient demographics were noted to align with preferences for chronic medicine service delivery characteristics including; mobile ordering, fee for service and location for delivery. Chi-square test and frequencies were employed to analyse data using SPSS version 23.

Results

A total of 116 patients participated in this study. Most were interested in a medicine delivery service (80.2%) and were willing to use a mobile application to order their medicines (84.5%). Almost all patients (96.8%) preferred that their medicines be delivered to their home. More than three quarters of participants were willing to pay for the service (77.6%). Chi-square test showed that gender, age group, employment status, distance to the health facility and /or average waiting time at the clinic significantly influenced the preference for certain characteristics of the medicine delivery service (p < 0.05).

Conclusion

Most participants were interested in a last kilometre medicine delivery service, especially those older than 45 years, waiting for more than 6 h at the facility, and staying within one kilometre radius of the clinic. More studies are needed to establish the influence of patients’ employment status and the distance to health facility on interest in the medicine delivery service.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12875-021-01392-1.

Mobile consulting as an option for delivering healthcare services in low-resource settings in low- and middle-income countries: A mixed-methods study

OBJECTIVE

Remote or mobile consulting is being promoted to strengthen health systems, deliver universal health coverage and facilitate safe clinical communication during coronavirus disease 2019 and beyond. We explored whether mobile consulting is a viable option for communities with minimal resources in low- and middle-income countries.

METHODS

We reviewed evidence published since 2018 about mobile consulting in low- and middle-income countries and undertook a scoping study (pre-coronavirus disease) in two rural settings (Pakistan and Tanzania) and five urban slums (Kenya, Nigeria and Bangladesh), using policy/document review, secondary analysis of survey data (from the urban sites) and thematic analysis of interviews/workshops with community members, healthcare workers, digital/telecommunications experts, mobile consulting providers, and local and national decision-makers. Project advisory groups guided the study in each country.

RESULTS

We reviewed four empirical studies and seven reviews, analysed data from 5322 urban slum households and engaged with 424 stakeholders in rural and urban sites. Regulatory frameworks are available in each country. Mobile consulting services are operating through provider platforms (n = 5-17) and, at the community level, some direct experience of mobile consulting with healthcare workers using their own phones was reported - for emergencies, advice and care follow-up. Stakeholder willingness was high, provided challenges are addressed in technology, infrastructure, data security, confidentiality, acceptability and health system integration. Mobile consulting can reduce affordability barriers and facilitate care-seeking practices.

CONCLUSIONS

There are indications of readiness for mobile consulting in communities with minimal resources. However, wider system strengthening is needed to bolster referrals, specialist services, laboratories and supply chains to fully realise the continuity of care and responsiveness that mobile consulting services offer, particularly during/beyond coronavirus disease 2019.

Barriers to Creating Scalable Business Models for Digital Health Innovation in Public Systems: Qualitative Case Study

Background

Health systems are increasingly looking toward the private sector to provide digital solutions to address health care demands. Innovation in digital health is largely driven by small- and medium-sized enterprises (SMEs), yet these companies experience significant barriers to entry, especially in public health systems. Complex and fragmented care models, alongside a myriad of relevant stakeholders (eg, purchasers, providers, and producers of health care products), make developing value propositions for digital solutions highly challenging.

Objective

This study aims to identify areas for health system improvement to promote the integration of innovative digital health technologies developed by SMEs.

Methods

This paper qualitatively analyzes a series of case studies to identify health system barriers faced by SMEs developing digital health technologies in Canada and proposed solutions to encourage a more innovative ecosystem. The Women’s College Hospital Institute for Health System Solutions and Virtual Care established a consultation program for SMEs to help them increase their innovation capacity and take their ideas to market. The consultation involved the SME filling out an onboarding form and review of this information by an expert advisory committee using guided considerations, leading to a recommendation report provided to the SME. This paper reports on the characteristics of 25 SMEs who completed the program and qualitatively analyzed their recommendation reports to identify common barriers to digital health innovation.

Results

A total of 2 central themes were identified, each with 3 subthemes. First, a common barrier to system integration was the lack of formal evaluation, with SMEs having limited resources and opportunities to conduct such an evaluation. Second, the health system’s current structure does not create incentives for clinicians to use digital technologies, which threatens the sustainability of SMEs’ business models. SMEs faced significant challenges in engaging users and payers from the public system due to perverse economic incentives. Physicians are compensated by in-person visits, which actively works against the goals of many digital health solutions of keeping patients out of clinics and hospitals.

Conclusions

There is a significant disconnect between the economic incentives that drive clinical behaviors and the use of digital technologies that would benefit patients’ well-being. To encourage the use of digital health technologies, publicly funded health systems need to dedicate funding for the evaluation of digital solutions and streamlined pathways for clinical integration.

Determinants of Scale-up From a Small Pilot to a National Electronic Immunization Registry in Vietnam: Qualitative Evaluation

Background

Digital health innovations can improve health system performance, yet previous experience has shown that many innovations do not advance beyond the pilot stage to achieve scale. Vietnam’s National Immunization Information System (NIIS) began as a series of digital health pilots, first initiated in 2010, and was officially launched nationwide in 2017. The NIIS is one of the few examples of an electronic immunization registry (EIR) at national scale in low- and middle-income countries.

Objective

The aim of this study was to understand the determinants of scale-up of the national EIR in Vietnam.

Methods

This qualitative study explored the facilitators and barriers to national scale-up of the EIR in Vietnam. Qualitative data were collected from October to December 2019 through in-depth key informant interviews and desk review. The mHealth Assessment and Planning for Scale (MAPS) Toolkit guided the development of the study design, interview guides, and analytic framework. MAPS defines the key determinants of success, or the “axes of scale,” to be groundwork, partnerships, financial health, technology and architecture, operations, and monitoring and evaluation.

Results

The partnership and operations axes were critical to the successful scale-up of the EIR in Vietnam, while the groundwork and monitoring and the evaluation axes were considered to be strong contributors in the success of all the other axes. The partnership model leveraged complementary strengths of the technical working group partners: the Ministry of Health General Department of Preventive Medicine, the National Expanded Program on Immunization, Viettel (the mobile network operator), and PATH. The operational approach to introducing the NIIS with lean, iterative, and integrated training and supervision was also a key facilitator to successful scale-up. The financial health, technology and architecture, and operations axes were identified as barriers to successful deployment and scale-up. Key barriers to scale-up included insufficient estimates of operational costs, unanticipated volume of data storage and transmission, lack of a national ID to support interoperability, and operational challenges among end users. Overall, the multiple phases of EIR deployment and scale-up from 2010 to 2017 allowed for continuous learning and improvement that strengthened all the axes and contributed to successful scale-up.

Conclusions

The results highlight the importance of the measured, iterative approach that was taken to gradually expand a series of small pilots to nationwide scale. The findings from this study can be used to inform other countries considering, introducing, or in the process of scaling an EIR or other digital health innovations.

WHO Digital Health Guidelines: a milestone for global health

In 2019, the World Health Organization (WHO) released the first-ever evidence-based guidelines for digital health. The guideline provides nine recommendations on select digital health interventions that involve the use of a mobile phone or device. It also provides information on implementation considerations, quality and certainty of extant evidence, factors related to acceptability and feasibility of the intervention, and gaps in the evidence that can inform future research. Given the pivotal role digital health can play in supporting health systems, seen especially in light of the COVID-19 pandemic, these guidelines can help provide a roadmap for governments and policymakers in introducing and scaling up digital health interventions to support population health outcomes.

Perceptions of factors influencing the introduction and adoption of electronic immunization registries in Tanzania and Zambia: a mixed methods study

Background

As technology has become cheaper and more accessible, health programs are adopting digital health interventions (DHI) to improve the provision of and demand for health services. These interventions are complex and require strong coordination and support across different health system levels and government departments, and they need significant capacities in technology and information to be properly implemented. Electronic immunization registries (EIRs) are types of DHI used to capture, store, access, and share individual-level, longitudinal health information in digitized records. The BID Initiative worked in partnership with the governments of Tanzania and Zambia to introduce an EIR at the sub-national level in both countries within 5 years as part of a multi-component complex intervention package focusing on data use capacity-building.

Methods

We aimed to gather and describe learnings from the BID experience by conducting a framework-based mixed methods study to describe perceptions of factors that influenced scale-up of the EIR. Data were collected through key informant interviews, a desk review, EIRs, and health management information systems. We described how implementation of the EIRs fulfilled domains described in our conceptual framework and used cases to illustrate the relationships and relative influence of domains for scale-up and adoption of the EIR.

Results

We found that there was no single factor that seemed to influence the introduction or sustained adoption of the EIR as many of the factors were interrelated. For EIR introduction, strong strategic engagement among partners was important, while EIR adoption was influenced by adequate staffing at facilities, training, use of data for supervision, internet and electricity connectivity, and community sensitization.

Conclusions

Organizations deploying DHIs in the future should consider how best to adapt their intervention to the existing ecosystem, including human resources and organizational capacity, as well as the changing technological landscape during planning and implementation.

Features, Functionality, and Acceptability of Internet-Based Cognitive Behavioral Therapy for Tinnitus in the United States

Objective Although tinnitus is one of the most commonly reported symptoms in the general population, patients with bothersome tinnitus are challenged by issues related to accessibility of care and intervention options that lack strong evidence to support their use. Therefore, creative ways of delivering evidence-based interventions are necessary. Internet-based cognitive behavioral therapy (ICBT) demonstrates potential as a means of delivering this support but is not currently available in the United States. This article discusses the adaptation of an ICBT intervention, originally used in Sweden, Germany, and the United Kingdom, for delivery in the United States. The aim of this study was to (a) modify the web platform's features to suit a U.S. population, (b) adapt its functionality to comply with regulatory aspects, and (c) evaluate the credibility and acceptability of the ICBT intervention from the perspective of health care professionals and patients with bothersome tinnitus. Materials/Method Initially, the iTerapi ePlatform developed in Sweden was adopted for use in the United States. Functional adaptations followed to ensure that the platform's functional and security features complied with both institutional and governmental regulations and that it was suitable for a U.S. population. Following these adaptations, credibility and acceptance of the materials were evaluated by both health care professionals (n = 11) and patients with bothersome tinnitus (n = 8). Results Software safety and compliance regulatory assessments were met. Health care professionals and patients reported favorable acceptance and satisfaction ratings regarding the content, suitability, presentation, usability, and exercises provided in the ICBT platform. Modifications to the features and functionality of the platform were made according to user feedback. Conclusions Ensuring that the ePlatform employed the appropriate features and functionalities for the intended population was essential to developing the Internet-based interventions. The favorable user evaluations indicated that the intervention materials were appropriate for the tinnitus population in the United States.

An online supported self-management toolkit for relatives of people with psychosis or bipolar experiences: the IMPART multiple case study

Background

Digital health interventions have the potential to improve the delivery of psychoeducation to people with mental health problems and their relatives. Despite substantial investment in the development of digital health interventions, successful implementation into routine clinical practice is rare.

Objectives

Use the implementation of the Relatives’ Education And Coping Toolkit (REACT) for psychosis/bipolar disorder to identify critical factors affecting uptake and use, and develop an implementation plan to support the delivery of REACT.

Design

This was an implementation study using a mixed-methods, theory-driven, multiple case study approach. A study-specific implementation theory for REACT based on normalisation process theory was developed and tested, and iterations of an implementation plan to address the key factors affecting implementation were developed.

Setting

Early-intervention teams in six NHS mental health trusts in England (three in the north and three in the south).

Participants

In total, 281 staff accounts and 159 relatives’ accounts were created, 129 staff and 23 relatives took part in qualitative interviews about their experiences, and 132 relatives provided demographic data, 56 provided baseline data, 21 provided data at 12 weeks’ follow-up and 20 provided data at 24 weeks’ follow-up.

Interventions

REACT is an online supported self-management toolkit, offering 12 evidence-based psychoeducation modules and support via a forum, and a confidential direct messaging service for relatives of people with psychosis or bipolar disorder. The implementation intervention was developed with staff and iteratively adapted to address identified barriers. Adaptations included modifications to the toolkit and how it was delivered by teams.

Main outcome measures

The main outcome was factors affecting implementation of REACT, assessed primarily through in-depth interviews with staff and relatives. We also assessed quantitative measures of delivery (staff accounts and relatives’ invitations), use of REACT (relatives’ logins and time spent on the website) and the impact of REACT [relatives’ distress (General Health Questionnaire-28), and carer well-being and support (Carer Well-being and Support Scale questionnaire)].

Results

Staff and relatives were generally positive about the content of REACT, seeing it as a valuable resource that could help services improve support and meet clinical targets, but only within a comprehensive service that included face-to-face support, and with some additional content. Barriers to implementation included high staff caseloads and difficulties with prioritising supporting relatives; technical difficulties of using REACT; poor interoperability with trust information technology systems and care pathways; lack of access to mobile technology and information technology training; restricted forum populations leading to low levels of use; staff fears of managing risk, online trolling, or replacement by technology; and uncertainty around REACT’s long-term availability. There was no evidence that REACT would reduce staff time supporting relatives (which was already very low), and might increase it by facilitating communication. In all, 281 staff accounts were created, but only 57 staff sent relatives invitations. In total, 355 relatives’ invitations were sent to 310 unique relatives, leading to the creation of 159 relatives’ accounts. The mean number of logins for relatives was 3.78 (standard deviation 4.43), but with wide variation from 0 to 31 (median 2, interquartile range 1–8). The mean total time spent on the website was 40.6 minutes (standard deviation 54.54 minutes), with a range of 0–298 minutes (median 20.1 minutes, interquartile range 4.9–57.5 minutes). There was a pattern of declining mean scores for distress, social dysfunction, depression, anxiety and insomnia, and increases in relatives’ well-being and eHealth literacy, but no changes were statistically significant.

Conclusions

Digital health interventions, such as REACT, should be iteratively developed, evaluated, adapted and implemented, with staff and service user input, as part of a long-term strategy to develop integrated technology-enabled services. Implementation strategies must instil a sense of ownership for staff and ensure that they have adequate training, risk protocols and resources to deliver the technology. Cost-effectiveness and impact on workload and inequalities in accessing health care need further testing, along with the generalisability of our findings to other digital health interventions.

Limitations

REACT was offered by the same team running the IMPlementation of A Relatives’ Toolkit (IMPART) study, and was perceived by staff and relatives as a time-limited research study rather than ongoing clinical service, which affected engagement. Access to observational data was limited.

Trial registration

Current Controlled Trials ISRCTN16267685.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 37. See the NIHR Journals Library website for further project information.

eHealth Interventions for Solid Organ Transplant Recipients: A Systematic Review and Meta-analysis of Randomized Controlled Trials

BACKGROUND

Lack of support for self-management may contribute to adverse health outcomes. eHealth has the potential to support self-management, but evidence in solid organ transplantation remains unclear. This review aims to evaluate the benefits and harms of eHealth interventions to support self-management in solid organ transplant recipients.

METHODS

We searched Cochrane Central Register of Controlled Trials, MEDLINE, and Embase databases for randomized trials of eHealth interventions in solid organ transplant recipients. We calculated the risk ratios or standardized mean difference of outcomes, and summary estimates were determined using random-effects models. The Cochrane risk of bias tool and Grading of Recommendations, Assessment, Development, and Evaluations were used to assess trial quality.

RESULTS

Twenty-one trials from 6 countries involving 2114 participants were included. Compared with standard care, eHealth interventions improved medication adherence (risk ratio, 1.34; CI, 1.12-2.56; I = 75%) and self-monitoring behavior (risk ratio, 2.58; CI, 1.56-4.27; I = 0%) up to 12 mo posttransplant. The treatment effects were largely consistent across different subgroups except for intervention functionality and mode of delivery. The effects on other outcomes were uncertain. Nine trials reported harms. The overall risk of bias was considered high or unclear, and the quality of evidence was low to very low for all outcomes.

CONCLUSIONS

eHealth interventions may improve medication adherence and self-monitoring behavior in the short term, but high-quality intervention studies are needed to determine whether eHealth will improve long-term patient-relevant outcomes.

Mobile health interventions in developing countries: A systematic review

This study reviews the quality of evidence reported in mobile health intervention literature in the context of developing countries. A systematic search of renowned databases was conducted to find studies related to mobile health applications published between a period of 2013 and 2018. After a methodological screening, a total of 31 studies were included for data extraction and synthesis. The mobile health Evidence Reporting and Assessment checklist developed by the World Health Organization was then used to evaluate the rigor and completeness in evidence reporting. We report several important and interesting findings. First, there is a very low level of familiarity with the mobile health Evidence Reporting and Assessment checklist among the researchers and mobile health intervention designers from developing countries. Second, most studies do not adequately meet the essential criteria of evidence reporting mentioned in the mobile health Evidence Reporting and Assessment checklist. Third, there is a dearth of application of design science-based methods and theory-based frameworks in developing mobile health interventions. Fourth, most of the mobile health interventions are not ready for interoperability and to be integrated into the existing health information systems. Based on these findings, we recommend for robust and inclusive study plans to deliver highly evidence-based reports by mobile health intervention studies that are conducted in the context of developing countries.

Use of interactive voice response technology to address barriers to fistula care in Nigeria and Uganda

BACKGROUND

The use of digital health technologies has expanded across low-resource settings, including in programs seeking to improve maternal health care seeking and service usage. However, there has been limited use of these technologies for screening and referral within maternal health, and many interventions have relied on SMS tools, which may have limited impact in settings with low female literacy. Digital health technologies have the potential to increase access to care for chronic maternal morbidities, such as obstetric fistula, and for women facing stigma, geographic isolation, and other sociocultural barriers to care seeking. This study documented the process of developing and implementing an innovative fistula screening and referral hotline using interactive voice response (IVR) technology, and described the service usage results and stakeholder perspectives associated with the hotline.

METHODS

The IVR hotline was introduced within the context of a broader Fistula Treatment Barriers Reduction Intervention implemented by the USAID-funded Fistula Care Plus project in Ebonyi and Katsina states in Nigeria and Kalungu district in Uganda. The intervention used three communication pathways to disseminate fistula information and conduct fistula screening: trained community agents, trained primary health care providers, and the IVR hotline paired with mass media messaging. All positively-screened women were eligible to receive vouchers for free transportation to an accredited fistula treatment center. Quantitative and qualitative data on intervention implementation and use across all three communication pathways were gathered during intervention implementation, at baseline, midline, and endline; as well as through ongoing program monitoring. This study presents findings specifically on service usage and stakeholder perspectives related to the IVR hotline.

RESULTS

Over a period of ten to twelve months of implementation, depending on the intervention area, a total of 566 women completed the IVR hotline screening process. Across the areas, 415 (73%) hotline callers screened positive for fistula symptoms. Hotline users and implementation partners reported positive impressions of the hotline, particularly the ability to preserve anonymity in seeking information and referral for fistula symptoms. Challenges to hotline use included limited mobile phone ownership and poor cellular network connectivity, affecting operability by women and community agents.

CONCLUSIONS

Implementation of the fistula screening hotline suggests that IVR-based interventions may be useful in expanding access to health services for stigmatized conditions, particularly in settings where literacy is limited. In the current context, such IVR tools require pairing with community and health system partners to complete referral and support clients. Further program experience and evaluation research is required to understand the options for integrating the IVR hotline or other interventions similarly using mobile technologies for screening and referral into broader digital health platforms that are sustained by national health systems or commercial business models.

Research and Implementation Lessons Learned From a Youth-Targeted Digital Health Randomized Controlled Trial (the ARMADILLO Study)

Background

Evidence is lacking on the efficacy of sexual and reproductive health (SRH) communication interventions for youth (aged 15-24 years), especially from low- and middle-income countries. Therefore, the World Health Organization initiated the Adolescent/Youth Reproductive Mobile Access and Delivery Initiative for Love and Life Outcomes (ARMADILLO) program, a free, menu-based, on-demand text message (SMS, short message service) platform providing validated SRH content developed in collaboration with young people. A randomized controlled trial (RCT) assessing the effect of the ARMADILLO intervention on SRH-related outcomes was implemented in Kwale County, Kenya.

Objective

This paper describes the implementation challenges related to the RCT, observed during enrollment and the intervention period, and their implications for digital health researchers and program implementers.

Methods

This was an open, three-armed RCT. Following completion of a baseline survey, participants were randomized into the ARMADILLO intervention (arm 1), a once-a-week contact SMS text message (arm 2), or usual care (arm 3, no intervention). The intervention period lasted seven weeks, after which participants completed an endline survey.

Results

Two study team decisions had significant implications for the success of the trial’s enrollment and intervention implementation: a hands-off participant recruitment process and a design flaw in an initial language selection menu. As a result, three weeks after recruitment began, 660 participants had been randomized; however, 107 (53%) participants in arm 1 and 136 (62%) in arm 2 were “stuck” at the language menu. The research team called 231 of these nonengaging participants and successfully reached 136 to learn reasons for nonengagement. Thirty-two phone numbers were found to be either not linked to our participants (a wrong number) or not in their primary possession (a shared phone). Among eligible participants, 30 participants indicated that they had assumed the introductory message was a scam or spam. Twenty-seven participants were confused by some aspect of the system. Eleven were apathetic about engaging. Twenty-four nonengagers experienced some sort of technical issue. All participants eventually started their seven-week study period.

Conclusions

The ARMADILLO study’s implementation challenges provide several lessons related to both researching and implementing client-side digital health interventions, including (1) have meticulous phone data collection protocols to reduce wrong numbers, (2) train participants on the digital intervention in efficacy assessments, and (3) recognize that client-side digital health interventions have analog discontinuation challenges. Implementation lessons were (1) determine whether an intervention requires phone ownership or phone access, (2) digital health campaigns need to establish a credible presence in a busy digital space, and (3) interest in a service can be sporadic or fleeting.

Clinical Trial

International Standard Randomized Controlled Trial Number (ISRCTN): 85156148; http://www.isrctn. com/ISRCTN85156148

Digital Health and Health Systems of the Future

Digital strategies have been formally recognized as a critical health systems strengthening strategy to help meet the Sustainable Development Goals and universal health coverage targets. This landscaping collection reviews multiple possible approaches across health system pillars, from digital referrals to decision support systems, identifying key knowledge gaps across these domains and recognizing the growth needed in the field to realize its full potential.