Correction: iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations

[This corrects the article DOI: 10.2196/46694.].

iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations

BACKGROUND

Implementation of digital health technologies has grown rapidly, but many remain limited to pilot studies due to challenges, such as a lack of evidence or barriers to implementation. Overcoming these challenges requires learning from previous implementations and systematically documenting implementation processes to better understand the real-world impact of a technology and identify effective strategies for future implementation.

OBJECTIVE

A group of global experts, facilitated by the Geneva Digital Health Hub, developed the Guidelines and Checklist for the Reporting on Digital Health Implementations (iCHECK-DH, pronounced "I checked") to improve the completeness of reporting on digital health implementations.

METHODS

A guideline development group was convened to define key considerations and criteria for reporting on digital health implementations. To ensure the practicality and effectiveness of the checklist, it was pilot-tested by applying it to several real-world digital health implementations, and adjustments were made based on the feedback received. The guiding principle for the development of iCHECK-DH was to identify the minimum set of information needed to comprehensively define a digital health implementation, to support the identification of key factors for success and failure, and to enable others to replicate it in different settings.

RESULTS

The result was a 20-item checklist with detailed explanations and examples in this paper. The authors anticipate that widespread adoption will standardize the quality of reporting and, indirectly, improve implementation standards and best practices.

CONCLUSIONS

Guidelines for reporting on digital health implementations are important to ensure the accuracy, completeness, and consistency of reported information. This allows for meaningful comparison and evaluation of results, transparency, and accountability and informs stakeholder decision-making. i-CHECK-DH facilitates standardization of the way information is collected and reported, improving systematic documentation and knowledge transfer that can lead to the development of more effective digital health interventions and better health outcomes.

Accuracy of a smartphone application for blood pressure estimation in Bangladesh, South Africa, and Tanzania

Undetected and unmonitored hypertension carries substantial mortality and morbidity, especially during pregnancy. We assessed the accuracy of OptiBPTM, a smartphone application for estimating blood pressure (BP), across diverse settings. The study was conducted in community settings: Gaibandha, Bangladesh and Ifakara, Tanzania for general populations, and Kalafong Provincial Tertiary Hospital, South Africa for pregnant populations. Based on guidance from the International Organization for Standardization (ISO) 81,060-2:2018 for non-invasive BP devices and global consensus statement, we compared BP measurements taken by two independent trained nurses on a standard auscultatory cuff to the BP measurements taken by a research version of OptiBPTM called CamBP. For ISO criterion 1, the mean error was 0.5 ± 5.8 mm Hg for the systolic blood pressure (SBP) and 0.1 ± 3.9 mmHg for the diastolic blood pressure (DBP) in South Africa; 0.8 ± 7.0 mmHg for the SBP and -0.4 ± 4.0 mmHg for the DBP in Tanzania; 3.3 ± 7.4 mmHg for the SBP and -0.4 ± 4.3 mmHg for the DBP in Bangladesh. For ISO criterion 2, the average standard deviation of the mean error per subject was 4.9 mmHg for the SBP and 3.4 mmHg for the DBP in South Africa; 6.3 mmHg for the SBP and 3.6 mmHg for the DBP in Tanzania; 6.4 mmHg for the SBP and 3.8 mmHg for the DBP in Bangladesh. OptiBPTM demonstrated accuracy against ISO standards in study populations, including pregnant populations, except in Bangladesh for SBP (criterion 2). Further research is needed to improve performance across different populations and integration within health systems.

Transitioning to Digital Systems: The Role of World Health Organization's Digital Adaptation Kits in Operationalizing Recommendations and Interoperability Standards

INTRODUCTION

The transition from paper to digital systems requires quality assurance of the underlying content and application of data standards for interoperability. The World Health Organization (WHO) developed digital adaptation kits (DAKs) as an operational and software-neutral mechanism to translate WHO guidelines into a standardized format that can be more easily incorporated into digital systems.

METHODS

WHO convened health program area and digital leads, reviewed existing approaches for requirements gathering, mapped to established standards, and incorporated research findings to define DAK components.

RESULTS

For each health domain area, the DAKs distill WHO guidelines to specify the health interventions, personas, user scenarios, business process workflows, core data elements mapped to terminology codes, decision-support logic, program indicators, and functional and nonfunctional requirements.

DISCUSSION

DAKs aim to catalyze quality of care and facilitate data use and interoperability as part of WHO's vision of SMART (Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable) guidelines. Efforts will be needed to strengthen a collaborative approach for the uptake of DAKs within the local digital ecosystem and national health policies.

Integration of new digital antenatal care tools using the WHO SMART guideline approach: Experiences from Rwanda and Zambia

Objectives

Digital tools for decision-support and health records can address the protracted process of guideline adoption at local levels and accelerate countries’ implementation of new health policies and programmes. World Health Organization (WHO) launched the SMART Guidelines approach to support the uptake of clinical, public health, and data recommendations within digital systems. SMART guidelines are a package of tools that include Digital Adaptation Kits (DAKs), which distill WHO guidelines into a format that facilitates translation into digital systems. SMART Guidelines also include reference software applications known as digital modules.

Methods

This paper details the structured process to inform the adaptation of the WHO antenatal care (ANC) digital module to align with country-specific ANC packages for Zambia and Rwanda using the DAK. Digital landscape assessments were conducted to determine potential integrations between the ANC digital module and existing systems. A multi-stakeholder team consisting of Ministry of Health technical officers representing maternal health, HIV, digital health, and monitoring and evaluation at district and national levels was assembled to review existing guidelines to adapt the DAK.

Results

The landscape analysis resulted in considerations for integrating the ANC module into the broader digital ecosystems of both countries. Adaptations to the DAK included adding national services not reflected in the generic DAK and modification of decision support logic and indicators. Over 80% of the generic DAK content was consistent with processes for both countries. The adapted DAK will inform the customization of country-specific ANC digital modules.

Conclusion

Both countries found that coordination between maternal and digital health leads was critical to ensuring requirements were accurately reflected within the ANC digital module. Additionally, DAKs provided a structured process for gathering requirements, reviewing and addressing gaps within existing systems, and aligning clinical content.

WHO Digital Health Guidelines: a milestone for global health

In 2019, the World Health Organization (WHO) released the first-ever evidence-based guidelines for digital health. The guideline provides nine recommendations on select digital health interventions that involve the use of a mobile phone or device. It also provides information on implementation considerations, quality and certainty of extant evidence, factors related to acceptability and feasibility of the intervention, and gaps in the evidence that can inform future research. Given the pivotal role digital health can play in supporting health systems, seen especially in light of the COVID-19 pandemic, these guidelines can help provide a roadmap for governments and policymakers in introducing and scaling up digital health interventions to support population health outcomes.

Mobile Technology for Community Health in Ghana: Is Maternal Messaging and Provider Use of Technology Cost-Effective in Improving Maternal and Child Health Outcomes at Scale?

BACKGROUND

Mobile technologies are emerging as tools to enhance health service delivery systems and empower clients to improve maternal, newborn, and child health. Limited evidence exists on the value for money of mobile health (mHealth) programs in low- and middle-income countries.

OBJECTIVE

This study aims to forecast the incremental cost-effectiveness of the Mobile Technology for Community Health (MOTECH) initiative at scale across 170 districts in Ghana.

METHODS

MOTECH's "Client Data Application" allows frontline health workers to digitize service delivery information and track the care of patients. MOTECH's other main component, the "Mobile Midwife," sends automated educational voice messages to mobile phones of pregnant and postpartum women. We measured program costs and consequences of scaling up MOTECH over a 10-year analytic time horizon. Economic costs were estimated from informant interviews and financial records. Health effects were modeled using the Lives Saved Tool with data from an independent evaluation of changes in key services coverage observed in Gomoa West District. Incremental cost-effectiveness ratios were presented overall and for each year of implementation. Uncertainty analyses assessed the robustness of results to changes in key parameters.

RESULTS

MOTECH was scaled in clusters over a 3-year period to reach 78.7% (170/216) of Ghana's districts. Sustaining the program would cost US $17,618 on average annually per district. Over 10 years, MOTECH could potentially save an estimated 59,906 lives at a total cost of US $32 million. The incremental cost per disability-adjusted life year averted ranged from US $174 in the first year to US $6.54 in the tenth year of implementation and US $20.94 (95% CI US $20.34-$21.55) over 10 years. Uncertainty analyses suggested that the incremental cost-effectiveness ratio was most sensitive to changes in health effects, followed by personnel time. Probabilistic sensitivity analyses suggested that MOTECH had a 100% probability of being cost-effective above a willingness-to-pay threshold of US $50.

CONCLUSIONS

This is the first study to estimate the value for money of the supply- and demand-side of an mHealth initiative. The adoption of MOTECH to improve MNCH service delivery and uptake represents good value for money in Ghana and should be considered for expansion. Integration with other mHealth solutions, including e-Tracker, may provide opportunities to continue or combine beneficial components of MOTECH to achieve a greater impact on health.

Digital Health and Health Systems of the Future

Digital strategies have been formally recognized as a critical health systems strengthening strategy to help meet the Sustainable Development Goals and universal health coverage targets. This landscaping collection reviews multiple possible approaches across health system pillars, from digital referrals to decision support systems, identifying key knowledge gaps across these domains and recognizing the growth needed in the field to realize its full potential.

The State of Digital Interventions for Demand Generation in Low- and Middle-Income Countries: Considerations, Emerging Approaches, and Research Gaps

The recent introduction of digital health into generating demand for health commodities and services has provided practitioners with an expanded universe of potential tools to strengthen demand and ensure service delivery receipt. However, considerable gaps remain in our understanding of which interventions are effective, which characteristics mediate their benefit for different target populations and health domains, and what is necessary to ensure effective deployment. This paper first provides an overview of the types of digital health interventions for demand generation, including untargeted client communication, client-to-client communication, on-demand information services, personal health tracking, client financial transactions, and targeted client communication. It then provides a general overview of 118 studies published between January 1, 2010, and October 3, 2017, that used digital interventions to generate demand for health interventions. The majority (61%) of these studies used targeted client communication to provide health education or reminders to improve treatment adherence, and the most frequently (27%) studied health condition was HIV/AIDS. Intervention characteristics that have been found to have some effect on gains in demand generation include modality, directionality, tailoring, phrasing, and schedule. The paper also explores new emergent digital approaches that expand the potential effect of traditional demand generation in terms of personalization of content and services, continuity of care, and accountability tracking. Applying existing frameworks for monitoring and evaluation and reporting, research on emerging approaches will need to consider not only their feasibility but also their effectiveness in achieving demand generation outcomes. We propose a research agenda to help guide the field of digital demand generation studies and programs within a broader health systems strengthening agenda, including establishing and documenting the influence of intervention characteristics within different populations and health domains and examining the long-term effects and cost-effectiveness of digital demand generation interventions, as well as equity in access to such interventions.

Mobile Technology for Community Health in Ghana: what happens when technical functionality threatens the effectiveness of digital health programs?

BACKGROUND

Despite the growing use of technology in the health sector, little evidence is available on the technological performance of mobile health programs nor on the willingness of target users to utilize these technologies as intended (behavioral performance). In this case study of the Mobile Technology for Health (MOTECH) program in Ghana, we assess the platform's effectiveness in delivering messages, along with user response across sites in five districts from 2011 to 2014.

METHODS

MOTECH is comprised of "Client Data Application" (CDA) which allows providers to digitize and track service delivery information for women and infants and "Mobile Midwife" (MM) which sends automated educational voice messages to the mobile phones of pregnant and postpartum women. Using a naturalist study design, we draw upon system generated data to evaluate message delivery, client engagement, and provider responsiveness to MOTECH over time and by level of facility.

RESULTS

A total of 7,370 women were enrolled in MM during pregnancy and 14,867 women were enrolled postpa1rtum. While providers were able to register and upload patient-level health information using CDA, the majority of these uploads occurred in Community-based facilities versus Health Centers. For MM, 25% or less of expected messages were received by pregnant women, despite the majority (>77%) owning a private mobile phone. While over 80% of messages received by pregnant women were listened to, postpartum rates of listening declined over time. Only 25% of pregnant women received and listened to at least 1 first trimester message. By 6-12 months postpartum, less than 6% of enrolled women were exposed to at least one message.

CONCLUSIONS

Caution should be exercised in assuming that digital health programs perform as intended. Evaluations should measure the technological, behavioral, health systems, and/or community factors which may lead to breaks in the impact pathway and influence findings on effectiveness. The MOTECH platform's technological limitations in 'pushing' out voice messages highlights the need for more timely use of data to mitigate delivery challenges and improve exposure to health information. Alternative message delivery channels (USSD or SMS) could improve the platform's ability to deliver messages but may not be appropriate for illiterate users.

TRIAL REGISTRATION

Not applicable.

eRegistries: Electronic registries for maternal and child health

Background

The Global Roadmap for Health Measurement and Accountability sees integrated systems for health information as key to obtaining seamless, sustainable, and secure information exchanges at all levels of health systems. The Global Strategy for Women’s, Children’s and Adolescent’s Health aims to achieve a continuum of quality of care with effective coverage of interventions. The WHO and World Bank recommend that countries focus on intervention coverage to monitor programs and progress for universal health coverage. Electronic health registries - eRegistries - represent integrated systems that secure a triple return on investments: First, effective single data collection for health workers to seamlessly follow individuals along the continuum of care and across disconnected cadres of care providers. Second, real-time public health surveillance and monitoring of intervention coverage, and third, feedback of information to individuals, care providers and the public for transparent accountability. This series on eRegistries presents frameworks and tools to facilitate the development and secure operation of eRegistries for maternal and child health.

Methods

In this first paper of the eRegistries Series we have used WHO frameworks and taxonomy to map how eRegistries can support commonly used electronic and mobile applications to alleviate health systems constraints in maternal and child health. A web-based survey of public health officials in 64 low- and middle-income countries, and a systematic search of literature from 2005–2015, aimed to assess country capacities by the current status, quality and use of data in reproductive health registries.

Results

eRegistries can offer support for the 12 most commonly used electronic and mobile applications for health. Countries are implementing health registries in various forms, the majority in transition from paper-based data collection to electronic systems, but very few have eRegistries that can act as an integrating backbone for health information. More mature country capacity reflected by published health registry based research is emerging in settings reaching regional or national scale, increasingly with electronic solutions. 66 scientific publications were identified based on 32 registry systems in 23 countries over a period of 10 years; this reflects a challenging experience and capacity gap for delivering sustainable high quality registries.

Conclusions

Registries are being developed and used in many high burden countries, but their potential benefits are far from realized as few countries have fully transitioned from paper-based health information to integrated electronic backbone systems. Free tools and frameworks exist to facilitate progress in health information for women and children.

Electronic supplementary material

The online version of this article (doi:10.1186/s12884-016-0801-7) contains supplementary material, which is available to authorized users.

Unmet Need: Improving mHealth Evaluation Rigor to Build the Evidence Base

mHealth-the use of mobile technologies for health-is a growing element of health system activity globally, but evaluation of those activities remains quite scant, and remains an important knowledge gap for advancing mHealth activities. In 2010, the World Health Organization and Columbia University implemented a small-scale survey to generate preliminary data on evaluation activities used by mHealth initiatives. The authors describe self-reported data from 69 projects in 29 countries. The majority (74%) reported some sort of evaluation activity, primarily nonexperimental in design (62%). The authors developed a 6-point scale of evaluation rigor comprising information on use of comparison groups, sample size calculation, data collection timing, and randomization. The mean score was low (2.4); half (47%) were conducting evaluations with a minimum threshold (4+) of rigor, indicating use of a comparison group, while less than 20% had randomized the mHealth intervention. The authors were unable to assess whether the rigor score was appropriate for the type of mHealth activity being evaluated. What was clear was that although most data came from mHealth projects pilots aimed for scale-up, few had designed evaluations that would support crucial decisions on whether to scale up and how. Whether the mHealth activity is a strategy to improve health or a tool for achieving intermediate outcomes that should lead to better health, mHealth evaluations must be improved to generate robust evidence for cost-effectiveness assessment and to allow for accurate identification of the contribution of mHealth initiatives to health systems strengthening and the impact on actual health outcomes.

Using the lives saved tool (LiST) to model mHealth impact on neonatal survival in resource-limited settings

While the importance of mHealth scale-up has been broadly emphasized in the mHealth community, it is necessary to guide scale up efforts and investment in ways to help achieve the mortality reduction targets set by global calls to action such as the Millennium Development Goals, not merely to expand programs. We used the Lives Saved Tool (LiST)–an evidence-based modeling software–to identify priority areas for maternal and neonatal health services, by formulating six individual and combined interventions scenarios for two countries, Bangladesh and Uganda. Our findings show that skilled birth attendance and increased facility delivery as targets for mHealth strategies are likely to provide the biggest mortality impact relative to other intervention scenarios. Although further validation of this model is desirable, tools such as LiST can help us leverage the benefit of mHealth by articulating the most appropriate delivery points in the continuum of care to save lives.

mHealth innovations as health system strengthening tools: 12 common applications and a visual framework

This new framework lays out 12 common mHealth applications used as health systems strengthening innovations across the reproductive health continuum.

Effective health interventions for adolescents that could be integrated with human papillomavirus vaccination programs

PURPOSE

We reviewed published data to identify health interventions for 9-15-year-old girls and boys that could to be usefully integrated with programs of human papillomavirus (HPV) vaccination in low- and middle-income countries (LMICs).

METHODS

Relevant literature reviews, bibliographic databases, and journals were searched to identify health-related interventions, other than immunizations, that had been found to have beneficial outcomes among adolescent girls and/or boys. An intervention was excluded if there was no evidence of its effective delivery in LMICs or no demonstrated potential for its adaptation for delivery in such countries, and/or if there was, apparently, no feasible way in which it could be delivered during a course of HPV vaccinations.

RESULTS

Overall, 33 different interventions were found to have had beneficial outcomes among adolescents living in LMICs. Of these, 19 were excluded because they were deemed too expensive or too difficult to deliver within the calendar of a HPV vaccination program. The remaining 14 health-related interventions, in the fields of screening (for schistosomiasis and defects in vision), health education (on mosquito-borne diseases, the benefits of exercise, accessing health care, and sexual and reproductive health), skills building (improving condom usage) and delivery of commodities (anthelminthic drugs, vitamin A supplements, soap and/or bed nets) were deemed potential candidates for delivery in conjunction with the HPV vaccine.

CONCLUSIONS

The potential benefits and selection of other health-related interventions that are delivered in conjunction with HPV vaccine will be influenced by a range of factors, including the ease of delivery, the epidemiology of the priority health problems affecting adolescents, the vaccine delivery schedule, and various environmental, economic, and social factors. However, there appear to be several interventions that could usefully be integrated in many, if not all, HPV vaccination programs. The ability to deliver multiple interventions along with HPV vaccine could not only offer important efficiencies but also serve as an entry point to increase adolescents' access to health care and services.

H_pe for mHealth: more "y" or "o" on the horizon?

OBJECTIVE

Efforts in the domain of mobile health, or mHealth, have been criticized for the unfettered proliferation of pilots and a lack of a rigorous evidence base to support these strategies. In this letter, we present the response of a group of researchers in the mHealth community to the recent calls for evidence issued by global health and funding agencies. We support our conclusions through a summary of the numerous ongoing mHealth studies listed in the US federal clinical trial registry.

METHODS

We conducted a search on the US federal clinicaltrials.gov database using the keywords "mHealth", "mobile" or "cell AND phone" to obtain 1678 results of studies. We manually inspected each result to check if it fit the purview of an mHealth study. Studies that were terminated or withdrawn prior to submission were excluded.

RESULTS

We identified 215 unique mHealth studies that were registered in the clinicaltrials.gov database, of which 8.4% (n=18) were observational in nature while the remaining 91.6% (n=197) were interventional. Of the 215 studies, 81.8% (n=176) studies used a classical randomized trial design and 40 new studies were added to the database between May and November 2012 alone. Based on these results, we posit that the field is entering a new 'era' where a body of rigorous evaluation of mHealth strategies is rapidly accumulating.

CONCLUSIONS

The transition into an era of evidence-based mHealth supports our position that innovation in this domain can be evaluated with the same rigor as other public health strategies, attenuating some of the hype previously associated with mHealth.

Pregnancy registration systems can enhance health systems, increase accountability and reduce mortality

As many low- to middle-income countries strive to achieve targets of reduced maternal, neonatal and infant mortality set by the Millennium Development Goals, health system innovations which can accelerate progress are being carefully examined. Among these are technologies and systems which aim to strengthen frontline health workers and the health systems within which they work, by enabling the registration of pregnancies, births and outcomes. Accurate, population-based numerators and denominators can help to improve accountability of the health system to provide expected routine antenatal and post-natal care, as well as emergency support and referral, as needed. The enumeration of women of reproductive age, followed by prospective, voluntary registration of pregnancies has the potential to support governments, health agencies, and the populations they serve, to ensure public health service delivery and to guide informed policies.