The Challenges Toward Real-world Implementation of Digital Health Design Approaches: Narrative Review

A multi-site study of clinician perspectives in the lifecycle of an algorithmic risk prediction tool

Recent advancements in the performative capacities of artificial intelligence (AI), machine learning (ML), and algorithmic-based tools open up numerous applications in modern medicine. There are, however, few studies that track the whole lifecycle of a digital healthcare tool as it evolves from conception, to design, and deployment in real world settings-especially with a focus on the social dynamics amongst the end-users of the tool: clinicians. In this paper, we present data from a multi-site, 5-year study focused on the development and deployment of an algorithmic risk calculator (HeartMate 3 Risk Score) into a validated and efficacy tested clinical decision support system (CDSS) for patients and clinicians engaging in shared decision making about left ventricular assist device (LVAD) therapy for advanced heart failure. We conducted a total of 76 interviews with 20 advanced heart failure cardiologists and 14 nurse coordinators with LVAD expertise (n=34) across different timepoints during the lifecycle of this digital healthcare tool. Results from Thematic Analysis revealed an array of social factors at play at each stage of the tool's development and implementation, from finding social consensus around risk messaging in the conception and design phases, to various social contingencies that served as facilitators and barriers to the successful integration of the tool in its later stages. Our findings confirm many previously raised issues with introducing new medical and digital healthcare tools into clinical care, and highlight new issues specific to the rapidly advancing technology in CDSS.

Identifying and Optimizing Factors Influencing the Implementation of a Fast Healthcare Interoperability Resources Accelerator: Qualitative Study Using the Consolidated Framework for Implementation Research-Expert Recommendations for Implementing Change Approach

BACKGROUND

Fragmented sharing of health information is known to negatively impact patient care and outcomes. To support the sharing of health information between systems, Fast Healthcare Interoperability Resources (FHIR) has emerged as the global interoperability standard for health information exchange. To speed up the process of adoption, various FHIR accelerator groups have been formed. FHIR accelerators such as the Sparked program in Australia enable communities and collaborative groups to develop high-quality FHIR standards for health care information exchange and encourage widespread uptake. However, limited research exists on the development, delivery, and implementation of FHIR accelerator programs.

OBJECTIVE

This study used qualitative methods to identify the key components of the Sparked FHIR accelerator, what factors influence implementation, and which strategies may help enhance its delivery.

METHODS

Semistructured interviews were conducted with Sparked stakeholders in the early stage of the program. The Sparked FHIR accelerator intervention components were described using a standardized reporting checklist (Template for Intervention Description and Replication). The Consolidated Framework for Implementation Research (CFIR) 2.0 was used to analyze factors influencing implementation. On the basis of a cumulative majority analysis, the most mentioned factors influencing implementation were identified. These factors were then mapped to the Expert Recommendations for Implementing Change (ERIC) tool to identify strategies for enhancing the implementation of the Sparked program.

RESULTS

A total of 17 participants were interviewed, including program leads, cochairs, representatives of software industry implementers, clinicians, and consumers. In total, 8 key CFIR influencing factors were identified: engaging, innovation design, assessing needs, local conditions, access to knowledge and information, partnerships and connections, capability, and work infrastructure. After mapping the top CFIR influencing factors to the ERIC tool, 5 strategy clusters were identified: adapt and tailor to context, develop stakeholder interrelations, support participants, train and educate stakeholders, and use evaluative and iterative strategies.

CONCLUSIONS

This study enabled the core components of the Sparked FHIR accelerator to be defined and identified the factors that have the strongest influence on program implementation. Using the CFIR-ERIC approach facilitated the generation of expert-informed recommendations for improving the implementation of Sparked, but researcher recommendations were needed to supplement the tool. This research offers valuable insights for decision makers and implementers.

Creating a Usable and Effective Digital Intervention to Support Men to Test for HIV and Link to Care in A Resource-Constrained Setting: Iterative Design Based on A Person-Based Approach and Human Computer Interaction Methods

BACKGROUND

It is challenging to design usable and effective digital health interventions (DHIs). The person-based approach (PBA) has been proposed to incorporate users' perspectives for the design of DHIs. However, it does not explicitly describe the iterative stages of design and evaluation that are essential in moving from early planning to deployment. For this, we draw on methods from human computer interaction (HCI) that have been developed for various situations.

OBJECTIVE

This study aimed to reflect on the adaptation and synthesis of PBA and HCI approaches to developing DHIs. We present a case study applying both approaches to develop Empowering People through Informed Choices for HIV (EPIC-HIV1), a DHI designed for men living in rural KwaZulu-Natal, South Africa, intended to support them in making an informed choice about whether to take an HIV test and, if necessary, engage in care.

METHODS

We conducted a retrospective analysis of the documentation generated during the development of EPIC-HIV1 including findings about requirements, design representations, and the results of iterative rounds of testing. We developed an account of the process, the outcomes, and the strengths and limitations of the design and evaluation techniques applied. We also present the design of EPIC-HIV1 and summarize considerations when designing for hard-to-reach people in such settings.

RESULTS

The PBA was applied to deliver a first prototype. This helped identify key messages to convey and how to manage issues such as user privacy, but the resulting prototype was judged by the team not to be engaging for potential users, and it was unclear whether the design was inclusive of people with low digital or health literacy. We therefore introduced methods from HCI to iteratively test and refine the app. Working with local community representatives, we conducted four refinement cycles with 29 participants, adapting and retesting the app until no further changes were needed. Key changes included making it clearer what the consequences of selecting options in the app were and changing wording to minimize misconceptions (eg, that the app would test for HIV) while addressing common concerns about testing and emphasizing long-term benefits of engaging with care, if needed.

CONCLUSIONS

Techniques for developing DHIs need to be situationally appropriate. The PBA enabled us to establish both empirical data and theory to design the content of EPIC-HIV1, but it did not directly inform interaction design to make the app usable and effective for the intended users; HCI techniques tailored to the setting enabled us to refine the app to be easy for men with little familiarity with digital technologies to use within the constraints of the setting. Iterative testing ensured the app was easy to use and that the intended clinical messages were communicated effectively.

Applying design to design: demonstrating how to create a human-centered design session interview guide for use with adolescents

Digital health interventions (DHIs) hold promise for improving the reach of mental health care for adolescents, particularly those from under-resourced communities who may face significant barriers to accessing in-person care. Yet, low engagement and uptake have challenged DHIs' potency. Human-centered design (HCD) integrates end-users (i.e., future users of the DHI) into iterative design processes, thereby prioritizing their needs and preferences. Clinical scientists are increasingly embracing HCD, but often lack expertise in how to apply these methods in practice. We provide a template for creating a design session interview guide in a needs assessment, which is the first phase in our HCD process to design a DHI for dysregulated eating in adolescents. To create the guide, we first conducted a "needs assessment" within our team to identify important topic areas that required feedback from adolescents ("investigate"). We then consolidated these ideas into structured domains through a brainstorming process ("ideate"), which resulted in an initial draft of a design session guide ("prototype"). Next, we piloted the prototype with members of our team and a technology-savvy adolescent ("evaluate") to refine it prior to administration with the target audience ("refine and develop"). Our internal needs assessment identified that we needed to learn adolescents' preferences for technology (e.g., desired features), clinical content (e.g., areas for specialized support), delivery (e.g., coaching), and developmental relevance (e.g., focus on self-regulation). We organized these topics into six domains: dysregulated eating experiences and current help-seeking behaviors, major challenges that impact dysregulated eating, preferred intervention features and skills, preferences for coaching support, the potential role of sensors to assess activity behaviors, and preferred aesthetics and brand. We created relevant prompts within each domain, revised, and reordered them to elicit more comprehensive responses during administration. Next, we practiced administering the guide internally amongst our team, then with a non-participant adolescent volunteer. Using HCD, we created a semi-structured design session interview guide that will be administered in an upcoming needs assessment with adolescents and will continue to evolve as we learn from adolescents. This case example unpacks the process of creating and iterating a design session guide that could be applied across clinical domains.

Development and usability of a mobile artificial intelligence platform for the management of childhood developmental disorders based on PHRs

Introduction: Emerging technologies, particularly artificial intelligence (AI), offer the potential to personalize healthcare for pediatric developmental disorders, but their development presents challenges. Methods: This study introduces IVORY, a mobile AI platform for managing personal health records (PHRs) in children with developmental disorders. IVORY integrates advanced optical character recognition (OCR)-based text recognition models optimized for diverse medical document types and template-matching algorithms, ensuring standardized data processing. The primary features include digitizing medical records, symptom interpretation, and AI-driven health recommendations. Results: Using pretrained OCR algorithms with 126 diverse medical report types, the platform achieved an OCR success rate of 81%. Input data include fMRI interpretations, psychological assessments, and laboratory findings, whereas outputs offer percentile-based insights and treatment recommendations. Caregivers (3.44 ± 0.67) and professionals (3.50 ± 0.63) highly rated the platform for usability. Conclusions: Despite OCR limitations for low-resolution data, IVORY has the potential to enhance data consolidation, accuracy, and scalability in personalized pediatric healthcare.

Examining Challenges to Co-Design Digital Health Interventions With End Users: Systematic Review

BACKGROUND

Digital health interventions (DHIs) are changing the dynamic of health care by providing personalized, private, and instantaneous solutions to end users. However, the explosion of digital health has been fraught with challenges. The approach to co-design with end users varies across a diverse domain of stakeholders, often resulting in siloed approaches with no clear consensus. The concept of validating user experiences contrasts greatly between digital stakeholders (ie, user experience and retention) and health stakeholders (ie, safety and efficacy). Several methodologies and frameworks are being implemented to address this challenge to varying degrees of success.

OBJECTIVE

We aimed to broadly examine the advancements and challenges to co-design DHIs with end users over the last decade. This task was undertaken to identify the key problem areas at the domain level, with the ultimate goal of creating recommendations for better approaches to co-design DHIs with end users.

METHODS

We conducted a systematic search of key databases for co-design studies involving end users in DHIs. Searches were divided into 3 relevant streams: health behavior, user experience, and digital methodologies and frameworks. The eligibility criteria were guided by the PerSPEcTiF framework and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. In line with this framework, studies were included in this review that (1) address research on DHIs; (2) focus on interaction and co-design with end users; (3) explain results such that uptake, effectiveness, satisfaction, and health outcomes are discernible, positively or negatively; and (4) describe actionable procedures for better DHI design. The search was conducted in a diverse group of 6 bibliographical databases from January 2015 to May 2024: PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Institute of Electrical and Electronics Engineers Xplore, and Scopus. From the 13,961 studies initially screened for titles and abstracts, 489 (3.6%) were eligible for a full-text screening, of which 171 (1.2%) studies matched the inclusion criteria and were included in a qualitative synthesis.

RESULTS

Of the 171 studies analyzed across 52 journals, we found 5 different research approaches, spanning 8 different digital health solution types and 5 different design methodologies. These studies identified several core themes when co-designing with end users: advancements, which included participatory co-design; challenges, which included participatory co-design, environment and context, testing, and cost and scale; and gaps, which included a pragmatic hybridized framework and industry implementability.

CONCLUSIONS

This research supports a pragmatic shift toward using mixed methods approaches at scale, methods that are primed to take advantage of the emerging big data era of digital health co-design. This organic outlook should blend the vision of digital health co-designers with the pragmatism of Agile design methodology and the rigor of health care metrics.

TRIAL REGISTRATION

PROSPERO CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021238164.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/28083.

Perceptions of eHealth and digitalization among professional anaesthesia personnel: A Swedish national study

BACKGROUND

The objective of this study was to evaluate anaesthesia care professionals' perceptions and attitudes regarding the implementation and advancement of digital solutions in perioperative care.

METHODS

Anaesthesia personnel working in public Swedish institutions where anaesthesia is administered were invited to respond to an online survey regarding their attitudes towards digitalization in the workplace and their perceptions of information provision and future digitalization within anaesthesia and surgical healthcare. Data were analyzed using descriptive statistics, independent-samples Kruskal-Wallis tests, and post-hoc pairwise comparisons.

RESULTS

The survey response rate was 64.0% (n = 627). Most respondents agreed/strongly agreed that digital solutions facilitate their work, the preoperative preparation, patient participation, and being involved in the patients' journeys throughout the perioperative care process. The majority also agreed/strongly agreed that digital solutions could make more patients adequately prepared before anaesthesia/surgery, reduce the number of non-optimized patients, and adapt the perioperative process to the patients' individual needs, as well as lead to reduced costs for the healthcare provider and reduced cancelled anaesthesia/surgeries. However, there were statistically significant differences between responses in relation to age groups, where the largest differences were observed between respondents in the age groups 20-30 and 61-70 years and in relation to what part of Sweden respondents worked in, with the largest differences between respondents working in Southern Sweden and the middle part of Sweden.

CONCLUSION

Swedish anaesthesia personnel are confident that digital solutions may enhance the efficiency of care within the anaesthesia setting. However, varying perceptions on the benefits and necessity of digital solutions are indicated.

Implementation of WHO SMART Guidelines-Digital Adaptation Kits in Pathfinder Countries in Africa: Processes and Early Lessons Learned

Background

The adoption of digital systems requires processes for quality assurance and uptake of standards to achieve universal health coverage. The World Health Organization developed the Digital Adaptation Kits (DAKs) within the SMART (Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable) guidelines framework to support the uptake of standards and recommendations through digital systems. DAKs are a software-neutral mechanism for translating narrative guidelines to support the design of digital systems. However, a systematic process is needed to implement and ensure the impact of DAKs in country contexts.

Objective

This paper details the structured process and stepwise approach to customize the DAKs to the national program and digital context in 5 countries in Africa with diverse program guideline uptake and significant digital health investments: Ethiopia, Ghana, Malawi, Zambia, and Zimbabwe. All these countries have existing digital systems, which have the potential to be updated with the DAKs.

Methods

A DAK assessment tool was developed and used to assess guideline digitization readiness and opportunities for system uptake in each country. Multistakeholder teams were established to conduct the content review and alignment of the generic DAK to national guidelines and protocols through a series of stakeholder consultations, including stakeholder orientation, content review and alignment, content validation, and software update meetings.

Unlabelled

Country adaptation processes identified requirements for national-level contextualization and highlighted opportunities for refinement of DAKs. Quality assurance of the content during the content review and validation processes ensured alignment with national protocols. Adaptation processes also facilitated the adoption of the DAKs approach into national guidelines and strategic documents for sexual and reproductive health.

Conclusions

Country experiences offered early insights into the opportunities and benefits of a structured approach to digitalizing primary health care services. They also highlighted how this process can be continuously refined and sustained to enhance country-level impact.

Impact of Online Interactive Decision Tools on Women's Decision-Making Regarding Breast Cancer Screening: Systematic Review and Meta-Analysis

BACKGROUND

The online nature of decision aids (DAs) and related e-tools supporting women's decision-making regarding breast cancer screening (BCS) through mammography may facilitate broader access, making them a valuable addition to BCS programs.

OBJECTIVE

This systematic review and meta-analysis aims to evaluate the scientific evidence on the impacts of these e-tools and to provide a comprehensive assessment of the factors associated with their increased utility and efficacy.

METHODS

We followed the 2020 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and conducted a search of MEDLINE, PsycINFO, Embase, CINAHL, and Web of Science databases from August 2010 to April 2023. We included studies reporting on populations at average risk of breast cancer, which utilized DAs or related e-tools, and assessed women's participation in BCS by mammography or other key cognitive determinants of decision-making as primary or secondary outcomes. We conducted meta-analyses on the identified randomized controlled trials, which were assessed using the revised Cochrane Risk of Bias 2 (RoB 2) tool. We further explored intermediate and high heterogeneity between studies to enhance the validity of our results.

RESULTS

In total, 22 different e-tools were identified across 31 papers. The degree of tailoring in the e-tools, specifically whether the tool was fully tailored or featured with tailoring, was the most influential factor in women's decision-making regarding BCS. Compared with control groups, tailored e-tools significantly increased women's long-term participation in BCS (risk ratio 1.14, 95% CI 1.07-1.23, P<.001, I2=0%). Tailored-to-breast-cancer-risk e-tools increased women's level of worry (mean difference 0.31, 95% CI 0.13-0.48, P<.001, I2=0%). E-tools also improved women's adequate knowledge of BCS, with features-with-tailoring e-tools designed and tested with the general population being more effective than tailored e-tools designed for or tested with non-BCS participants (χ21=5.1, P=.02). Features-with-tailoring e-tools increased both the rate of women who intended not to undergo BCS (risk ratio 1.88, 95% CI 1.43-2.48, P<.001, I2=0%) and the rate of women who had made an informed choice regarding their intention to undergo BCS (risk ratio 1.60, 95% CI 1.09-2.33, P=.02, I2=91%). Additionally, these tools decreased the proportion of women with decision conflict (risk ratio 0.77, 95% CI 0.65-0.91, P=.002, I2=0%). Shared decision-making was not formally evaluated. This review is limited by small sample sizes, including only a few studies in the meta-analysis, some with a high risk of bias, and high heterogeneity between the studies and e-tools.

CONCLUSIONS

Features-with-tailoring e-tools could potentially negatively impact BCS programs by fostering negative intentions and attitudes toward BCS participation. Conversely, tailored e-tools may increase women's participation in BCS but, when tailored to risk, they may elevate their levels of worry. To maximize the effectiveness of e-tools while minimizing potential negative impacts, we advocate for an "on-demand" layered approach to their design.

V3+ extends the V3 framework to ensure user-centricity and scalability of sensor-based digital health technologies

We propose the addition of usability validation to the extended V3 framework, now "V3+", and describe a pragmatic approach to ensuring that sensor-based digital health technologies can be used optimally at scale by diverse users. Alongside the original V3 components (verification; analytical validation; clinical validation), usability validation will ensure user-centricity of digital measurement tools, paving the way for more inclusive, reliable, and trustworthy digital measures within clinical research and clinical care.

Positive deviance/hearth intervention in collaboration between academia and NGOs: a realist evaluation

Numerous NGOs and donors from high-income countries (HICs) offer diverse funding to assist LMICs. A collaborative effort in nutrition programs in urban communities was conducted by an international NGO with a local university, representing the NGO's first mode of partnership with academia. This study used realist evaluation to understand how and why Positive Deviance/Hearth intervention conducted by NGOs collaborated with university work or failed to work in urban population. We uncovered the underlying mechanisms that lead to certain outcomes in different situations of academia-NGO partnership. The initial program theory was formulated after discussions with main program officers from both NGO and academia. We conducted stakeholder interviews and FGD, along with validation and reviewing secondary data. The initial program theory underwent testing and refinement through a series of Focus Group Discussions (FGDs) involving program officers and local government health offices. We adhered to the RAMESES II reporting standards for realist evaluations when presenting the findings. Five Context Mechanism-Outcome (CMO) configurations are identified from three program theories. Our findings illustrate the importance of considering the local context of the population when implementing standardized international NGO interventions. All the CMOs indicate that at the very least, collaborative programs between academia and NGOs should involve more stakeholder involvement and build clearer expectations between stakeholders.

Practical guidelines for developing digital health solutions to increase equity in dementia care in the UK

Background

Digital Healthcare Solutions (DHS) are transforming healthcare by improving patients' experiences, safety and quality of care. However, despite all the proposed and observed advantages of DHS, a growing body of research suggests that these DHS are not equally accessible to all. This research aimed to assess whether equity frameworks for digital health solutions can be used to guide the development of digital health solutions to increase access to care for dementia patients in the UK and, thereafter, develop practical guidelines to guide the design of equitable DHS products to address this growing issue.

Methods

A scoping review across four databases and grey literature was done to identify equity frameworks and design principles for DHS. The equity frameworks and design principles were analyzed to make recommendations on increasing equity in the product.

Results

34 publications and reports met the inclusion criteria. Four equity frameworks were analyzed and one was selected. Equitable product development guidelines were created based on patient-centered design principles.

Conclusion

Although DHS can increase inequity in healthcare, concrete methods and practical guidelines can minimize this if DHS developers design for maximum equity and closely collaborate with healthcare providers and end-users in product development. Future research could use these guidelines to test usability for developers and investigate other equitable approaches like institutional barriers to adoption.

Operationalizing implementation science frameworks to plan a hybrid effectiveness-implementation study of a digital health intervention

Background

Evaluating implementation of digital health interventions (DHIs) in practice settings is complex, involving diverse users and multistep processes. Proactive planning can ensure desired implementation determinants and outcomes are captured for hybrid studies, but operational guidance for DHI studies is limited.

Methods

We planned a cluster randomized, type II hybrid effectiveness-implementation trial testing PositiveLinks, a smartphone application for HIV care, compared to usual care (n = 6 sites per arm), among HIV outpatient sites in the DC Cohort Longitudinal HIV Study in Washington, DC. Our process included: 1) Defining components of the DHI and associated implementation strategy, 2) Selecting implementation science frameworks to accomplish evaluation aims, 3) Mapping framework dimensions, domains, and constructs to implementation strategy steps, 4) Modifying/creating instruments to collect data for implementation outcome measures and determinants and 5) Developing a compatible implementation science data collection and management plan.

Results

Specification of components of the DHI and implementation strategy identified relevant platform usage data and necessary implementer actions, toward planning measurement of program reach and adoption. A priori mapping of implementation strategy steps to the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework identified how data would be collected for each step/corresponding outcome measure. The multi-site cohort study provided infrastructure for prospective, scaled implementation research, including site research assistants (RAs) coordinating implementation. Existing cohort tools (periodic site assessments, patient consent logs) were adapted for the evaluation to further capture representativeness and reach/adoption 'denominators.' New survey instruments allowed for framework-guided evaluation of provider adaptations/dose/fidelity to planned implementation across a large number of participants and multiple timepoints. Some aspects of real-world implementation were challenging to mirror within the planned hybrid trial (e.g. RAs selected as de facto site implementation leads) or were modified to preserve internal validity of effectiveness measurement (e.g. PositiveLinks 'community of practice').

Conclusions

Challenges and opportunities arose in planning the implementation evaluation for PositiveLinks within a hybrid trial in a cohort population. Prospective hybrid trial planning must balance generalizability of implementation processes to 'real world' conditions, with rigorous trial procedures to measure intervention effectiveness. Rapid, scalable tools require further study to enable evaluations within large, multi-site hybrid studies.

Exploring the Needs of People With Chronic Low Back Pain and Health Care Professionals for mHealth Devices to Support Self-Managed Physical Activity and Pain: User-Centered Design Approach

BACKGROUND

Chronic low back pain (CLBP) is a major economic and social problem worldwide. Despite the variety of recommended treatments, long-term self-management of this condition is complex and requires the development of innovative interventions. Mobile health (mHealth) technologies hold great promise for the management of chronic pain, particularly to support physical activity. However, their implementation is challenged by a lack of user compliance and limited engagement, which may be due to insufficient consideration of the needs of potential users during development.

OBJECTIVE

This study aims to explore the needs of people with CLBP and health care professionals regarding mHealth technologies to support self-managed physical activity, and to delineate design recommendations based on identified needs.

METHODS

A participatory study was conducted using a 3-phase, user-centered design approach: needs investigation with a group of experts in a workshop (phase 1), needs exploration with end users in focus groups (phase 2), and validation of needs using Delphi questionnaires followed by the development of a set of recommendations (phase 3).

RESULTS

A total of 121 people with CLBP, expert patients, health care professionals, rehabilitation researchers, and biomechanical engineers participated in this study. The results indicated how technology could help people with CLBP overcome their difficulties with managing physical activity. Specific needs were formulated concerning device objectives, expected strategies, functionalities, technical features, conditions of use, and potential facilitators and barriers to use. These needs were validated by consensus from the potential end users and translated into design recommendations.

CONCLUSIONS

This study provides design recommendations for the development of an mHealth device specifically adapted for people with CLBP.

The challenges of and solutions for combining cocreation and agile in the development of health information technologies

BACKGROUND

Involving users has become a prominent principle in the development of Health Information Technologies (HIT) and has led to an uprise in agile and cocreation methods. Previous literature shows how the two can be combined in one method, but also suggest that using such a method may come with challenges, for which the solutions are unclear.

PURPOSE

To identify the challenges of using a method that combines agile and cocreation, provide solutions for these challenges, and evaluate its usage.

METHODS AND MATERIALS

The setting for this research was the Agile Cocreation of Robots for Aging (ACCRA) project. The research consisted of three phases: 1) evaluating the Agile Cocreation method to identify challenges in its usage, by analysing documents from the project (e-mails, meeting notes), 2) solving the challenges to improve the method, by designing solutions in a cocreation session; and 3) evaluating the usage of the improved version via a survey among engineers and user researchers involved in the project.

RESULTS

We identified three main challenges and developed three solutions, which were used in the next phase of the project. First, to engage all stakeholders in cocreation, we implemented more fun and playful materials. Second, to bridge the differences between engineers and user researchers we invested in face-to-face meetings. Third, to manage knowledge in the project we intensified our meeting schedule to weekly meetings. In the quantitative evaluation of the improved cocreation method, the engineers and user researchers were positive about the agile cocreation method and about our improvements.

CONCLUSION

When developing HIT, a method that combines agile and cocreation is useful because it helps to identify user needs and to translate these needs into technology. To identify the needs of these users and other stakeholders it is important to involve them as active partners in cocreation using fun and playful materials. Engineers and user researchers should bridge their differences and meet face-to-face as much as possible.

Using the Person-Based Approach to Co-Create and Optimize an App-Based Intervention to Support Better Sleep for Adolescents in the United Kingdom: Mixed Methods Study

BACKGROUND

Poor sleep is a common problem in adolescents aged 14 to 18 years. Difficulties with sleep have been found to have a bidirectional link to mental health problems.

OBJECTIVE

This new research sought to involve young people in the co-creation of a new app, particularly those from underserved communities. The Sleep Solved app uses science-based advice to improve sleep-related behaviors and well-being. The app was developed using the person-based approach, underpinned by the social cognitive theory and the social-ecological model of sleep health.

METHODS

Young people (aged 14-18 y) were recruited from across the United Kingdom to contribute to patient and public involvement (PPI) activities. In partnership with our peer researcher (MHJ), we used a multitude of methods to engage with PPI contributors, including web-based workshops, surveys, think-aloud interviews, focus groups, and app beta testing.

RESULTS

A total of 85 young people provided PPI feedback: 54 (64%) young women, 27 (32%) young men, 2 (2%) genderfluid people, 1 (1%) nonbinary person, and 1 (1%) who reported "prefer not to say." Their levels of deprivation ranged from among the 40% most deprived to the 20% least deprived areas. Most had self-identified sleep problems, ranging from 2 to 3 times per week to >4 times per week. Attitudes toward the app were positive, with praise for its usability and use of science-based yet accessible information. Think-aloud interviews and a focus group identified a range of elements that may influence the use of the app, including the need to pay attention to language choices and readability. User experiences in the form of narrated audio clips were used to normalize sleep problems and provide examples of how the app had helped these users.

CONCLUSIONS

Young people were interested in using an app to better support their sleep and mental health. The app was co-created with strong links to theory- and evidence-based sleep hygiene behaviors. Future work to establish the effectiveness of the intervention, perhaps in a randomized controlled trial, would provide support for potential UK-wide rollout.

Considerations When Designing Inclusive Digital Health Solutions for Older Adults Living With Frailty or Impairments

This viewpoint is written by authors with industrial, clinical, and academic backgrounds within medical and social sciences. The purpose is to share our experiences with digital health innovation from a sociotechnical perspective. The audience for the viewpoint is innovators, researchers, service designers, and project managers with little or some experience with theory-informed programs, complex interventions, and implementation or reorganization of sociotechnical ecosystems in health care. In digital health innovation projects, barriers related to traditions and cultures among researchers, clinicians, and industry may arise. Moreover, the final digital solutions may not always fit into existing digital ecosystems and may thus require a reorganization of how health care is provided at horizontal and vertical levels. The collaborating researchers have experience working in the field of digital health innovation for more than a decade, and we have developed and used 4 frameworks and models that are particularly relevant for theory-based complex interventions and can be used to inform inclusive co-design of digital health solutions with a sociotechnical perspective. These are (1) the 4E, a matrix to include, engage, empower, and emancipate marginalized people; (2) the GO-TO model, which can be used as a design navigator; (3) the Epital Care Model, to inform infrastructure; and (4) the Readiness and Enablement Index for Health Technology instrument, to stratify service users. From January 2021 to September 2024, we had the opportunity to apply these into practice in 4 living labs located in Denmark, Norway, the Netherlands, and Canada as a part of a European Union-funded project on "Smart Inclusive Living Environments." The goal was to cocreate a digital solution and reorganize health care services to reduce social isolation, increase health literacy, and enhance well-being for older adults living with frailty or impairments. Based on our experiences with the Smart Inclusive Living Environments project, we have formed a proposal for how design guidelines for sociotechnical innovation projects can be structured, backed up with reflections based on our experiences. With that, design guidelines should include three areas: (1) a common vocabulary including theories, frameworks, and models; (2) templates and protocols for methods, including detailed guidelines and templates for the planned development of the technologies; and (3) methods to implement and provide education and training of service users and informal and formal caregivers. In the design process, we emphasize the importance of involving relevant stakeholders in the implementation of the created design guidelines to obtain preparedness in the organizations, as well as including putative service users to ensure the likelihood of adoption. Moreover, it is important to align expectations, have a common understanding of the applied frameworks and methods, and have access to the necessary resources to reach successful results.

Discursive framework for a multi-disease digital health passport in Africa: a perspective

Africa's dual burden of rising incidence of infectious diseases and increasing prevalence of non-communicable diseases (NCDs), such as cardiovascular diseases and diabetes, demands innovative approaches to disease surveillance, response, and cross-border health management in response to growing economic integration and global connectivity. In this context, we propose a discursive framework for the development and implementation of a multi-disease digital health passport (MDDHP) in Africa. The MDDHP would serve as a secure platform for storing and sharing individual health data, offering a comprehensive solution to track and respond to infectious diseases, facilitate the management of NCDs, and improve healthcare access across borders. Empowering individuals to proactively manage their health and improve overall outcomes is a key aspect of the MDDHP. In the paper, we examine the key elements necessary to effectively implement MDDHP, focusing on minimizing risks, maintaining efficacy, and driving its adoption while also taking into consideration the unique contexts of the continent. The paper is intended to provide an understanding of the key principles involved and contribute to the discussion on the development and successful implementation of MDDHP in Africa.

Process evaluation of a Structured E-parenting Support (STEPS) in the OPTIMA randomised controlled trial: a protocol

INTRODUCTION

Structured E-parenting Support (STEPS) is a digital application (app) designed to help parents manage behaviour of their children who are referred to mental health services and are waiting for an assessment or treatment. STEPS is currently being evaluated in the Online Parent Training for the Initial Management of Attention-Deficit/Hyperactivity Disorder randomised controlled trial. Alongside the examination of STEPS' clinical and cost-effectiveness, we are conducting a process evaluation to better understand the contextual factors that may influence study outcomes. The purpose of this protocol is to describe the aims, objectives and methodology of the process evaluation prior to it taking place to add to the fidelity and rigour of the trial process and outcomes. Our goal is to adapt STEPS to optimise its benefits in future applications.

METHODS

In line with the Medical Research Council guidelines for evaluating complex interventions, the process evaluation will adopt a mixed method design using qualitative data collected from clinicians and parent interviews and app usage data from participants assigned to the intervention arm.

ANALYSIS

Qualitative data from semistructured interviews and free text box responses included in trial questionnaires will be analysed thematically using framework analysis to better understand how parents use STEPS, how it works and key factors that could aid or hinder its effective implementation in routine clinical practice.

ETHICS

The application for ethical approval for the study was submitted to the North West-Liverpool Central Research Ethics Committee and received a favourable opinion on further information on 26 November 2021, reference number 21/NW/0319.

DISSEMINATION

The process evaluation aims to explore how a digital app might support parents in managing their child's behaviour. Implications for policy and research will be explored and the clinical implications of offering the app to a wider audience to address the lack of support to parents as highlighted in this paper. We plan to publish findings in international, peer-reviewed journals as well as present at conferences.

TRIAL REGISTRATION NUMBER

The trial has been prospectively registered on 18 November 2021; ISRCTN816523503. https://www.isrctn.com/ISRCTN16523503.

"Endless opportunities": A qualitative exploration of facilitators and barriers to scale-up of two-way texting follow-up after voluntary medical male circumcision in Zimbabwe

In Zimbabwe, the ZAZIC consortium employs two-way, text-based (2wT) follow-up to strengthen post-operative care for voluntary medical male circumcision (VMMC). 2wT scaled nationally with evidence of client support and strengthened follow-up. However, 2wT uptake among healthcare providers remains suboptimal. Understanding the gap between mobile health (mHealth) potential for innovation expansion and scale-up realization is critical for 2wT and other mHealth innovations. Therefore, we conducted an exploratory qualitative study with the objective of identifying 2wT program strengths, challenges, and suggestions for scale up as part of routine VMMC services. A total of 16 in-depth interviews (IDIs) with diverse 2wT stakeholders were conducted, including nurses, monitoring & evaluation teams, and technology partners-a combination of perspectives that provide new insights. We used both inductive and deductive coding for thematic analysis. Among 2wT drivers of expansion success, interviewees noted: 2wT care benefits for clients; effective hands-on 2wT training; ease of app use for providers; 2wT saved time and money; and 2wT strengthened client/provider interaction. For 2wT scale-up challenges, staff shortages; network infrastructure constraints; client costs; duplication of paper and electronic reporting; and complexity of digital tools integration. To improve 2wT robustness, respondents suggested: more staff training to offset turnover; making 2wT free for clients; using 2wT to replace paper VMMC reporting; integrating with routine VMMC reporting systems; and expanding 2wT to other health areas. High stakeholder participation in app design, implementation strengthening, and evaluation were appreciated. Several 2wT improvements stemmed from this study, including enrollment of multiple people on one number to account for phone sharing; 2wT inclusion of minors ages 15+; clients provided with $1 to offset SMS costs; and reduced SMS messages to clients. Continued 2wT mentoring for staff, harmonization of 2wT with Ministry e-health data systems, and increased awareness of 2wT's client and provider benefits will help ensure successful 2wT scale-up.

Centering healthcare workers in digital health design: Usability and acceptability of two-way texting to improve retention in antiretroviral therapy in a public HIV clinic in Lilongwe, Malawi

New initiates on antiretroviral therapy (ART) are at high risk of treatment discontinuation, putting their health at risk. In low- and middle-income countries, like Malawi, appropriate digital health applications (apps) must fit into local clinic, connectivity and resource constraints. We describe the human centered design (HCD) and development process of an open-source, hybrid, two-way texting (2wT) system to improve ART retention. We detail the critical role of diverse healthcare workers (HCWs) in the HCD process to inform app usability, create buy-in, and ensure appropriate optimization for the local context. We optimized 2wT usability and acceptability over three HCD phases: 1) informal feedback sessions with diverse 2wT stakeholders, 2) a small pilot, and 3) key informant interviews. Phase one included four sessions with diverse HCWs, including "expert ART clients", clinical, technical, supervisory, and evaluation teams to inform 2wT design. In phase 2, a small pilot with 50 participating ART clients aimed to inform implementation improvement. Phase three included interviews with ten HCWs to deepen understanding of 2wT acceptability and usability, documenting strengths and weaknesses to inform optimization. Multi-phase feedback sessions with HCWs helped refine 2wT language and message timing for both weekly and tailored client-specific visit reminders. The pilot led to improvements in educational materials to guide client responses and ease interaction with HCWs. In interviews, the HCWs appreciated the HCD co-creation process, suggested ways to increase access for low-literacy clients or those without consistent phone access, and felt integrating 2wT with other eHealth platforms would improve scalability. Inclusion of HCWs across phases of HCD design, adaption, and optimization increased 2wT usability and acceptability among HCWs in this setting. Engaging HCWs into 2wT co-ownership from inception appears successful in co-creation of an app that will meet HCW needs, and therefore, enhance support for 2wT clients to attend visits and remain in care.

From digital health to learning health systems: four approaches to using data for digital health design

Digital health technologies, powered by digital data, provide an opportunity to improve the efficacy and efficiency of health systems at large. However, little is known about different approaches to the use of data for digital health design, or about their possible relations to system-level dynamics. In this contribution, we identify four existing approaches to the use of data for digital health design, namely the silent, the overt, the data-enabled, and the convergent. After characterising the approaches, we provide real-life examples of each. Furthermore, we compare the approaches in terms of selected desirable characteristics of the design process, highlighting relative advantages and disadvantages. Finally, we reflect on the system-level relevance of the differentiation between the approaches and point towards future research directions. Overall, the contribution provides researchers and practitioners with a broad conceptual framework to examine data-related challenges and opportunities in digital health design.

Users' involvement in digital health collaborative projects

PURPOSE

This research analyzes the roles of users in innovative digital health collaborative projects from the perspective of the user by considering three dimensions: their motivation, project activities and the support of the partnership for their effective involvement.

DESIGN/METHODOLOGY/APPROACH

The authors unraveled profiles of users by using a Q-methodological analysis of 24 statements and 44 service users. The statements for the three dimensions were designed according to previous models of stakeholder identification and customer participation in new product management.

FINDINGS

The authors obtained two profiles that advocate active participation of users, though with a different degree of involvement. One of them supports the role of users as "advisors" of users' preferences and needs, and the other indicates a higher involvement of users as "cocreators" of the innovation, with the same contribution and responsibility as the other partners.

ORIGINALITY/VALUE

Previous research has analyzed user involvement in digital health, as part of wider research on factors leading to the success and adoption of innovations. Moreover, previous research has analyzed user involvement in innovation projects, but without differentiating between projects carried out by an individual organization and those conducted by a partnership. This research contributes to filling this gap by revealing users' expectations about their involvement and how they think they will fit in with the dynamics of collaborative projects.

Digital health implementation in Australia: A scientometric review of the research

Objective

Australia is committed to establishing a digitally enabled healthcare system that fosters innovation, strengthens data capabilities, and establishes a foundation for future digital health reform. This study provides a comprehensive overview of digital health implementation research in Australia, employing scientometric analysis and data visualization. We assess the existing knowledge base, identify key research areas and frontier trends, and explore their implications for healthcare delivery in rural and remote settings.

Methods

A systematic search of the Web of Science Core Collection database was conducted for relevant documents up to December 31, 2023. Analysis of annual growth patterns, journals, institutional and authorship contributions, reference co-citation patterns, and keyword co-occurrence was conducted using scientometrics to create outputs in the form of graphs and tables. Evolutionary analyses were undertaken to delineate the current knowledge base, predominant research themes, and frontier trends in the field.

Results

A total of 196 documents related to digital health implementation in Australia were identified, demonstrating sustained growth since 2019. The evolution of the field is characterized by four distinct phases, with a pronounced focus on telehealth, particularly in the context of the COVID-19 pandemic. 'Remote health' emerged as a significant area of contemporary interest.

Conclusions

This scientometric study contributes to our understanding of digital health implementation research in Australia. Despite a considerable body of research, there remains a relative paucity of studies focused on implementation in underserved rural and remote areas which arguably stand to benefit the most from digital health advancements. Continued research in this field is crucial to ensure equitable access to the benefits offered by digital health innovations.

Burst versus continuous delivery design in digital mental health interventions: Evidence from a randomized clinical trial

Objective

Digital mental health interventions delivered via smartphone-based apps effectively treat various conditions; however, optimizing their efficacy while minimizing participant burden remains a key challenge. In this study, we investigated the potential benefits of a burst delivery design (i.e. interventions delivered only in pre-defined time intervals) in comparison to the continuous delivery of interventions.

Methods

We randomly assigned 93 participants to the continuous delivery (CD) or burst delivery (BD) group. The CD group engaged in ReApp, a mobile app that increases positive cognitive reappraisal with a consistent delivery schedule that provides five prompts per day throughout the 3-week-long study, while the BD group received five daily prompts only in the first and third weeks of the study.

Results

No significant differences were found between the groups in terms of adherence, mental health outcomes (specifically depressive and anxiety symptoms), level of perceived stress, and perceived helpfulness of intervention. The BD group showed a significantly decreased perceived difficulty of intervention over time.

Conclusions

The results suggest that the burst delivery may be as suitable for digital mental health interventions as the continuous delivery. The perceived difficulty of the intervention declined more steeply for the BD group, indicating that it improved the feasibility of the positive cognitive reappraisal intervention without hurting its efficacy. This outcome may inform the design of less burdensome interventions with improved outcomes in future research.

A Scoping Review of mHealth Interventions for Secondary Prevention of Stroke: Implications for Policy and Practice

Secondary prevention is a major priority for those living with stroke and may be improved through the use of mobile Health (mHealth) interventions. While evidence for the effectiveness of mHealth interventions for secondary prevention of stroke is growing, little attention has been given to the translation of these interventions into real-world use. In this review, we aimed to provide an update on the effectiveness of mHealth interventions for secondary prevention of stroke, and investigate their translation into real-world use. Four electronic databases and the gray literature were searched for randomized controlled trials of mHealth interventions for secondary prevention of stroke published between 2010 and 2023. Qualitative and mixed-methods evaluations of the trials were also included. Data were extracted regarding study design, population, mHealth technology involved, the intervention, and outcomes. Principal researchers from these trials were also contacted to obtain further translational information. From 1151 records, 13 randomized controlled trials and 4 evaluations were identified; sample sizes varied widely (median, 56; range, 24-4298). Short message service messages (9/13) and smartphone applications (6/13) were the main technologies used to deliver interventions. Primary outcomes of feasibility of the intervention were achieved in 4 trials, and primary outcomes of changes in risk factors, lifestyle behaviors, and adherence to medication improved in 6 trials. Only 1 trial had a hard end point (ie, stroke recurrence) as a primary outcome, and no significant differences were observed between groups. There was evidence for only 1 intervention being successfully translated into real-world use. Further evidence is required on the clinical effectiveness of mHealth interventions for preventing recurrent stroke, and the associated delivery costs and cost-effectiveness, before adoption into real-world settings.

Designing personalised mHealth solutions: An overview

INTRODUCTION

Mobile health, or mHealth, is based on mobile information and communication technologies and provides solutions for empowering individuals to participate in healthcare. Personalisation techniques have been used to increase user engagement and adherence to interventions delivered as mHealth solutions. This study aims to explore the current state of personalisation in mHealth, including its current trends and implementation.

MATERIALS AND METHODS

We conducted a review following PRISMA guidelines. Four databases (PubMed, ACM Digital Library, IEEE Xplore, and APA PsycInfo) were searched for studies on mHealth solutions that integrate personalisation. The retrieved papers were assessed for eligibility and useful information regarding integrated personalisation techniques.

RESULTS

Out of the 1,139 retrieved studies, 62 were included in the narrative synthesis. Research interest in the personalisation of mHealth solutions has increased since 2020. mHealth solutions were mainly applied to endocrine, nutritional, and metabolic diseases; mental, behavioural, or neurodevelopmental diseases; or the promotion of healthy lifestyle behaviours. Its main purposes are to support disease self-management and promote healthy lifestyle behaviours. Mobile applications are the most prevalent technological solution. Although several design models, such as user-centred and patient-centred designs, were used, no specific frameworks or models for personalisation were followed. These solutions rely on behaviour change theories, use gamification or motivational messages, and personalise the content rather than functionality. A broad range of data is used for personalisation purposes. There is a lack of studies assessing the efficacy of these solutions; therefore, further evidence is needed.

DISCUSSION

Personalisation in mHealth has not been well researched. Although several techniques have been integrated, the effects of using a combination of personalisation techniques remain unclear. Although personalisation is considered a persuasive strategy, many mHealth solutions do not employ it.

CONCLUSIONS

Open research questions concern guidelines for successful personalisation techniques in mHealth, design frameworks, and comprehensive studies on the effects and interactions among multiple personalisation techniques.

Advancing Understanding of Just-in-Time States for Supporting Physical Activity (Project JustWalk JITAI): Protocol for a System ID Study of Just-in-Time Adaptive Interventions

BACKGROUND

Just-in-time adaptive interventions (JITAIs) are designed to provide support when individuals are receptive and can respond beneficially to the prompt. The notion of a just-in-time (JIT) state is critical for JITAIs. To date, JIT states have been formulated either in a largely data-driven way or based on theory alone. There is a need for an approach that enables rigorous theory testing and optimization of the JIT state concept.

OBJECTIVE

The purpose of this system ID experiment was to investigate JIT states empirically and enable the empirical optimization of a JITAI intended to increase physical activity (steps/d).

METHODS

We recruited physically inactive English-speaking adults aged ≥25 years who owned smartphones. Participants wore a Fitbit Versa 3 and used the study app for 270 days. The JustWalk JITAI project uses system ID methods to study JIT states. Specifically, provision of support systematically varied across different theoretically plausible operationalizations of JIT states to enable a more rigorous and systematic study of the concept. We experimentally varied 2 intervention components: notifications delivered up to 4 times per day designed to increase a person's steps within the next 3 hours and suggested daily step goals. Notifications to walk were experimentally provided across varied operationalizations of JIT states accounting for need (ie, whether daily step goals were previously met or not), opportunity (ie, whether the next 3 h were a time window during which a person had previously walked), and receptivity (ie, a person previously walked after receiving notifications). Suggested daily step goals varied systematically within a range related to a person's baseline level of steps per day (eg, 4000) until they met clinically meaningful targets (eg, averaging 8000 steps/d as the lower threshold across a cycle). A series of system ID estimation approaches will be used to analyze the data and obtain control-oriented dynamical models to study JIT states. The estimated models from all approaches will be contrasted, with the ultimate goal of guiding rigorous, replicable, empirical formulation and study of JIT states to inform a future JITAI.

RESULTS

As is common in system ID, we conducted a series of simulation studies to formulate the experiment. The results of our simulation studies illustrated the plausibility of this approach for generating informative and unique data for studying JIT states. The study began enrolling participants in June 2022, with a final enrollment of 48 participants. Data collection concluded in April 2023. Upon completion of the analyses, the results of this study are expected to be submitted for publication in the fourth quarter of 2023.

CONCLUSIONS

This study will be the first empirical investigation of JIT states that uses system ID methods to inform the optimization of a scalable JITAI for physical activity.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05273437; https://clinicaltrials.gov/ct2/show/NCT05273437.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/52161.

Centering Frontline Health Care Workers in Digital Innovation Design to Inform the Optimization of an App for Improved Male Circumcision Follow-up in South Africa: Qualitative Usability Study

BACKGROUND

Voluntary medical male circumcision (VMMC) is a safe and effective HIV prevention strategy. However, adherence to recommended in-person, postoperative follow-up is inefficient for procedures with few adverse events. Two-way texting (2wT)-based follow-up appears to be a safe and an efficient alternative to scheduled clinic visits for low-risk patients who underwent VMMC. To ensure that 2wT responds to the needs of health care workers (HCWs) and patients, HCWs were closely involved in app design using a human-centered design (HCD) approach.

OBJECTIVE

Embedded within an ongoing randomized controlled trial of 2wT in South Africa and complementary HCD processes of 2wT app optimization, this qualitative study aimed to use key informant interviews (KIIs) to explore the thoughts, suggestions, and opinions on and perceptions of 2wT's usability and acceptability among HCWs involved in 2wT implementation in both urban and rural South Africa.

METHODS

A total of 7 HCWs using 2wT in Gauteng and the North West province participated in KIIs regarding the usability and acceptability of 2wT. HCWs were asked for their opinions on 2wT as a viable, useful, appropriate, and accessible method of postoperative VMMC care. They were also asked about 2wT-related working, exploring areas where 2wT could add to or reduce their daily tasks. The KII data were explored, coded, and analyzed by 3 qualitative researchers using thematic content analysis and the ATLAS.ti (ATLAS.ti Scientific Software Development GmbH) software.

RESULTS

Most HCWs felt confident, comfortable, satisfied, and well supported using a 2wT-based follow-up as an alternative to in-person clinical reviews. They felt that 2wT was easy to use and required little technical support after initial mentoring on how to use the 2wT system. Few noted safety concerns, as men can receive clinical guidance, reassurance, and referral via 2wT. Although fewer in-person visits reduced the in-person review workload and eased clinical flow, HCWs noted the added burden of having to interact with clients via SMS text messages on evenings or weekends. HCWs reinforced the need for enhanced postoperative counseling to ensure that 2wT patients could recognize and understood how to respond to early signs of complications. HCWs suggested a rotation to spread the evening and weekend workload and ensure swift patient responses.

CONCLUSIONS

In this formative qualitative study focused on HCWs, 2wT was a highly usable alternative to in-person postoperative reviews for patients who underwent VMMC in South Africa. The HCD processes likely improved the usability and acceptability of 2wT for HCWs. HCWs supported the scale-up of 2wT given the distance from the clinic to the men's homes and the potential for reducing workload. To ensure success, providers urged sensitizing patients to the fact that 2wT augments, but does not replace, the existing after-hours and emergency care services.