Evaluating the quality and safety of health-related apps and e-tools: Adapting the Mobile App Rating Scale and developing a quality assurance protocol

Mobile phone apps for adolescent and young adult symptom management during cancer treatment: a scoping review

BACKGROUND

Mobile health (mHealth) technology has rapidly developed in response to a need for reduced hospitalisation. Its use in the care of adolescents and young adults (AYAs) offers many potential advantages, including symptom tracking.

AIM

To identify and describe apps for AYA symptom management during cancer treatment.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) scoping review guidelines were followed. Six databases (Medline (OVID), CINAHL, Embase, Scopus, Web of Science, and ProQuest) were systematically searched.

RESULTS

The authors included 23 reports on 13 apps. The Memorial Symptom Assessment Scale was the most common symptom measurement tool.

CONCLUSIONS

Continuing testing of symptom management apps is warranted to ensure patient safety, with alerts and age-appropriate gamification.

mHealth Applications for Childhood Anxiety Disorders: Current Landscape, Challenges, and Future Directions

Purpose of Review

We review the literature on mHealth applications for childhood anxiety disorders, highlighting current use, limitations, and future directions.

Recent Findings

Although mHealth apps targeting childhood anxiety disorders have recently increased, most have not been evaluated for clinical effectiveness, data security, or quality standards. Many apps do not contain scientifically based content and do not adequately incorporate the principles of exposure-based cognitive behavioral therapy. In addition, many apps have significant deficiencies in age-appropriateness, data security, privacy, cultural sensitivity, and accessibility. Limitations remain in terms of the standards for evaluating apps in pediatric populations.

Summary

While interest in mHealth applications for childhood anxiety disorders is increasing, more research is needed to develop effective, safe, and age-appropriate digital interventions. Priorities include scientific grounding, privacy, equitable access, gamification, and parental involvement. Studies in this direction will increase the quality of applications and strengthen equality in access to mental health services.

mHealth Applications for Childhood Anxiety Disorders: Current Landscape, Challenges, and Future Directions

PURPOSE OF REVIEW

We review the literature on mHealth applications for childhood anxiety disorders, highlighting current use, limitations, and future directions.

RECENT FINDINGS

Although mHealth apps targeting childhood anxiety disorders have recently increased, most have not been evaluated for clinical effectiveness, data security, or quality standards. Many apps do not contain scientifically based content and do not adequately incorporate the principles of exposure-based cognitive behavioral therapy. In addition, many apps have significant deficiencies in age-appropriateness, data security, privacy, cultural sensitivity, and accessibility. Limitations remain in terms of the standards for evaluating apps in pediatric populations. While interest in mHealth applications for childhood anxiety disorders is increasing, more research is needed to develop effective, safe, and age-appropriate digital interventions. Priorities include scientific grounding, privacy, equitable access, gamification, and parental involvement. Studies in this direction will increase the quality of applications and strengthen equality in access to mental health services.

Digital Health Resilience and Well-Being Interventions for Military Members, Veterans, and Public Safety Personnel: Environmental Scan and Quality Review

BACKGROUND

Accessible mental health care, delivered via mobile apps or web-based services, may be essential for military members, public safety personnel (PSP), and veterans, as they report numerous barriers to seeking in-person care and are at an increased risk for a number of psychological disorders.

OBJECTIVE

We aimed to identify, describe, and evaluate apps, resource banks (RBs), and web-based programs (WBPs), referred to as digital mental health interventions (DMHIs), recommended for military members, PSP, and veterans. A multidimensional and multisystemic view of resilience and well-being were maintained throughout this environmental scan.

METHODS

Information was gathered from a comprehensive review of peer-reviewed literature, a Google search, and a targeted search of websites relevant to the study populations. DMHIs aimed at supporting resilience or well-being were included in the review, including those published in peer-reviewed articles, and those offered to these populations without research or literature backing their use.

RESULTS

In total, 69 DMHIs were identified in this study, including 42 apps, 19 RBs, and 8 WBPs, and were described based on 3 questions related to purpose, strategies, and evidence from the adapted Mobile App Rating Scale and the Mobile App Rating Scale. Each WBP and RB was then reviewed via the adapted Mobile App Rating Scale and each app via the Alberta Rating Index for Apps (ARIA). Overall, 24 (35%) of the DMHIs were recommended for military members, 20 (29%) for PSP, and 41 (59%) for veterans. The most common aim across apps, RBs, and WBPs was to increase happiness and well-being, and the most common strategies were advice, tips, and skills training. In total, 2 apps recommended for military members-PTSD Coach and Virtual Hope Box-received a high rating on the ARIA subscales and have also been trialed in pilot randomized control trial (RCT) and RCT evaluations, respectively, with positive initial results. Similarly, 2 apps recommended for PSP-PeerConnect and R2MR-have been trialed in non-RCT studies, with partially positive outcomes or little to no contradictory evidence and received a high rating on the ARIA. Finally, 2 apps recommended for veteran populations-PTSD Coach and VetChange-received high ratings on the ARIA and have been trialed via pilot-RCT and RCT studies, respectively, with positive outcomes.

CONCLUSIONS

In conclusion, there is a need for efficacy and effectiveness trials for DMHIs for military members, PSP, and veterans to ensure that they are effectively meeting the population's needs. While there appears to be many promising DMHIs, further research is needed before these interventions continue to be promoted as effective and widely distributed.

Design and Evaluation of ACAFiB-APP, a clinical decision support system for anticoagulant considerations in patients with atrial fibrillation

BACKGROUND

Patients with atrial fibrillation are at risk for various complications, including thromboembolic events. This study involves developing and evaluating a Clinical Decision Support System (CDSS) to select appropriate anticoagulant drug, considering comorbidities, laboratory data, and concurrent medications. The system is based on a streamlined interpretation of the most recent globally accepted clinical guidelines.

METHODS

Primarily a semi-structured interview regarding the challenges in the field of thromboprophylaxis for AF and clinical pharmacists and cardiologists' needs in practice was conducted. Then the required data were extracted from the latest guidelines and confirmed by the expert panel. Using Microsoft Visio software each scenario and its corresponding rules were modeled. Dart programming language, the Flutter framework, and the Visual Studio editor were used to develop the application. Finally, the uMARS questionnaire was used to evaluate the application quality.

RESULTS

The selection of the anticoagulants was reported to be the most challenging domain by 78.6% of the participants in the interview. According to the designed algorithms, the application was developed using Asp.net with the Microsoft SQL Server database platform. This CDSS is called ACAFiB-APP, which stands for Anticoagulant in AF Application. The user goes through various calculators and obtains the required data, moreover, the user will choose one or more comorbidities/clinical scenarios. Finally, ACAFiB-APP will represent the proper anticoagulant options with dosing and related considerations. All of the sections in the uMARS questionnaire received acceptable scores.

CONCLUSIONS

The CDSS will facilitate the informed selection of anticoagulants for complicated AF cases by considering the patient clinical scenario.

CLINICAL TRIAL NUMBER

Not applicable.

Apps and Digital Resources for Child Neurodevelopment, Mental Health, and Well-Being: Review, Evaluation, and Reflection on Current Resources

BACKGROUND

An increase in the prevalence of neurodevelopmental conditions worldwide, alongside resource constraints within clinical services, has led to increased interest in health information technologies, such as apps and digital resources. Digital tools are often viewed as a solution to bridge this divide and to increase supports for families. There is, however, a paucity of research that has evaluated digital health tools, their potential benefits for child neurodevelopment and associated concerns (eg, mental health, well-being), and their benefit for families.

OBJECTIVE

This study conducted the first review of existing mobile apps and digital resources targeted at supporting the needs of children with developmental concerns or neurodevelopmental conditions.

METHODS

We identified 3435 separate resources, of which 112 (43 apps and 69 digital resources) met the criteria. These resources were categorized according to their purpose or target and were then reviewed based on their engagement, information quality, and evidence base using the Adapted Mobile App Rating Scale.

RESULTS

The most common condition of concern targeted by apps and digital resources was autism (19/112, 17% resources), with retrieved resources focusing on supporting challenging behaviors, promoting speech, language, and social development, and providing options for alternative and assistive communication. Other common areas of concern targeted by apps and digital resources included language and communication (16/112, 14.3%) and attention-deficit/hyperactivity disorder (11/112, 9.8%). Results showed that reviewed resources were engaging, with high levels of accessibility and functionality. Resources had various functions, including developmental or behavioral tasks targeted at children, assistive communication support, scheduling support, journaling, and advice, activities, and strategies for parents. The information quality of resources, such as credibility of source and evidence base was, however, mostly low. Apps and digital resources with good credibility and an existing evidence base were largely developed in partnership with research, health, or government institutions, and were rated significantly higher on overall quality compared with apps and digital resources not developed in partnership with such institutions (apps; t41=-4.35, P<.001; digital resources; t67=-4.95, P<.001).

CONCLUSIONS

The lack of evidence base across resources means that it is extremely difficult to provide recommendations to families with respect to apps or digital resources that may support their needs. Frameworks for the development of new tools are discussed, highlighting the novel approaches required to demonstrate the efficacy of tools for improving outcomes for children and families. Such a framework requires collaboration with multiple stakeholders (software developers, researchers, regulatory bodies, clinicians, children, and families) and engagement across multiple levels of expertise (app development, implementation, and dissemination within services, policy, and clinical regulations), to harness the potential of digital health for improving outcomes and promoting support in child neurodevelopment, which at this juncture remains largely underdeveloped.

Developing and validating a content quality evaluation tool for cancer mobile applications

INTRODUCTION

Cancer remains a leading cause of death, underscoring the importance of reliable health apps. This study aims to design and validate a content quality evaluation tool for cancer mobile applications ensuring they provide standard services and features to users.

METHOD

This descriptive-analytical study was conducted in steps including identifying mobile application contents, designing and validating evaluation tool. Studies published until February 19, 2023, in PubMed, Web of Science, Scopus, and Scientific Information Database (SID) databases, as well as mobile applications in Myket, Bazaar, and Google Play, were searched. Data extraction and expert panel validation followed, leading to four expert sessions to design the tool. Nineteen oncologists, health information management and medical informatics specialties were included in the study as a research community. The tool's face validity, content validity and reliability were assessed using CVR (Content Validity Ratio), CVI (Content Validity Index), and Cronbach's alpha respectively.

RESULTS

Out of 242 initial search results in databases and 125 mobile applications in app stores, 25 articles and 20 mobile applications included based on the search limits. The designed tools consist of 8 main themes (prevention, diagnosis, treatment, follow-up, education, communication, requests/order and other) with 43 question items. All items received scores above 0.79 in CVI and 0.42 in CVR, leading to their approval and the reliability of the tool was accepted with a score of 0.967.

CONCLUSION

Evaluating cancer mobile applications can be very helpful in developing useful and supportive mobile apps, and assist health policymakers in identifying and recommending high-quality cancer-related apps. This will enable patients to choose the most suitable apps, enhancing the quality of cancer care.

What do older women want from a physical activity program? Stakeholder consultation to optimise design and recruitment for the Active Women over 50 trial

BACKGROUND

Effective recruitment and retention of participants are prerequisites for high quality physical activity intervention programs and evaluation trials, but this is underreported in the literature making it difficult to identify the most promising strategies. Incorporating stakeholder feedback in the design of program components and recruitment materials can optimise recruitment reach and the engagement of participants throughout programs and trials.

METHODS

The Active Women over 50 randomised controlled trial is testing a program designed to support women aged 50 + to be more physically active. To optimise program design and recruitment flyers, we conducted one focus group and 17 interviews with diverse purposively sampled women aged 50 + living in New South Wales, Australia. Women were asked to review recruitment flyers and the four proposed program components: (1) health coaching, (2) dedicated website with resources, (3) private Facebook group and (4) motivational email and SMS messages. Data analysis incorporated framework methods, deductive analysis using the Adapted Mobile App Rating Scale for evaluating websites and abductive analysis to critique the underlying program theory.

RESULTS

Five themes were identified in relation to recruitment: I want to see (women like) myself, Keep it real, Readability is for everyone, Why should I do it? and Find us where we live. The four program components were strongly supported as a package, but were valued differently for their relative importance. Results were used to refine the health coaching scheduling; website appearance and content; promotion and moderation of the Facebook group; and the structure, appearance and content of messages. Not all suggestions were actionable due to technological and time constraints, and the desire to keep program costs low enough for delivery at scale during the study and beyond. The program theory was expanded to encompass two emergent concepts.

CONCLUSIONS

This consultation resulted in substantial refinements to recruitment flyers and strategies, and all four program components. We anticipate that these refinements will increase the reach and appeal of the trial and optimise future scale-up. Consultation feedback, while specific to this program, may have wider transferability for recruitment and the design of programs with similar components targeting women aged 50+.

Computerised health interventions targeting Australian veterans and their families: A scoping review

ISSUE ADDRESSED

Australian veterans suffer higher rates of both mental and physical health conditions than civilians, yet many do not seek treatment. Computerised Interventions (CIs) may provide an alternative approach to management, which can overcome some barriers to treatment uptake. We aim to evaluate the scope and quality of CIs designed specifically for Australian veterans and their families.

METHODS

A manual search of the Department of Veterans' Affairs and other Ex-service organisation websites was performed to map and describe CIs for Australian veterans and their families, followed by a scoping review of four databases to identify evaluations of relevant CIs.

RESULTS

Our search identified 10 CIs specific to Australian veterans and their families. The majority were structured, self-guided CIs, designed to elicit cognitive/behavioural change that addressed mental health and psychosocial needs during transition. Three evaluations examined previously identified CIs. The results showed mixed reviews from participants and clinicians, in two separate evaluations, regarding user experience, quality and perceived benefit. In addition, positive psychological outcomes, including the reduction of post-traumatic stress disorder (PTSD) symptoms, were demonstrated for participants of the online intervention.

CONCLUSION

While Australian veteran-centric online interventions primarily focus on psychological conditions, the prevalent physical health concerns within the veteran population remain unaddressed. Additionally, despite the documented impact of military experience on family members, there is a lack of specific Australian interventions designed for families. Few tools have been formally evaluated, yet show promise as supportive self-directed resources for veterans with PTSD, and for navigating transition challenges. SO WHAT?: Further development of online interventions addressing prevalent physical and family needs, and conduction of comprehensive evaluations are needed to enhance overall quality, accessibility and holistic effectiveness of interventions for the Australian veteran community.

Feasibility of a digital palliative care intervention (Convoy-Pal) for older adults with heart failure and multiple chronic conditions and their caregivers: a waitlist randomized control trial

BACKGROUND

Although older adults with heart failure (HF) and multiple chronic conditions (MCC) frequently rely on caregivers for health management, digital health systems, such as patient portals and mobile apps, are designed for individual patients and often exclude caregivers. There is a need to develop approaches that integrate caregivers into care. This study tested the feasibility of the Social Convoy Palliative Care intervention (Convoy-Pal), a 12-week digital self-management program that includes assessment tools and resources for clinical palliative care, designed for both patients and their caregivers.

METHODS

A randomized waitlist control feasibility trial involving patients over 65 years old with MCC who had been hospitalized two or more times for HF in the past 12 months and their caregivers. Descriptive statistics were used to evaluate recruitment, retention, missing data, self-reported social functioning, positive aspects of caregiving, and the acceptability of the intervention.

RESULTS

Of 126 potentially eligible patients, 11 were ineligible and 69 were deceased. Of the 46 eligible patients, 31 enrolled in the trial. Although 48 caregivers were identified, only 15 enrolled. The average age was 76.3 years for patients and 71.6 years for caregivers, with most participants being non-Hispanic White. Notably, 4% did not have access to a personal mobile device or computer. Retention rates were 79% for intervention patients, 57% for intervention caregivers, and 60% for control participants. Only 4.6% of survey subscales were missing, aided by robust technical support. Intervention patients reported improved social functioning (SF-36: 64.6 ± 25.8 to 73.2 ± 31.3) compared to controls (64.6 ± 27.1 to 67.5 ± 24.4). Intervention caregivers also reported increased positive perceptions of caregiving (29.5 ± 5.28 to 35.0 ± 5.35) versus control caregivers (29.4 ± 8.7 to 28.0 ± 4.4). Waitlist control participants who later joined the Convoy-Pal program showed similar improvements. The intervention was well-rated for acceptability, especially regarding the information provided (3.96 ± .57 out of 5).

CONCLUSIONS

Recruiting informal caregivers proved challenging. Nonetheless, Convoy-Pal retained patients and collected meaningful self-reported outcomes, showing potential benefits for both patients and caregivers. Given the importance of a patient and caregiver approach in palliative care, further research is needed to design digital tools that cater to multiple simultaneous users.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier NCT04779931. Date of registration: March 3, 2021.

Evolutionary Trends in the Adoption, Adaptation, and Abandonment of Mobile Health Technologies: Viewpoint Based on 25 Years of Research

Over the past quarter-century, mobile health (mHealth) technologies have experienced significant changes in adoption rates, adaptation strategies, and instances of abandonment. Understanding the underlying factors driving these trends is essential for optimizing the design, implementation, and sustainability of interventions using these technologies. The evolution of mHealth adoption has followed a progressive trajectory, starting with cautious exploration and later accelerating due to technological advancements, increased smartphone penetration, and growing acceptance of digital health solutions by both health care providers and patients. However, alongside widespread adoption, challenges related to usability, interoperability, privacy concerns, and socioeconomic disparities have emerged, necessitating ongoing adaptation efforts. While many mHealth initiatives have successfully adapted to address these challenges, technology abandonment remains common, often due to unsustainable business models, inadequate user engagement, and insufficient evidence of effectiveness. This paper utilizes the Nonadoption, Abandonment, Scale-Up, Spread, and Sustainability (NASSS) framework to examine the interplay between the academic and industry sectors in patterns of adoption, adaptation, and abandonment, using 3 major mHealth innovations as examples: health-related SMS text messaging, mobile apps and wearables, and social media for health communication. Health SMS text messaging has demonstrated significant potential as a tool for health promotion, disease management, and patient engagement. The proliferation of mobile apps and devices has facilitated a shift from in-person and in-clinic practices to mobile- and wearable-centric solutions, encompassing everything from simple activity trackers to advanced health monitoring devices. Social media, initially characterized by basic text-based interactions in chat rooms and online forums, underwent a paradigm shift with the emergence of platforms such as MySpace and Facebook. This transition ushered in an era of mass communication through social media. The rise of microblogging and visually focused platforms such as Twitter(now X), Instagram, Snapchat, and TikTok, along with the integration of live streaming and augmented reality features, exemplifies the ongoing innovation within the social media landscape. Over the past 25 years, there have been remarkable strides in the adoption and adaptation of mHealth technologies, driven by technological innovation and a growing recognition of their potential to revolutionize health care delivery. Each mobile technology uniquely enhances public health and health care by catering to different user needs. SMS text messaging offers wide accessibility and proven effectiveness, while mobile apps and wearables provide comprehensive functionalities for more in-depth health management. Social media platforms amplify these efforts with their vast reach and community-building potential, making it essential to select the right tool for specific health interventions to maximize impact and engagement. Nevertheless, continued efforts are needed to address persistent challenges and mitigate instances of abandonment, ensuring that mHealth interventions reach their full potential in improving health outcomes and advancing equitable access to care.

Assessing the Utility of a Patient-Facing Diagnostic Tool Among Individuals With Hypermobile Ehlers-Danlos Syndrome: Focus Group Study

BACKGROUND

Hypermobile Ehlers-Danlos syndrome (hEDS), characterized by joint hypermobility, skin laxity, and tissue fragility, is thought to be the most common inherited connective tissue disorder, with millions affected worldwide. Diagnosing this condition remains a challenge that can impact quality of life for individuals with hEDS. Many with hEDS describe extended diagnostic odysseys involving exorbitant time and monetary investment. This delay is due to the complexity of diagnosis, symptom overlap with other conditions, and limited access to providers. Many primary care providers are unfamiliar with hEDS, compounded by genetics clinics that do not accept referrals for hEDS evaluation and long waits for genetics clinics that do evaluate for hEDS, leaving patients without sufficient options.

OBJECTIVE

This study explored the user experience, quality, and utility of a prototype of a patient-facing diagnostic tool intended to support clinician diagnosis for individuals with symptoms of hEDS. The questions included within the prototype are aligned with the 2017 international classification of Ehlers-Danlos syndromes. This study explored how this tool may help patients communicate information about hEDS to their physicians, influencing the diagnosis of hEDS and affecting patient experience.

METHODS

Participants clinically diagnosed with hEDS were recruited from either a medical center or private groups on a social media platform. Interested participants provided verbal consent, completed questionnaires about their diagnosis, and were invited to join an internet-based focus group to share their thoughts and opinions on a diagnostic tool prototype. Participants were invited to complete the Mobile App Rating Scale (MARS) to evaluate their experience viewing the diagnostic tool. The MARS is a framework for evaluating mobile health apps across 4 dimensions: engagement, functionality, esthetics, and information quality. Qualitative data were analyzed using affinity mapping to organize information and inductively create themes that were categorized within the MARS framework dimensions to help identify strengths and weaknesses of the diagnostic tool prototype.

RESULTS

In total, 15 individuals participated in the internet-based focus groups; 3 (20%) completed the MARS. Through affinity diagramming, 2 main categories of responses were identified, including responses related to the user interface and responses related to the application of the tool. Each category included several themes and subthemes that mapped well to the 4 MARS dimensions. The analysis showed that the tool held value and utility among the participants diagnosed with hEDS. The shareable ending summary sheet provided by the tool stood out as a strength for facilitating communication between patient and provider during the diagnostic evaluation.

CONCLUSIONS

The results provide insights on the perceived utility and value of the tool, including preferred phrasing, layout and design preferences, and tool accessibility. The participants expressed that the tool may improve the hEDS diagnostic odyssey and help educate providers about the diagnostic process.

Improving the Well-Being of People With Advanced Cancer and Their Family Caregivers: Protocol for an Effectiveness-Implementation Trial of a Dyadic Digital Health Intervention (FOCUSau)

BACKGROUND

Advanced cancer significantly impacts patients' and family caregivers' quality of life. When patients and caregivers are supported concurrently as a dyad, the well-being of each person is optimized. Family, Outlook, Communication, Uncertainty, Symptom management (FOCUS) is a dyadic, psychoeducational intervention developed in the United States, shown to improve the well-being and quality of life of patients with advanced cancer and their primary caregivers. Originally, a nurse-delivered in-person intervention, FOCUS has been adapted into a self-administered web-based intervention for European delivery.

OBJECTIVE

The aims of this study are to (1) adapt FOCUS to the Australian context (FOCUSau); (2) evaluate the effectiveness of FOCUSau in improving the emotional well-being and self-efficacy of patients with advanced cancer and their primary caregiver relative to usual care control group; (3) compare health care use between the intervention and control groups; and (4) assess the acceptability, feasibility, and scalability of FOCUSau in order to inform future maintainable implementation of the intervention within the Australian health care system.

METHODS

FOCUS will be adapted prior to trial commencement, using an iterative stakeholder feedback process to create FOCUSau. To examine the efficacy and cost-effectiveness of FOCUSau and assess its acceptability, feasibility, and scalability, we will undertake a hybrid type 1 implementation study consisting of a phase 3 (clinical effectiveness) trial along with an observational implementation study. Participants will include patients with cancer who are older than 18 years, able to access the internet, and able to identify a primary support person or caregiver who can also be approached for participation. The sample size consists of 173 dyads in each arm (ie, 346 dyads in total). Patient-caregiver dyad data will be collected at 3 time points-baseline (T0) completed prerandomization; first follow-up (T1; N=346) at 12 weeks post baseline; and second follow-up (T2) at 24 weeks post baseline.

RESULTS

The study was funded in March 2022. Recruitment commenced in July 2024.

CONCLUSIONS

If shown to be effective, this intervention will improve the well-being of patients with advanced cancer and their family caregivers, regardless of their location or current level of health care support.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06082128; https://clinicaltrials.gov/study/NCT06082128.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/55252.

Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study)

BACKGROUND

Artificial intelligence (AI)-based chatbots could help address some of the challenges patients face in acquiring information essential to their self-health management, including unreliable sources and overburdened health care professionals. Research to ensure the proper design, implementation, and uptake of chatbots is imperative. Inclusive digital health research and responsible AI integration into health care require active and sustained patient and stakeholder engagement, yet corresponding activities and guidance are limited for this purpose.

OBJECTIVE

In response, this manuscript presents a master protocol for the development, testing, and implementation of a chatbot family in partnership with stakeholders. This protocol aims to help efficiently translate an initial chatbot intervention (MARVIN) to multiple health domains and populations.

METHODS

The MARVIN chatbots study has an adaptive platform trial design consisting of multiple parallel individual chatbot substudies with four common objectives: (1) co-construct a tailored AI chatbot for a specific health care setting, (2) assess its usability with a small sample of participants, (3) measure implementation outcomes (usability, acceptability, appropriateness, adoption, and fidelity) within a large sample, and (4) evaluate the impact of patient and stakeholder partnerships on chatbot development. For objective 1, a needs assessment will be conducted within the setting, involving four 2-hour focus groups with 5 participants each. Then, a co-construction design committee will be formed with patient partners, health care professionals, and researchers who will participate in 6 workshops for chatbot development, testing, and improvement. For objective 2, a total of 30 participants will interact with the prototype for 3 weeks and assess its usability through a survey and 3 focus groups. Positive usability outcomes will lead to the initiation of objective 3, whereby the public will be able to access the chatbot for a 12-month real-world implementation study using web-based questionnaires to measure usability, acceptability, and appropriateness for 150 participants and meta-use data to inform adoption and fidelity. After each objective, for objective 4, focus groups will be conducted with the design committee to better understand their perspectives on the engagement process.

RESULTS

From July 2022 to October 2023, this master protocol led to four substudies conducted at the McGill University Health Centre or the Centre hospitalier de l'Université de Montréal (both in Montreal, Quebec, Canada): (1) MARVIN for HIV (large-scale implementation expected in mid-2024), (2) MARVIN-Pharma for community pharmacists providing HIV care (usability study planned for mid-2024), (3) MARVINA for breast cancer, and (4) MARVIN-CHAMP for pediatric infectious conditions (both in preparation, with development to begin in early 2024).

CONCLUSIONS

This master protocol offers an approach to chatbot development in partnership with patients and health care professionals that includes a comprehensive assessment of implementation outcomes. It also contributes to best practice recommendations for patient and stakeholder engagement in digital health research.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05789901; https://classic.clinicaltrials.gov/ct2/show/NCT05789901.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54668.

The Effectiveness of Preoperative Outpatient and Home Rehabilitation and the Impact on the Results of Hip Arthroplasty: Introductory Report

A total hip replacement is the treatment of choice for end-stage hip osteoarthritis. Rehabilitation performed before surgery (called prehabilitation) is used to improve the results of surgical treatment. However, the results of studies have not unquestionably confirmed the effectiveness of preoperative rehabilitation and its impact on the outcome of surgery. The aim of this study is to assess the effectiveness of preoperative outpatient and home rehabilitation in relation to a control group not subject to these forms of influence. A total of 61 patients qualified for primary hip arthroplasty were randomly assigned to a group with outpatient rehabilitation before surgery, exercises performed at home, or a group without any intervention before surgery. Three weeks after surgery, the patients were re-qualified and underwent three weeks of outpatient rehabilitation in the day rehabilitation department. The patients from all three groups were evaluated in terms of functionality and pain using point scales upon enrolment in the study, on admission to the day rehabilitation department, and after 3 weeks of rehabilitation in the department. A total of 50 subjects completed the study. The study results did not reveal statistically significant differences between preoperative rehabilitation and no intervention. Patients rehabilitated at home gave up self-therapy more often than those undergoing outpatient rehabilitation.

Implementation of an Electronic Mental Health Platform for Youth and Young Adults in a School Context Across Alberta, Canada: Thematic Analysis of the Perspectives of Stakeholders

BACKGROUND

Youth, aged 15 to 24 years, are more likely to experience mental health (MH) or substance use issues than other age groups. This is a critical period for intervention because MH disorders, if left unattended, may become chronic and serious and negatively affect many aspects of a young person's life. Even among those who are treated, poor outcomes will still occur for a percentage of youth. Electronic MH (eMH) tools have been implemented in traditional MH settings to reach youth requiring assistance with MH and substance use issues. However, the utility of eMH tools in school settings has yet to be investigated.

OBJECTIVE

The objective of this study was to gain an understanding of the perspectives of key school staff stakeholders regarding barriers and facilitators to the implementation of the Innowell eMH platform in secondary schools across the province of Alberta, Canada.

METHODS

Guided by a qualitative descriptive approach, focus groups were conducted to elicit stakeholder perspectives on the perceived implementation challenges and opportunities of embedding the Innowell eMH platform in secondary school MH services. In total, 8 focus groups were conducted with 52 key school staff stakeholders.

RESULTS

Themes related to barriers and facilitators to youth and school MH care professional (MHCP) capacity in implementing and using eMH tools were identified. With respect to youth capacity barriers, the following themes were inductively generated: (1) concerns about some students not being suitable for eMH services, (2) minors requiring consent from parents or caregivers to use eMH services as well as confidentiality and privacy concerns, and (3) limited access to technology and internet service among youth. A second theme related to school MHCP barriers to implementation, which included (1) feeling stretched with high caseloads and change fatigue, (2) concerns with risk and liability, and (3) unmasking MH issues in the face of limited resources. In contrast to the barriers to youth and MHCP capacity, many facilitators to implementation were discussed. Youth capacity facilitators included (1) the potential for youth to be empowered using eMH tools, (2) the platform fostering therapeutic relationships with school personnel, and (3) enhancing access to needed services and resources. MHCP capacity facilitators to implementation were (1) system transformation through flexibility and problem-solving, (2) opportunities for collaboration with youth and MHCPs and across different systems, and (3) an opportunity for the continuity of services.

CONCLUSIONS

Our findings highlight nuanced school MHCP perspectives that demonstrate critical youth and MHCP capacity concerns, with consideration for organizational factors that may impede or enhance the implementation processes for embedding eMH in a school context. The barriers and facilitators to implementation provide future researchers and decision makers with challenges and opportunities that could be addressed in the preimplementation phase.

Assessing the Quality, Privacy, and Security of Breast Cancer Apps for Arabic Speakers: Systematic Search and Review of Smartphone Apps

BACKGROUND

Breast cancer is a widespread disease, and its incidence is rapidly increasing in the Middle East and North Africa region. With the increasing availability of smartphone apps for various health purposes, breast cancer apps have emerged as tools for raising awareness, providing support, and empowering women affected by this disease. These apps offer many features, including information on breast cancer risk factors, self-examination guides, appointment reminders, and community support groups or hotlines. Using apps raises the risk of privacy and security issues, and we hope that examining these features of the apps will contribute to the understanding of how technology can be used to improve these apps and provide insights for future development and improvement of breast cancer apps.

OBJECTIVE

This study aims to critically review the quality, privacy, and security of breast cancer apps available to Arabic speakers.

METHODS

Similar to several recent studies, we used a systematic search for apps available in Google Play and Apple App stores using both the web interface and the built-in native stores installed on smartphones. The search was conducted in mid-December 2022 in Arabic using the following keywords: سرطان الثدي - فحص سرطان الثدي - علاج سرطان الثدي - مرض سرطان الثدي - أعراض سرطان الثدي - فحص الثدي (breast cancer, breast cancer treatment, breast cancer disease, breast cancer symptoms, breast cancer screening, and breast test). These preidentified search terms are based on earlier work concerning the top searched breast cancer topics by Arabic speakers through Google's search engine. We excluded apps that did not have an Arabic interface, were developed for non-Arabic speakers, were paid, needed a subscription, or were directed toward health care workers. The Mobile App Rating Scale was used to evaluate the quality of the apps concerning their engagement, functionality, aesthetics, and information. A risk score was calculated for the apps to determine their security risk factors.

RESULTS

Only 9 apps were included, with most (6/9, 67%) being supported by advertisements and categorized as informational. Overall, the apps had low numbers of downloads (>10 to >1000). The majority of the included apps (8/9, 89%) requested dangerous access permissions, including access to storage, media files, and the camera. The average security score of the included apps was 3.22, while only 2 apps provided information about data security and privacy. The included apps achieved an overall average quality score of 3.27, with individual dimension scores of 4.75 for functionality, 3.04 for information, 3.00 for aesthetics, and 2.32 for engagement.

CONCLUSIONS

The limited availability of breast cancer apps available to Arabic speakers should be a call to action and prompt health care organizations and developers to join forces and collaboratively develop information-rich, usable, functional, engaging, and secure apps.

A sociotechnical framework to assess patient-facing eHealth tools: results of a modified Delphi process

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

Electronic Medication Reconciliation Tools Aimed at Healthcare Professionals to Support Medication Reconciliation: a Systematic Review

The development of health information technology available and accessible to professionals is increasing in the last few years. However, a low number of electronic health tools included some kind of information about medication reconciliation. To identify all the electronic medication reconciliation tools aimed at healthcare professionals and summarize their main features, availability, and clinical impact on patient safety. A systematic review of studies that included a description of an electronic medication reconciliation tool (web-based or mobile app) aimed at healthcare professionals was conducted. The review protocol was registered with PROSPERO: registration number CRD42022366662, and followed PRISMA guidelines. The literature search was performed using four healthcare databases: PubMed, EMBASE, Cochrane Library, and Scopus with no language or publication date restrictions. We identified a total of 1227 articles, of which only 12 met the inclusion criteria.Through these articles,12 electronic tools were detected. Viewing and comparing different medication lists and grouping medications into multiple categories were some of the more recurring features of the tools. With respect to the clinical impact on patient safety, a reduction in adverse drug events or medication discrepancies was detected in up to four tools, but no significant differences in emergency room visits or hospital readmissions were found. 12 e-MedRec tools aimed at health professionals have been developed to date but none was designed as a mobile app. The main features that healthcare professionals requested to be included in e-MedRec tools were interoperability, "user-friendly" information, and integration with the ordering process.

Assessment of healthcare application quality: Development of a standardized methods for healthcare professionals

The advancement of the mobile app market is reshaping healthcare, emphasizing the imperative for quality and efficacy in health applications. This methodology has been devised to assess mobile health applications, aiming to assist healthcare professionals in selecting apps for e-healthcare consumers. Key facets of this approach are: •A stringent selection process within mobile app stores•A standardized assessment using the Mobile App Rating Scale to achieve consistent and replicable evaluations, systematically organizing app evaluations•A comprehensive framework guiding healthcare practitioners in determining which apps to integrate into clinical practice and which to exclude Central to this method is the emphasis on distinguishing apps that enhance clinical practice from those that fall short in important areas such as the effectiveness of proposed health features, data accuracy, adherence to clinical guidelines, data security, and user privacy. With heightened attention to usability and accessibility, the methodology also addresses the common risks inherent in mHealth implementation, ensuring that selected apps not only meet technical criteria but also align with the broader healthcare ecosystem's needs and challenges.

Customized virtual reality naturalistic scenarios promoting engagement and relaxation in patients with cognitive impairment: a proof-of-concept mixed-methods study

Being immersed in a natural context has a beneficial and pervasive impact on well-being. Virtual Reality (VR) is a technology that can help expose people to naturalistic scenarios virtually, overcoming obstacles that prevent them from visiting real natural environments. VR could also increase engagement and relaxation in older adults with and without cognitive impairment. The main aim of this study is to investigate the feasibility of a customized naturalistic VR scenario by assessing motion-sickness effects, engagement, pleasantness, and emotions felt. Twenty-three individuals with a diagnosis of cognitive impairment living in a long-term care home participated in our study. At the end of the entire VR experimental procedure with older adults, five health staff operators took part in a dedicated assessment phase focused on evaluating the VR procedure's usability from their individual perspectives. The tools administered were based on self-reported and observational tools used to obtain information from users and health care staff professionals. Feasibility and acceptance proved to be satisfactory, considering that the VR experience was well-tolerated and no adverse side effects were reported. One of the major advantages emerged was the opportunity to deploy customized environments that users are not able to experience in a real context.Trial Registration: National Institute of Health (NIH) U.S. National Library of Medicine, ClinicalTrials.gov NCT05863065 (17/05/2023).

Parent and clinician perceptions and recommendations on a pediatric cancer pain management app: A qualitative co-design study

Pain is one of the most prevalent and burdensome pediatric cancer symptoms for young children and their families. A significant proportion of pain episodes are experienced in environments where management options are limited, including at home. Digital innovations such as apps may have positive impacts on pain outcomes for young children in these environments. Our overall aim is to co-design such an app and the objective of this study was to explore the perceptions of children's parents about app utility, needed system features, and challenges. We recruited parents of young children with cancer and multidisciplinary pediatric oncology clinicians from two pediatric cancer care centers to participate in audio-recorded, semi-structured, co-design interviews. We conducted interviews structured around technology acceptance and family caregiving theories until data saturation was reached. Audio-recordings were then transcribed, coded, and analyzed using thematic analysis. Forty-two participants took part in the process. Participants endorsed the concept of an app as a useful, safe, and convenient way to engage caregivers in managing their young child's pain. Overall, the app was valued as a means to provide real-time, multimodal informational and procedural pain support to parents, while also reducing the emotional burden of pain care. Recommendations for intervention design included accessibility-focused features, comprehensive symptom tracking, and embedded scientific- and clinically-sound symptom assessments and management advice. Predicted challenges to app use included the workload burden it may place on parents and clinicians. The insights gathered will inform the design principles of our future childhood cancer pain digital research.

Impact of Apps as Assistive Devices for Visually Impaired Persons

The pervasiveness of mobile devices and other associated technologies has affected all aspects of our daily lives. People with visual impairments are no exception, as they increasingly tend to rely on mobile apps for assistance with various visual tasks in daily life. Compared to dedicated visual aids, mobile apps offer advantages such as affordability, versatility, portability, and ubiquity. We have surveyed hundreds of mobile apps of potential interest to people with vision impairments, either released as special assistive apps claiming to help in tasks such as text or object recognition (n = 68), digital accessibility (n = 84), navigation (n = 44), and remote sighted service (n = 4), among others, or marketed as general camera magnification apps that can be used for visual assistance (n = 77). While assistive apps as a whole received positive feedback from visually impaired users, as reported in various studies, evaluations of the usability of every app were typically limited to user reviews, which are often not scientifically informative. Rigorous evaluation studies on the effect of vision assistance apps on daily task performance and quality of life are relatively rare. Moreover, evaluation criteria are difficult to establish, given the heterogeneity of the visual tasks and visual needs of the users. In addition to surveying literature on vision assistance apps, this review discusses the feasibility and necessity of conducting scientific research to understand visual needs and methods to evaluate real-world benefits.

Importance-performance analysis in fitness apps. A study from the viewpoint of gender and age

Background

We are currently undergoing a profound process of digital transformation that has favoured the development and use of apps in multiple facets of people's daily lives. In the fitness industry, this situation has facilitated the control of exercise and the maintenance of healthier lifestyles. However, it is not known how the perceived quality and importance of fitness apps vary for users based on gender and age, which is the objective of this study conducted among users of fitness centres.

Methods

By means of a convenience sample, 321 users from different centres of the boutique fitness chain Sano Centre (238 females and 83 males) took part in the study. They answered the 16 items of the MARS scale, distributed in four dimensions, in terms of importance and perceived quality. The existence of significant differences was analysed using non-parametrical statistics such as the U-Mann-Whitney (gender) and the H-Kruskal-Wallis (age). In addition, a cluster analysis, combining hierarchical and non-hierarchical methods, was analysed considering as a dependent variable the level of recommendation of fitness apps.

Results

Considering gender, in importance-performance analysis (IPA), credibility was the most important attribute for females and quality of information for males. In the case of age, credibility was the most important attribute in all the ranges. The cluster analysis established two groups (high and low recommendations of the fitness app). In importance, the first group scored better on all factors except entertainment and interest. In valuation, the scores were lower than on importance, especially in the low recommendation group.

Conclusion

Regarding usage behaviour, credibility is the factor to which users attach the highest importance and rating, regardless of gender and age. The main demand focuses on improving the gestural design and visual appeal, which will facilitate a better user experience.

Design, Development and Evaluation of an Application based on Clinical Decision Support Systems (CDSS) for Over-The-Counter (OTC) Therapy: An Educational Interventions in Community Pharmacists

INTRODUCTION

Mobile health (mHealth) technology-based applications provide strong medical health-care support. Applications have an important impact as tools to improve the knowledge and support the health-care team practice. In this study, an over-the-counter (OTC) therapy application was developed based on Clinical Decision Support Systems (CDSS). CDSS is a key to improve health-related decisions and healthcare delivery. Furthermore, the quality and effectiveness of this application were evaluated among community pharmacists.

METHODS

The application was designed and developed for 10 topics of OTC therapy. After the approval of the expert panel, 40 pharmacists affiliated with Tehran University of Medical Science (TUMS) participated in this before and after quasi-experimental study. The related scenarios and checklists were designed for the ten topics. The participants had to manage the scenarios first by their knowledge and then with the application. The knowledge and pharmaceutical skills in OTC therapy were evaluated based on the obtained scores and the time recorded. The quality of the application was evaluated by pharmacists using user version of mobile application rating scale (uMARS) questionnaire. To compare before/after measurements of parametric and non-parametric data, we used the paired t-test and Wilcoxon matched-pairs test, respectively. Besides, the variables was compared using Mann-Whitney test. The statistical significance was considered at P<0.05. The analyses were performed using the statistical software Stata (ver. 13).

RESULTS

All scores after using the application increased, and the P-value was not significant. Also, the recorded time was increased after the use of the application, and the P-value was not significant. The minimum mean scores of the six uMARS questionnaire sections were 3. It means that acceptable scores were obtained in all sections of the questionnaire. The "App quality score" section of the application was reported 3.45±0.94. No relationship was found between gender and the median score of each section of the uMARS questionnaire.

CONCLUSION

The OTC therapy application developed in this study will help Persian-speaking pharmacists to increase their knowledge and pharmaceutical skills.

Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis

BACKGROUND

Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges.

OBJECTIVE

This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives).

METHODS

A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes.

RESULTS

Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer.

CONCLUSIONS

This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users' adoption to ensure uptake and adherence in the long term.

A Systematic Assessment of the Quality of Smartphone Applications for Gastroesophageal Reflux Disease

Background and Aims

Smartphone applications aimed at patients with gastroesophageal reflux disease (GERD) have been downloaded more than 100,000 times, yet no systematic assessment of their quality has been completed. This study aimed to objectively assess the quality of GERD smartphone applications for patient education and disease management.

Methods

The Apple App Store and Google Play Store were systematically searched for relevant applications. Two independent reviewers performed the application screening and eligibility assessment. Included applications were graded using the validated Mobile Application Rating Scale, which encompasses 4 domains (engagement, functionality, aesthetics, and information) as well as an overall application quality score. The associations between overall application quality, user ratings and download numbers were evaluated.

Results

Of the 4816 unique applications identified, 46 met inclusion criteria (patient education = 37, disease management = 9). Mean overall application quality score was 3.02 ± 0.40 out of 5 ("acceptable"), with 61% (28/46) rated as "poor" (score 2.0-2.9). Applications scored highest for aesthetics (3.24 ± 0.48) and functionality (3.88 ± 0.37) and lowest for information (2.58 ± 0.64) and engagement (2.39 ± 0.65). Disease management applications were of significantly higher quality than education-focused applications (3.59 ± 0.38 vs 2.88 ± 0.26, P < .001). There was no correlation between graded quality and either user ratings or the number of downloads.

Conclusion

While numerous smartphone applications exist to support patients with GERD, their quality is variable. Patient education applications are of particularly low quality. Our findings can help to inform the selection of applications by patients and guide clinicians' recommendations. This study also highlights the need for higher-quality, evidence-informed applications aimed at GERD patient education.

Exploring how informed mental health app selection may impact user engagement and satisfaction

The prevalence of mental health app use by people suffering from mental health disorders is rapidly growing. The integration of mental health apps shows promise in increasing the accessibility and quality of treatment. However, a lack of continued engagement is one of the significant challenges of such implementation. In response, the M-health Index and Navigation Database (MIND)- derived from the American Psychiatric Association's app evaluation framework- was created to support patient autonomy and enhance engagement. This study aimed to identify factors influencing engagement with mental health apps and explore how MIND may affect user engagement around selected apps. We conducted a longitudinal online survey over six weeks after participants were instructed to find mental health apps using MIND. The survey included demographic information, technology usage, access to healthcare, app selection information, System Usability Scale, the Digital Working Alliance Inventory, and the General Self-Efficacy Scale questions. Quantitative analysis was performed to analyze the data. A total of 321 surveys were completed (178 at the initial, 90 at the 2-week mark, and 53 at the 6-week mark). The most influential factors when choosing mental health apps included cost (76%), condition supported by the app (59%), and app features offered (51%), while privacy and clinical foundation to support app claims were among the least selected filters. The top ten apps selected by participants were analyzed for engagement. Rates of engagement among the top-ten apps decreased by 43% from the initial to week two and 22% from week two to week six on average. In the context of overall low engagement with mental health apps, implementation of mental health app databases like MIND can play an essential role in maintaining higher engagement and satisfaction. Together, this study offers early data on how educational approaches like MIND may help bolster mental health apps engagement.

Assessing the Dynamics of the Mental Health Apple and Android App Marketplaces

Barriers to mental health care, including stigma, costs, and mental health professional shortages, have been exacerbated by the COVID-19 pandemic. Smartphone apps have the potential to increase scalability and improve access to mental health information, support, and interventions. However, evaluating these apps and selecting ones for use in care remain challenging, especially as apps are often updating and changing. Recommending apps requires knowledge of how stable apps are as the experience of one user several months ago may or may not be the same. A sample of 347 apps of the 650 apps on the M-health Index and Navigation Database (MIND) https://mindapps.org were reviewed between September 1, 2021, and January 5, 2022. Apps were selected by time since their last review, with updates occurring on average approximately 4 months from the last review. Eleven trained app evaluators reviewed apps across 105 evaluation criteria in 9 categories. Results were compared to initial ratings, identifying the changes that occurred. The average app updates every 433 days, though 19% were updated in the last 3 months and some nearly weekly. Changes in privacy and features made up the highest percentage of changes, both at 38%. The most frequently observed privacy-related change was increased privacy policy reading level. Functionality parameters changed in 28% of apps. The most common functionality change was the removal of an accessibility feature. Clinical foundations changed in 18% of apps and 9% added supporting studies. Cost structure changed in 17% of apps, with 10% adding a fee for use of the app. Engagement features changed in 17% of the apps, with additions and removals of validated assessments or screeners most common. The dynamic nature of the app stores is reflected in app privacy, features, and functionality. These changes, reflected by the increased reading levels required to understand privacy policies, the decrease in accessibility features, and the additions of fees to access mobile apps, reflect the need to constantly review apps and understand how they are evolving. Patient and clinicians should use the most recent and updated possible when evaluating apps.

Current challenges for evaluating mobile health applications

As mobile health applications continue to proliferate without clear regulation, the need for app evaluation frameworks to offer guidance to patients and clinicians also expands. However, this expanding number of app evaluation frameworks itself can be a source of confusion and often contradictory recommendations. In pursuit of better frameworks that offer innovation for app evaluation, we present 4 challenges that app evaluation frameworks must overcome as well as examples from our own experience toward overcoming them. The recommendations are applicable to all health apps from any field of medicine, although we use examples from mental health as they are illustrative.

Proposal of a New Rating Concept for Digital Health Applications in Orthopedics and Traumatology

BACKGROUND

Health-related mobile applications (apps) are rapidly increasing in number. There is an urgent need for assessment tools and algorithms that allow the usability and content criteria of these applications to be objectively assessed. The aim of this work was to establish and validate a concept for orthopedic societies to rate health apps to set a quality standard for their safe use.

METHODS

An objective rating concept was created, consisting of nine quality criteria. A self-declaration sheet for app manufacturers was designed. Manufacturers completed the self-declaration, and the app was examined by independent internal reviewers. The pilot validation and analysis were performed on two independent health applications. An algorithm for orthopedic societies was created based on the experiences in this study flow.

RESULTS

"Sprunggelenks-App" was approved by the reviewers with 45 (98%) fulfilled criteria and one (2%) unfulfilled criterion. "Therapie-App" was approved, with 28 (61%) met criteria, 6 (13%) unfulfilled criteria and 12 (26%) criteria that could not be assessed. The self-declaration completed by the app manufacturer is recommended, followed by a legal and technical rating performed by an external institution. When rated positive, the societies' internal review using independent raters can be performed. In case of a positive rating, a visual certification can be granted to the manufacturer for a certain time frame.

CONCLUSION

An objective rating algorithm is proposed for the assessment of digital health applications. This can help societies to improve the quality assessment, quality assurance and patient safety of those apps. The proposed concept must be further validated for inter-rater consistency and reliability.

Systematic assessment of the quality and integrity of popular mental health smartphone apps using the American Psychiatric Association's app evaluation model

Mobile phones are playing an increasingly important role in supporting mental health, by providing confidential, accessible and scalable support for individuals who may not seek or have means of accessing professional help. There are concerns, however, that many apps claiming to support mental health do not meet professional, ethical or evidence-based standards. App store search algorithms favour popularity (reviews and downloads) and commercial factors (in-app purchases), with what appears to be low prioritisation of safety or effectiveness features. In this paper, the most visible 100 apps for “depression”, “anxiety” and/or “mood” on the Google Play and Apple App stores were selected for assessment using the American Psychiatric Association App Evaluation model. This model systematically assesses apps across five broad steps: accessibility, integrity, clinical and research evidence base, user engagement and interoperability. Assessment is hierarchical, with the most fundamental requirements of apps assessed first, with apps excluded at each step if they do not meet the criteria. The relationship between app quality and app store visibility was first analysed. App quality was also compared across four different app function types: mental health promotion or psychoeducation; monitoring or tracking; assessment or prevention; and intervention or treatment. Of the 92 apps assessed (after eight failed to meet inclusion criteria), half failed to meet the first criterion step of accessibility, and a further 20% of the remaining apps failed to meet the second criterion step of security and privacy. Only three of the 10 apps most visible on app stores met the criteria for research/clinical base and engagement/ease of use, and only one app fulfilled all five criterion steps of the evaluation model. Quality did not differ significantly across app function type. There was no significant correlation between app quality and app store visibility, which presents a potential risk to vulnerable consumers. The findings of this review highlight the need for greater accountability of app developers to meet, and report, at least minimum quality and integrity standards for their apps. Recommendations are also provided to assist users and clinicians to make informed choices in their selection of reputable and effective mental health apps.

Exploring the Use of a Web-Based Menu Planning Tool in Childcare Services: Qualitative Cross-sectional Survey Study

Background

Early childhood is a critical period for supporting the development of healthy eating habits, which may affect lifelong health. Childcare services are important settings for promoting early childhood nutrition; however, food provision in childcare frequently does not align with dietary guidelines. Web-based menu planning tools are well suited to support healthy food provision in childcare, although little is known about their use. Research is needed to understand how web-based menu planning tools are used in the childcare setting and how they can effectively support healthy menu planning and food provision for children in childcare.

Objective

We aimed to explore the use of a web-based menu planning tool called FoodChecker, which is available to childcare services in Victoria, Australia. We also aimed to gain insights and perspectives from childcare staff involved in menu planning about their use of the tool to plan healthy menus and guide healthy food provision for children.

Methods

We conducted a qualitative descriptive study using a cross-sectional web-based survey completed by the staff involved in menu planning in childcare services. Thematic analysis was performed using NVivo software. Emergent themes were mapped against constructs of the Technology Acceptance Model regarding perceived usefulness, perceived ease of use, and external variables influencing perceptions and use.

Results

The participants included 30 cooks and 34 directors from 53 childcare services. Participants perceived the web-based menu planning tool as useful for supporting child nutrition and health, improving organizational processes, and aiding the menu planner role. Perceptions regarding ease of use were mixed. External variables influencing perceptions and use included awareness of the tool, perceived need, time, resources, organizational support, and the food budget. Participants made recommendations to improve the tool, particularly the need to integrate functionality to make it easier and faster to use or to include more links to resources to support healthy menu planning.

Conclusions

The web-based menu planning tool was perceived as useful for cooks and directors in childcare services. Areas for improvement were identified; for example, the need for integrated digital features to make the tool easier and faster to use. As the first qualitative study to explore childcare staff experiences with a web-based menu planning tool, these findings inform future research and development of such tools to aid scalable and sustainable support for healthier food provision in the childcare sector.

Stimuli Influencing Engagement, Satisfaction, and Intention to Use Telemedicine Services: An Integrative Model

Telemedicine ensures quality, cost-effective, and equally accessible healthcare services for everyone. Nonetheless, a poor usage rate could curb its progression in developing cultures like Bangladesh. Therefore, this research examines how external stimuli promote the continuous usage intentions of synchronous telemedicine services through engagement and satisfaction by deploying the stimulus-organism-response framework. A final sample of 312 telemedicine users was analyzed using the structural equation modeling in AMOS. The average age of the participants was 26.28 (std. deviation 5.53), and their average use of telemedicine was 2.39 times (std. deviation 1.31) over the last six months. This study empirically endorsed that the stimuli, including performance expectancy, information quality, and contamination avoidance, as well as organismic factors such as engagement and satisfaction, directly impacted the continuance desires for telemedicine use. In addition, the analyses validated the mediation roles of engagement and satisfaction. Furthermore, performance and effort expectancies influenced engagement, which affected satisfaction along with performance expectancy, functionality, and information quality. Accordingly, telemedicine facilitators should integrate these critical attributes into the system to sustain engagement, satisfaction, and usage intentions. This study has pioneered the effects of performance and effort expectancies on continuous usage intentions facilitated by engagement and satisfaction in the telemedicine landscape.

Digital Health-Enabled Clinical Trials in Stroke: Ready for Prime Time?

As stroke continues to represent a major global health care problem, advancing our knowledge of new effective and safe stroke interventions represents a public health priority. The identification of these therapies requires the conduct of high-quality and well-powered randomized clinical trials. Despite its potential to inform clinical practice, traditional randomized clinical trial models have their drawbacks, including elevated costs, long completion times, failure to recruit the target sample sizes, lack of diversity, and complex operational procedures. Therefore, improving the participants' experience and trials' overall efficiency constitutes an important unmet need. Innovative models such as virtual and decentralized patient-centric trials have been proposed as a valuable strategy in this pursuit. In this narrative review, we discuss the limitations of traditional randomized clinical trial models and present the concept, advantages, and challenges of decentralized digitally enabled approaches to the conduct of stroke clinical trials.

Informing the Future of Integrated Digital and Clinical Mental Health Care: Synthesis of the Outcomes From Project Synergy

BACKGROUND

Globally, there are fundamental shortcomings in mental health care systems, including restricted access, siloed services, interventions that are poorly matched to service users' needs, underuse of personal outcome monitoring to track progress, exclusion of family and carers, and suboptimal experiences of care. Health information technologies (HITs) hold great potential to improve these aspects that underpin the enhanced quality of mental health care.

OBJECTIVE

Project Synergy aimed to co-design, implement, and evaluate novel HITs, as exemplified by the InnoWell Platform, to work with standard health care organizations. The goals were to deliver improved outcomes for specific populations under focus and support organizations to enact significant system-level reforms.

METHODS

Participating health care organizations included the following: Open Arms-Veterans & Families Counselling (in Sydney and Lismore, New South Wales [NSW]); NSW North Coast headspace centers for youth (Port Macquarie, Coffs Harbour, Grafton, Lismore, and Tweed Heads); the Butterfly Foundation's National Helpline for eating disorders; Kildare Road Medical Centre for enhanced primary care; and Connect to Wellbeing North Coast NSW (administered by Neami National), for population-based intake and assessment. Service users, families and carers, health professionals, and administrators of services across Australia were actively engaged in the configuration of the InnoWell Platform to meet service needs, identify barriers to and facilitators of quality mental health care, and highlight potentially the best points in the service pathway to integrate the InnoWell Platform. The locally configured InnoWell Platform was then implemented within the respective services. A mixed methods approach, including surveys, semistructured interviews, and workshops, was used to evaluate the impact of the InnoWell Platform. A participatory systems modeling approach involving co-design with local stakeholders was also undertaken to simulate the likely impact of the platform in combination with other services being considered for implementation within the North Coast Primary Health Network to explore resulting impacts on mental health outcomes, including suicide prevention.

RESULTS

Despite overwhelming support for integrating digital health solutions into mental health service settings and promising impacts of the platform simulated under idealized implementation conditions, our results emphasized that successful implementation is dependent on health professional and service readiness for change, leadership at the local service level, the appropriateness and responsiveness of the technology for the target end users, and, critically, funding models being available to support implementation. The key places of interoperability of digital solutions and a willingness to use technology to coordinate health care system use were also highlighted.

CONCLUSIONS

Although the COVID-19 pandemic has resulted in the widespread acceptance of very basic digital health solutions, Project Synergy highlights the critical need to support equity of access to HITs, provide funding for digital infrastructure and digital mental health care, and actively promote the use of technology-enabled, coordinated systems of care.

Digital Health in Primordial and Primary Stroke Prevention: A Systematic Review

The stroke burden continues to grow across the globe, disproportionally affecting developing countries. This burden cannot be effectively halted and reversed without effective and widely implemented primordial and primary stroke prevention measures, including those on the individual level. The unprecedented growth of smartphone and other digital technologies with digital solutions are now being used in almost every area of health, offering a unique opportunity to improve primordial and primary stroke prevention on the individual level. However, there are several issues that need to be considered to advance development and use this important digital strategy for primordial and primary stroke prevention. Using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines we provide a systematic review of the current knowledge, challenges, and opportunities of digital health in primordial and primary stroke prevention.

Spanish adaptation and validation of the User Version of the Mobile Application Rating Scale (uMARS)

OBJECTIVE

While the professional version of the Mobile App Rating Scale (MARS) has already been translated, and validated into the Spanish language, its user-centered counterpart has not yet been adapted. Furthermore, no other similar tools exist in the Spanish language. The aim of this paper is to adapt and validate User Version of the MARS (uMARS) into the Spanish language.

MATERIALS AND METHODS

Cross-cultural adaptation, translation, and metric evaluation. The internal consistency and test-retest reliability of the Spanish version of the uMARS were evaluated using the RadarCovid app. Two hundred and sixteen participants rated the app using the translated scale. The app was then rated again 2 weeks later by 21 of these participants to measure test-retest reliability.

RESULTS

No major differences were observed between the uMARS original and the Spanish version. Discrimination indices (item-scale correlation) obtained appropriate results for both raters. The Spanish uMARS presented with excellent internal consistency, α = .89 and .67 for objective and subjective quality, respectively, and temporal stability (r > 0.82 for all items and subscales).

DISCUSSION

The Spanish uMARS is a useful tool for health professionals to recommend high-quality mobile apps to their patients based on the user's perspective and for researchers and app developers to use end-user feedback and evaluation, to help them identify highly appraised and valued components, as well as areas for further development, to continue ensuring the increasing quality and prominence of the area of mHealth.

CONCLUSION

uMARS Spanish version is an instrument with adequate metric properties to assess the quality of health apps from the user perspective.

[Identification of rheumatological health apps in the Apple app store applying the "semiautomatic retrospective app store analysis" method : A longitudinal observation]

Hintergrund

Die App Stores von Apple und Google bieten eine Vielzahl von Gesundheits-Apps an. Das Auffinden qualitativ hochwertiger Apps ist immer noch eine Herausforderung.

Fragestellung

Lassen sich unter Anwendung der SARASA(„semiautomated retrospective App Store analysis“)-Methode für das Fachgebiet Rheumatologie deutschsprachige Apps identifizieren?

Material und Methode

SARASA ist eine Methode zur teilautomatisierten Auswahl und Charakterisierung von App Store-gelisteten Apps nach formalen Kriterien. Nach der ersten Anwendung in 02/2018 wurde SARASA 02/2020 erneut auf den Apple App Store angewendet.

Ergebnisse

In 02/2018 konnten für Apps in den Store-Kategorien „Medizin“ oder „Gesundheit und Fitness“ Metadaten zu 103.046 Apps und bei einer erneuten Erhebung in 02/2020 Daten zu 94.735 Apps über das deutsche Frontend des Apple App Stores ausgelesen werden. Im Jahr 2018 wurden nach Anwendung der Suchbegriffe 59 Apps mit einer deutschsprachigen Beschreibung für das Fachgebiet Rheumatologie identifiziert, 2020 waren dies 53 Apps, die jeweils manuell weiter überprüft wurden; 2018 waren noch mehr der gefundenen Apps für Patienten als für Ärzte vorgesehen, dies war 2020 ausgeglichener. Zudem zeigte sich, dass bei bestimmten Krankheitsbildern von den App-Entwicklern keine Bearbeitungen erfolgten. Die prozentuale Verteilung von Treffern nach Suchbegriffen zeigte im Vergleich von 2018 zu 2020 große Schwankungen.

Diskussion

Die SARASA-Methode stellt ein hilfreiches Werkzeug dar, um Gesundheits-Apps teilautomatisiert zu identifizieren, die vordefinierten, formalen Kriterien entsprechen. Die inhaltliche Qualität muss anschließend manuell überprüft werden. Weiterentwicklungen der SARASA-Methode und die weitere Konsentierung und Standardisierung von Qualitätskriterien sind sinnvoll. Qualitätskriterien sollten beim Angebot von Gesundheits-Apps in den App-Stores berücksichtigt werden.

The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality

As the COVID-19 pandemic has largely increased the utilization of telehealth, mobile mental health technologies - such as smartphone apps, vir-tual reality, chatbots, and social media - have also gained attention. These digital health technologies offer the potential of accessible and scalable interventions that can augment traditional care. In this paper, we provide a comprehensive update on the overall field of digital psychiatry, covering three areas. First, we outline the relevance of recent technological advances to mental health research and care, by detailing how smartphones, social media, artificial intelligence and virtual reality present new opportunities for "digital phenotyping" and remote intervention. Second, we review the current evidence for the use of these new technological approaches across different mental health contexts, covering their emerging efficacy in self-management of psychological well-being and early intervention, along with more nascent research supporting their use in clinical management of long-term psychiatric conditions - including major depression; anxiety, bipolar and psychotic disorders; and eating and substance use disorders - as well as in child and adolescent mental health care. Third, we discuss the most pressing challenges and opportunities towards real-world implementation, using the Integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework to explain how the innovations themselves, the recipients of these innovations, and the context surrounding innovations all must be considered to facilitate their adoption and use in mental health care systems. We conclude that the new technological capabilities of smartphones, artificial intelligence, social media and virtual reality are already changing mental health care in unforeseen and exciting ways, each accompanied by an early but promising evidence base. We point out that further efforts towards strengthening implementation are needed, and detail the key issues at the patient, provider and policy levels which must now be addressed for digital health technologies to truly improve mental health research and treatment in the future.