An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process

Testing the Effectiveness of a Mobile Smartphone App Designed to Improve the Mental Health of Junior Physicians: Protocol for a Randomized Controlled Trial

BACKGROUND

Shift (Black Dog Institute) is the first mobile health smartphone app created to support the mental health of junior physicians. Junior physicians experience demanding work conditions, leading to high levels of psychological distress and burnout. However, they are often concerned about the potential career impacts of seeking mental health support. The confidentiality and ease of access of digital interventions may be particularly suited to address these concerns. The Shift app provides therapeutic and psychoeducational content and strategies contextualized for the specific needs of physicians in training. App content includes information on mental health, help seeking, mindfulness, and common workplace-related concerns of junior physicians.

OBJECTIVE

This study aims to test, at scale, the effectiveness of Shift among junior physicians working in Australia using a randomized controlled trial design. The primary aim is to examine whether junior physicians using Shift experience a reduction in depressive symptoms compared with a waitlist control group. The secondary aim is to examine whether the app intervention group experiences improvements in anxiety, work and social functioning, help seeking, quality of life, and burnout compared with the control group.

METHODS

A total of 778 junior physicians were recruited over the internet through government and nongovernment medical organizations across Australia, as well as through paid social media advertisements. They were randomly allocated to one of 2 groups: (1) the intervention group, who were asked to use the Shift app for a period of 30 days, or (2) the waitlist control group, who were placed on a waitlist and were asked to use the app after 3 months. Participants completed psychometric measures for self-assessing mental health and wellbeing outcomes, with assessments occurring at baseline, 1 month after completing the baseline period, and 3 months after completing the baseline period. Participants in the waitlist control group were asked to complete an additional web-based questionnaire 1 month after receiving access to the app or 4 months after completing the baseline survey. Participants took part in the study on the internet; the study was completely automated.

RESULTS

The study was funded from November 2022 to December 2024 by the New South Wales Ministry of Health. Data collection for the study occurred between January and August 2024, with 780 participants enrolling in the study during this time. Data analysis is underway; the effectiveness of the intervention will be estimated on an intention-to-treat basis using a mixed-model, repeated measures analysis. Results are expected to be submitted for publication in 2025.

CONCLUSIONS

To the best of our knowledge, this is the first randomized controlled trial to examine the effectiveness of a mobile health smartphone app specifically designed to support the mental health of junior physicians.

TRIAL REGISTRATION

Australia and New Zealand Clinical Trials Registry ACTRN12623000664640; https://tinyurl.com/7xt24dhk.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/58288.

Understanding and Defining Young People's Involvement and Under-Representation in Mental Health Research: A Delphi Study

INTRODUCTION

The mental health of young people (aged 16-25 years) is a growing public health concern in the United Kingdom due to the increasing numbers of young people experiencing mental health difficulties, with many not in contact with mental health services. To design services that meet the needs of all young people, a diversity of young people must be involved in mental health research, beyond being participants. This Delphi study aimed to identify different types of 'involvement' and to define and describe 'under-representation' in young people's involvement in mental health research.

METHODS

Twenty-seven experts in young people's mental health research completed a series of online questionnaires. The experts were academic researchers, patient and public involvement (PPI) professionals and young 'experts by experience'. Round 1 generated panellists' views on 'involvement' and 'under-representation'. Round 2 summarised panellists' responses from Round 1 and sought consensus (minimum 70% agreement) in nine question areas. Round 3 validated the findings of the previous rounds.

RESULTS

Consensus was achieved in eight out of nine areas, resulting in a matrix (with definitions) of the different types of young people's involvement in mental health research, from being advisors to involvement ambassadors. The findings generated an agreed-upon definition of under-representation, an identification of when in the research process there is under-representation and the characteristics of the young people who are under-represented. Experts further agreed on demographic data that should be collected to improve reporting on involvement.

CONCLUSIONS

This study adds to our understanding of involvement and under-representation in the context of young people's mental health research through expert consensus. It provides a practical resource for researchers considering involving young people in the research process and suggests the data that should be collected to improve reporting on the diversity of the young people involved.

PATIENT AND PUBLIC CONTRIBUTION

A research oversight group of five young people advised on this study. They contributed throughout the project-from endorsing the research question to commenting on the findings and dissemination. Two of the group reviewed all participant materials and piloted the initial questionnaire.

Investigating the characteristics of health-related data collection tools used in randomised controlled trials in low-income and middle-income countries: protocol for a systematic review

INTRODUCTION

Health-related data collection tools, including digital ones, have become more prevalent across clinical studies in the last number of years. However, using digital data collection tools in low-income and middle-income countries presents unique challenges. In this review, we aim to provide an overview of the data collection tools currently being used in randomised controlled trials (RCTs) conducted in low-resource settings and evaluate the tools based on the characteristics outlined in the modified Mobile Survey Tool framework. These include functionality, reliability, usability, efficiency, maintainability, portability, effectiveness, cost-benefit, satisfaction, freedom from risk and context coverage. This evidence may provide a guide to selecting a suitable data collection tool for researchers planning to conduct research in low-income and middle-income countries for future studies.

METHODS AND ANALYSIS

Searches will be conducted in four electronic databases: PubMed, CINAHL, Web of Science and EMBASE. For inclusion, studies must be a RCT, mention a health-related data collection tool and conducted in a low- and middle-income country. Only studies with available full-text and written in English will be included. The search was restricted to studies published between January 2005 and June 2023. This systematic review will use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) tool. Two review authors will screen the titles and abstracts of search results independently for inclusion. In the initial screening process, the full-text articles will be retrieved if the abstract contains limited information about the study. Disagreements will be resolved through discussion. If the disagreement cannot be resolved, a third author (JO'D) will adjudicate. The study selection process will be outlined in a PRISMA flow-diagram. Data will be analysed using a narrative synthesis approach. The included studies and their outcomes will be presented in a table.

ETHICS AND DISSEMINATION

Formal ethical approval is not required as primary data will not be collected in this study. The findings from this systematic review will be published in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42023405738.

Evaluation of Chinese HIV Mobile Apps by Researchers and Patients With HIV: Quality Evaluation Study

BACKGROUND

Against the backdrop of globalization, China remains one of the most heavily burdened countries in Asia with regard to AIDS. However, many high-risk groups and patients affected by AIDS may be less likely to actively seek care from medical institutions because of fear of experiencing shame or discrimination. Mobile apps provide a promising avenue for supporting the prevention, diagnosis, and treatment of AIDS. However, a comprehensive systematic evaluation of these mobile apps' functionality and quality has not been conducted yet.

OBJECTIVE

This study aims to identify the available mobile apps for AIDS in China, assess and discuss the functional features and quality of these Chinese AIDS mobile apps, and offer decision support for patients and clinical practitioners in accessing high-quality AIDS mobile apps. Furthermore, based on the evaluation results, recommendations for improvement will be provided.

METHODS

A systematic search was conducted on the Qimai app data platform, the Aladdin WeChat applet data platform, and WeChat to identify mobile apps related to AIDS. A snowball sampling method was used to supplement the potentially overlooked apps. The selected mobile apps underwent a rigorous screening process based on unified criteria. Subsequently, assessments were independently undertaken by 3 separate researchers and 2 patients with HIV, using both the Mobile App Rating Scale (MARS) and the User Mobile App Rating Scale (uMARS). Quantitative interpretations of the data were facilitated by the MedCalc statistical software (version 20.217, MedCalc Software).

RESULTS

A total of 2901 AIDS mobile apps were included in the study, with 2897 identified through information retrieval and an additional 4 added via snowball sampling. After a rigorous selection process, 21 apps were determined to be usable. Among them, the Hong Feng Wan app achieved the highest combined average score, calculated based on the MARS (3.96, SD 0.33) and uMARS (4.47, SD 0.26). Overall, there was no significant correlation between MARS and uMARS (rapp quality total score=0.41; P=.07; rsubjective quality=0.39; P=.08). A notable issue was the widespread lack of user privacy protection, with only 24% (5/21) of the apps offering this feature.

CONCLUSIONS

The number of available Chinese AIDS mobile apps is limited, with WeChat applets dominating the market. Nonetheless, the performance of WeChat mini-apps is generally inferior to that of independent apps, and there may be significant discrepancies between assessments conducted by researchers and those provided by genuine end users, emphasizing the necessity of involving real users in the development and evaluation of HIV mobile apps. In addition, developers of these Chinese HIV mobile apps need to devote attention to improving privacy protection mechanisms, in addition to considering the evaluations of researchers and real users. This will help attract more users and increase user loyalty.

Sleep well, worry less: A co-design study for the development of the SMILE app

Objective

With the coronavirus disease 2019 pandemic exacerbating mental health concerns, the prevalence rates of anxiety and sleep problems have increased alarmingly among youth. Although 90% of patients with anxiety experience sleep problems, current interventions for anxiety often do not target sleep problems in youth. Given this lack, we designed the SMILE app, an intervention that addresses both anxiety and sleep problems simultaneously.

Methods

As users' perspectives are essential to ensure app engagement and uptake, the features, designs, and functions of the SMILE app were evaluated using a participatory app design approach. Participants (N = 17) were youth aged 15 to 25 who reported co-morbid anxiety and sleep issues above clinical thresholds. After completing an online screening survey assessing demographics, anxiety, and sleep problems, participants shared app feedback through group-based, semi-structured co-design sessions. Qualitative analyses were conducted to identify common themes from participants' feedback.

Results

While participants expressed enthusiasm for the SMILE app's features, particularly the Visualization, Journaling, and Psychoeducation features, and their variety, they criticized the design aspects of the app, such as the font and text amount. Most participants stated they would use the SMILE app or recommend it to a friend.

Conclusion

By actively involving the target population in the design process, the SMILE app has the potential to notably improve the mental well-being of youth, though further research and development are required to realize this potential fully.

A sociotechnical framework to assess patient-facing eHealth tools: results of a modified Delphi process

Among the thousands of eHealth tools available, the vast majority do not get past pilot phases because they cannot prove value, and only a few have been systematically assessed. Although multiple eHealth assessment frameworks have been developed, these efforts face multiple challenges. This study aimed to address some of these challenges by validating and refining an initial list of 55 assessment criteria based on previous frameworks through a two-round modified Delphi process with in-between rounds of interviews. The expert panel (n = 57) included participants from 18 countries and 9 concerned parties. A consensus was reached on 46 criteria that were classified into foundational and contextual criteria. The 36 foundational criteria focus on evaluating the eHealth tool itself and were grouped into nine clusters: technical aspects, clinical utility and safety, usability and human centricity, functionality, content, data management, endorsement, maintenance, and developer. The 10 contextual criteria focus on evaluating the factors that vary depending on the context the tool is being evaluated for and were grouped into seven clusters: data-protection compliance, safety regulatory compliance, interoperability and data integration, cultural requirements, affordability, cost-benefit, and implementability. The classification of criteria into foundational and contextual helps us assess not only the quality of an isolated tool, but also its potential fit in a specific setting. Criteria subscales may be particularly relevant when determining the strengths and weaknesses of the tool being evaluated. This granularity enables different concerned parties to make informed decisions about which tools to consider according to their specific needs and priorities.

Stakeholders' views and opinions on existing guidelines on "How to Choose Mental Health Apps"

Background

Mental health Applications (Mhealth Apps) can change how healthcare is delivered. However, very little is known about the efficacy of Mhealth Apps. Currently, only minimum guidance is available in Assessment and Evaluation Tools (AETs). Therefore, this project aims to understand AET developers' perspectives and end users' experiences and opinions on "how to choose a Mhealth App".

Objective

The primary objectives were: (1) obtaining stakeholder's opinions and experiences of development and use of AETs for Mhealth Apps, their weaknesses and strengths, and barriers in their implementation of Mhealth Apps; (2) the experiences of App users, their analyzation and, obstacles in the use of apps; and (3) to quantify themes related to choosing a Mhealth App.

Methods

This qualitative study, used a sampling method to recruit six stakeholders (one App developer, two AET developers, an individual with lived experience of mental health illness, and two physicians) who were interviewed using a topic guide. These were examined by researchers (CT, WK, & FN) using thematic content analysis. Additionally, an anonymous online survey of 107 individuals was conducted.

Findings

Our analyses revealed six main themes: (a) needs and opportunities; (b) views on Mhealth apps; (c) views & opinions on AETs; (d) implementation barriers; (e) system of evaluation and; (f) future directions. The first key concept was, all stakeholders agreed that Apps could significantly impact mental health and that end-users were unaware of mental health AETs and Apps. Secondly, due to commercial interests, end-users reliability of App evaluations requires clear conflict-free guidelines. Thirdly, AETs should be evaluated and developed through a rigorous methodology. Finally, stakeholders shared insights into future developments for AETs and Mhealth Apps. Additionally, online survey respondents chose a "health professional" as their preferred source of guidance in selecting a Mhealth app (84%) and best suited to develop guidelines (70%).

Conclusion

The interviews and survey highlight the need for Mhealth Apps to be regulated and the importance of health professionals' engagement in the implementation process. Similarly, without well-defined roles for App evaluations within the health care system, it is unlikely that AETs will have wider spread use and impact without risk.

Female youth and mental health service providers' perspectives on the JoyPop™ app: a qualitative study

Introduction

Mobile health (mHealth) apps are a promising adjunct to traditional mental health services, especially in underserviced areas. Developed to foster resilience in youth, the JoyPop™ app has a growing evidence base showing improvement in emotion regulation and mental health symptoms among youth. However, whether this novel technology will be accepted among those using or providing mental health services remains unknown. This study aimed to evaluate the JoyPop™ app's acceptance among (a) a clinical sample of youth and (b) mental health service providers.

Method

A qualitative descriptive approach involving one-on-one semi-structured interviews was conducted. Interviews were guided by the Technology Acceptance Model and were analyzed using a deductive-inductive content analysis approach.

Results

All youth (n = 6 females; Mage = 14.60, range 12-17) found the app easy to learn and use and expressed positive feelings towards using the app. Youth found the app useful because it facilitated accessibility to helpful coping skills (e.g., journaling to express their emotions; breathing exercises to increase calmness) and positive mental health outcomes (e.g., increased relaxation and reduced stress). All service providers (n = 7 females; Mage = 43.75, range 32-60) perceived the app to be useful and easy to use by youth within their services and expressed positive feelings about integrating the app into usual care. Service providers also highlighted various organizational factors affecting the app's acceptance. Youth and service providers raised some concerns about apps in general and provided recommendations to improve the JoyPop™ app.

Discussion

Results support youth and service providers' acceptance of the JoyPop™ app and lend support for it as an adjunctive resource to traditional mental health services for youth with emotion regulation difficulties.

Understanding Mobile Health and Youth Mental Health: Scoping Review

BACKGROUND

A total of 75% of people with mental health disorders have an onset of illness between the ages of 12 and 24 years. Many in this age group report substantial obstacles to receiving quality youth-centered mental health care services. With the rapid development of technology and the recent COVID-19 pandemic, mobile health (mHealth) has presented new opportunities for youth mental health research, practice, and policy.

OBJECTIVE

The research objectives were to (1) synthesize the current evidence supporting mHealth interventions for youths who experience mental health challenges and (2) identify current gaps in the mHealth field related to youth's access to mental health services and health outcomes.

METHODS

Guided by the methods of Arksey and O'Malley, we conducted a scoping review of peer-reviewed studies that used mHealth tools to improve youth mental health (January 2016-February 2022). We searched MEDLINE, PubMed, PsycINFO, and Embase databases using the following key terms: (1) mHealth; (2) youth and young adults; and (3) mental health. The current gaps were analyzed using content analysis.

RESULTS

The search produced 4270 records, of which 151 met inclusion criteria. Included articles highlight the comprehensive aspects of youth mHealth intervention resource allocation for targeted conditions, mHealth delivery methods, measurement tools, evaluation of mHealth intervention, and youth engagement. The median age for participants in all studies is 17 (IQR 14-21) years. Only 3 (2%) studies involved participants who reported their sex or gender outside of the binary option. Many studies (68/151, 45%) were published after the onset of the COVID-19 outbreak. Study types and designs varied, with 60 (40%) identified as randomized controlled trials. Notably, 143 out of 151 (95%) studies came from developed countries, suggesting an evidence shortfall on the feasibility of implementing mHealth services in lower-resourced settings. Additionally, the results highlight concerns related to inadequate resources devoted to self-harm and substance uses, weak study design, expert engagement, and the variety of outcome measures selected to capture impact or changes over time. There is also a lack of standardized regulations and guidelines for researching mHealth technologies for youths and the use of non-youth-centered approaches to implementing results.

CONCLUSIONS

This study may be used to inform future work as well as the development of youth-centered mHealth tools that can be implemented and sustained over time for diverse types of youths. Implementation science research that prioritizes youths' engagement is needed to advance the current understanding of mHealth implementation. Moreover, core outcome sets may support a youth-centered measurement strategy to capture outcomes in a systematic way that prioritizes equity, diversity, inclusion, and robust measurement science. Finally, this study suggests that future practice and policy research are needed to ensure the risk of mHealth is minimized and that this innovative health care service is meeting the emerging needs of youths over time.

Assessing the Quality and Impact of eHealth Tools: Systematic Literature Review and Narrative Synthesis

BACKGROUND

Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges.

OBJECTIVE

This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives).

METHODS

A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes.

RESULTS

Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer.

CONCLUSIONS

This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users' adoption to ensure uptake and adherence in the long term.

Critical Criteria and Countermeasures for Mobile Health Developers to Ensure Mobile Health Privacy and Security: Mixed Methods Study

BACKGROUND

Despite the importance of the privacy and confidentiality of patients' information, mobile health (mHealth) apps can raise the risk of violating users' privacy and confidentiality. Research has shown that many apps provide an insecure infrastructure and that security is not a priority for developers.

OBJECTIVE

This study aims to develop and validate a comprehensive tool to be considered by developers for assessing the security and privacy of mHealth apps.

METHODS

A literature search was performed to identify papers on app development, and those papers reporting criteria for the security and privacy of mHealth were assessed. The criteria were extracted using content analysis and presented to experts. An expert panel was held for determining the categories and subcategories of the criteria according to meaning, repetition, and overlap; impact scores were also measured. Quantitative and qualitative methods were used for validating the criteria. The validity and reliability of the instrument were calculated to present an assessment instrument.

RESULTS

The search strategy identified 8190 papers, of which 33 (0.4%) were deemed eligible. A total of 218 criteria were extracted based on the literature search; of these, 119 (54.6%) criteria were removed as duplicates and 10 (4.6%) were deemed irrelevant to the security or privacy of mHealth apps. The remaining 89 (40.8%) criteria were presented to the expert panel. After calculating impact scores, the content validity ratio (CVR), and the content validity index (CVI), 63 (70.8%) criteria were confirmed. The mean CVR and CVI of the instrument were 0.72 and 0.86, respectively. The criteria were grouped into 8 categories: authentication and authorization, access management, security, data storage, integrity, encryption and decryption, privacy, and privacy policy content.

CONCLUSIONS

The proposed comprehensive criteria can be used as a guide for app designers, developers, and even researchers. The criteria and the countermeasures presented in this study can be considered to improve the privacy and security of mHealth apps before releasing the apps into the market. Regulators are recommended to consider an established standard using such criteria for the accreditation process, since the available self-certification of developers is not reliable enough.

Artificial intelligence technologies and compassion in healthcare: A systematic scoping review

Background

Advances in artificial intelligence (AI) technologies, together with the availability of big data in society, creates uncertainties about how these developments will affect healthcare systems worldwide. Compassion is essential for high-quality healthcare and research shows how prosocial caring behaviors benefit human health and societies. However, the possible association between AI technologies and compassion is under conceptualized and underexplored.

Objectives

The aim of this scoping review is to provide a comprehensive depth and a balanced perspective of the emerging topic of AI technologies and compassion, to inform future research and practice. The review questions were: How is compassion discussed in relation to AI technologies in healthcare? How are AI technologies being used to enhance compassion in healthcare? What are the gaps in current knowledge and unexplored potential? What are the key areas where AI technologies could support compassion in healthcare?

Materials and methods

A systematic scoping review following five steps of Joanna Briggs Institute methodology. Presentation of the scoping review conforms with PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews). Eligibility criteria were defined according to 3 concept constructs (AI technologies, compassion, healthcare) developed from the literature and informed by medical subject headings (MeSH) and key words for the electronic searches. Sources of evidence were Web of Science and PubMed databases, articles published in English language 2011-2022. Articles were screened by title/abstract using inclusion/exclusion criteria. Data extracted (author, date of publication, type of article, aim/context of healthcare, key relevant findings, country) was charted using data tables. Thematic analysis used an inductive-deductive approach to generate code categories from the review questions and the data. A multidisciplinary team assessed themes for resonance and relevance to research and practice.

Results

Searches identified 3,124 articles. A total of 197 were included after screening. The number of articles has increased over 10 years (2011, n = 1 to 2021, n = 47 and from Jan-Aug 2022 n = 35 articles). Overarching themes related to the review questions were: (1) Developments and debates (7 themes) Concerns about AI ethics, healthcare jobs, and loss of empathy; Human-centered design of AI technologies for healthcare; Optimistic speculation AI technologies will address care gaps; Interrogation of what it means to be human and to care; Recognition of future potential for patient monitoring, virtual proximity, and access to healthcare; Calls for curricula development and healthcare professional education; Implementation of AI applications to enhance health and wellbeing of the healthcare workforce. (2) How AI technologies enhance compassion (10 themes) Empathetic awareness; Empathetic response and relational behavior; Communication skills; Health coaching; Therapeutic interventions; Moral development learning; Clinical knowledge and clinical assessment; Healthcare quality assessment; Therapeutic bond and therapeutic alliance; Providing health information and advice. (3) Gaps in knowledge (4 themes) Educational effectiveness of AI-assisted learning; Patient diversity and AI technologies; Implementation of AI technologies in education and practice settings; Safety and clinical effectiveness of AI technologies. (4) Key areas for development (3 themes) Enriching education, learning and clinical practice; Extending healing spaces; Enhancing healing relationships.

Conclusion

There is an association between AI technologies and compassion in healthcare and interest in this association has grown internationally over the last decade. In a range of healthcare contexts, AI technologies are being used to enhance empathetic awareness; empathetic response and relational behavior; communication skills; health coaching; therapeutic interventions; moral development learning; clinical knowledge and clinical assessment; healthcare quality assessment; therapeutic bond and therapeutic alliance; and to provide health information and advice. The findings inform a reconceptualization of compassion as a human-AI system of intelligent caring comprising six elements: (1) Awareness of suffering (e.g., pain, distress, risk, disadvantage); (2) Understanding the suffering (significance, context, rights, responsibilities etc.); (3) Connecting with the suffering (e.g., verbal, physical, signs and symbols); (4) Making a judgment about the suffering (the need to act); (5) Responding with an intention to alleviate the suffering; (6) Attention to the effect and outcomes of the response. These elements can operate at an individual (human or machine) and collective systems level (healthcare organizations or systems) as a cyclical system to alleviate different types of suffering. New and novel approaches to human-AI intelligent caring could enrich education, learning, and clinical practice; extend healing spaces; and enhance healing relationships.

Implications

In a complex adaptive system such as healthcare, human-AI intelligent caring will need to be implemented, not as an ideology, but through strategic choices, incentives, regulation, professional education, and training, as well as through joined up thinking about human-AI intelligent caring. Research funders can encourage research and development into the topic of AI technologies and compassion as a system of human-AI intelligent caring. Educators, technologists, and health professionals can inform themselves about the system of human-AI intelligent caring.

Comparing Professional and Consumer Ratings of Mental Health Apps: Mixed Methods Study

Background

As the number of mental health apps has grown, increasing efforts have been focused on establishing quality tailored reviews. These reviews prioritize clinician and academic views rather than the views of those who use them, particularly those with lived experiences of mental health problems. Given that the COVID-19 pandemic has increased reliance on web-based and mobile mental health support, understanding the views of those with mental health conditions is of increasing importance.

Objective

This study aimed to understand the opinions of people with mental health problems on mental health apps and how they differ from established ratings by professionals.

Methods

A mixed methods study was conducted using a web-based survey administered between December 2020 and April 2021, assessing 11 mental health apps. We recruited individuals who had experienced mental health problems to download and use 3 apps for 3 days and complete a survey. The survey consisted of the One Mind PsyberGuide Consumer Review Questionnaire and 2 items from the Mobile App Rating Scale (star and recommendation ratings from 1 to 5). The consumer review questionnaire contained a series of open-ended questions, which were thematically analyzed and using a predefined protocol, converted into binary (positive or negative) ratings, and compared with app ratings by professionals and star ratings from app stores.

Results

We found low agreement between the participants’ and professionals’ ratings. More than half of the app ratings showed disagreement between participants and professionals (198/372, 53.2%). Compared with participants, professionals gave the apps higher star ratings (3.58 vs 4.56) and were more likely to recommend the apps to others (3.44 vs 4.39). Participants’ star ratings were weakly positively correlated with app store ratings (r=0.32, P=.01). Thematic analysis found 11 themes, including issues of user experience, ease of use and interactivity, privacy concerns, customization, and integration with daily life. Participants particularly valued certain aspects of mental health apps, which appear to be overlooked by professional reviewers. These included functions such as the ability to track and measure mental health and providing general mental health education. The cost of apps was among the most important factors for participants. Although this is already considered by professionals, this information is not always easily accessible.

Conclusions

As reviews on app stores and by professionals differ from those by people with lived experiences of mental health problems, these alone are not sufficient to provide people with mental health problems with the information they desire when choosing a mental health app. App rating measures must include the perspectives of mental health service users to ensure ratings represent their priorities. Additional work should be done to incorporate the features most important to mental health service users into mental health apps.

The design and development of an experience measure for a peer community moderated forum in a digital mental health service

Online digital mental health communities can contribute to users' mental health positively and negatively. Yet the measurement of experience, outcomes and impact mechanisms relating to digital mental health communities is difficult to capture. In this paper we demonstrate the development of an online experience measure for a specific children and young people's community forum inside a digital mental health service. The development of the Peer Online Community Experience Measure (POCEM) is informed by a multi-phased design: (i) item reduction through Estimate-Talk-Estimate modified Delphi methods, (ii) user testing with think-aloud protocols and (iii) a pilot study within the digital service community to explore observational data within the platform. Experts in the field were consulted to help reduce the items in the pool and to check their theoretical coherence. User testing workshops helped to inform the usability appearance, wording, and purpose of the measure. Finally, the pilot results highlight completion rates, differences in scores for age and roles and "relate to others", as the most frequent domain mechanism of support for this community. Outcomes frequently selected show the importance of certain aspects of the community, such as safety, connection, and non-judgment previously highlighted in the literature. Experience measures like this one could be used as indicators of active therapeutic engagement within the forum community and its content but further research is required to ascertain its acceptability and validity. Multi-phased approaches involving stakeholders and user-centred design activities enhances the development of digitally enabled measurement tools.

Mental Health Mobile Apps in the French App Store: Assessment of Functionality and Quality (Preprint)

Background

Approximately 800 million people, representing 11% of the world’s population, are affected by mental health problems. The COVID-19 pandemic exacerbated problems and triggered a decline in well-being, with drastic increase in the incidence of conditions such as anxiety, depression, and stress. Approximately 20,000 mental health apps are listed in mobile app stores. However, no significant evaluation of mental health apps in French, spoken by approximately 300 million people, has been identified in the literature yet.

Objective

This study aims to review the mental health mobile apps currently available on the French Apple App Store and Google Play Store and to evaluate their quality using Mobile App Rating Scale–French (MARS-F).

Methods

Screening of mental health apps was conducted from June 10, 2022, to June 17, 2022, on the French Apple App Store and Google Play Store. A shortlist of 12 apps was identified using the criteria of selection and assessed using MARS-F by 9 mental health professionals. Intraclass correlation was used to evaluate interrater agreement. Mean (SD) scores and their distributions for each section and item were calculated.

Results

The highest scores for MARS-F quality were obtained by Soutien psy avec Mon Sherpa (mean 3.85, SD 0.48), Evoluno (mean 3.54, SD 0.72), and Teale (mean 3.53, SD 0.87). Mean engagement scores (section A) ranged from 2.33 (SD 0.69) for Reflexe reussite to 3.80 (SD 0.61) for Soutien psy avec Mon Sherpa. Mean aesthetics scores (section C) ranged from 2.52 (SD 0.62) for Mental Booster to 3.89 (SD 0.69) for Soutien psy avec Mon Sherpa. Mean information scores (section D) ranged from 2.00 (SD 0.75) for Mental Booster to 3.46 (SD 0.77) for Soutien psy avec Mon Sherpa. Mean Mobile App Rating Scale subjective quality (section E) score varied from 1.22 (SD 0.26) for VOS – journal de l’humeur to 2.69 (SD 0.84) for Soutien psy avec Mon Sherpa. Mean app specificity (section F) score varied from 1.56 (SD 0.97) for Mental Booster to 3.31 (SD 1.22) for Evoluno. For all the mental health apps studied, except Soutien psy avec Mon Sherpa (11/12, 92%), the subjective quality score was always lower than the app specificity score, which was always lower than the MARS-F quality score, and that was lower than the rating score from the iPhone Operating System or Android app stores.

Conclusions

Mental health professionals assessed that, despite the lack of scientific evidence, the mental health mobile apps available on the French Apple App Store and Google Play Store were of good quality. However, they are reluctant to use them in their professional practice. Additional investigations are needed to assess their compliance with recommendations and their long-term impact on users.

Review of Mobile Apps for Women With Anxiety in Pregnancy: Maternity Care Professionals' Guide to Locating and Assessing Anxiety Apps

Background

Mental health and pregnancy apps are widely available and have the potential to improve health outcomes and enhance women’s experience of pregnancy. Women frequently access digital information throughout their pregnancy. However, health care providers and women have little information to guide them toward potentially helpful or effective apps.

Objective

This review aimed to evaluate a methodology for systematically searching and reviewing commercially available apps that support pregnant women with symptoms of anxiety in order to assist maternity care professionals in identifying resources that they could recommend for these women.

Methods

A stepwise systematic approach was used to identify, select, describe, and assess the most popular and highly user-rated apps available in the United Kingdom from January to March 2021. This included developing a script-based search strategy and search process, writing evaluation criteria, and conducting a narrative description and evaluation of the selected apps.

Results

Useful search terms were identified, which included nonclinical, aspirational, and problem-based phrases. There were 39 apps selected for inclusion in the review. No apps specifically targeted women with anxiety in pregnancy. Of the 39 apps included in the review, 33 (85%) focused solely on mind-body techniques to promote relaxation, stress reduction, and psychological well-being. Only 8 of the 39 (21%) apps included in the review reported that health care professionals had contributed to app development and only 1/39 (3%) provided empirical evidence on the effectiveness and acceptability of the app. The top 12/39 (31%) apps were evaluated by 2 independent reviewers using the developed criteria and scores. There was a small negative correlation between the reviewers’ scores and app user rating scores, with higher user rating scores associated with lower reviewer scores.

Conclusions

App developers, publishers, and maternity care professionals should seek advice from women with lived experience of anxiety symptoms in pregnancy to locate, promote, and optimize the visibility of apps for pregnant women. There is a lack of resources that provide coping strategies based on current evidence for the treatment of anxiety in pregnancy. Maternity care providers are limited in their ability to locate and recommend acceptable and trustworthy apps because of the lack of information on the evidence base, development, and testing of apps. Maternity care professionals and women need access to libraries of trusted apps that have been evaluated against relevant and established criteria.

How Does Long-Term Orientation Influence the Investments of Venture Capitals? Evidence From the Organizational Level

Amid great uncertainty along with the possibility of huge returns, venture investment decisions are both technical and artistic. Past studies have paid much attention to the influences of objective factors on venture investment. However, subjective factors have been relatively ignored. As a salient psychological mechanism, temporal focus is of great importance for venture capitalists when making their investment decisions. This study performed content analysis to investigate how temporal focus at the organizational level affects investment decisions of venture capital (VC) firms. The results revealed that VCs with higher level of long-term orientation prefer to invest in less popular industries and ventures in the expansion period. Meanwhile, they are less likely to invest in very new start-ups. Moreover, long-term oriented VCs tend to re-invest in start-ups in their portfolios instead of just shooting once on numerous single start-ups. However, the author did not find any support on preferences of VCs for ventures with high level of human capital.

Considerations of diversity, equity, and inclusion in mental health apps: A scoping review of evaluation frameworks

Mental health applications or apps are a promising approach to reduce the pervasive unmet mental health need observed among marginalized groups. However, these groups face unique challenges to engage in and benefit from these interventions, and apps that consider issues of diversity, equity, and inclusion (DEI) may be better positioned to reach them. App evaluation frameworks emerge as crucial tools for researchers, clinicians, and users to select evidence-based apps. Nevertheless, it is unclear the extent to which existing assessment tools capture DEI factors. The present scoping review identified 68 studies that employed one or more assessment tools to evaluate a mental health app, leading to 44 unique app evaluation frameworks. Results showed that most frameworks were developed after 2015, and only 58% of them considered at least one DEI criterion. Frameworks that performed exceptionally well were those that combined multiple standardized and validated measures. This lack of consideration for DEI variables may limit the ability of app-based interventions to serve marginalized communities, or even worse, create new disparities. Therefore, we provide recommendations to improve current app evaluation frameworks' cultural robustness and clinical utility, maximizing their effectiveness when working with individuals from marginalized communities.

Apps and gaps in bipolar disorder: A systematic review on electronic monitoring for episode prediction

BACKGROUND

Long-term clinical monitoring in bipolar disorder (BD) is an important therapeutic tool. The availability of smartphones and wearables has sparked the development of automated applications to remotely monitor patients. This systematic review focus on the current state of electronic (e-) monitoring for episode prediction in BD.

METHODS

We systematically reviewed the literature on e-monitoring for episode prediction in adult BD patients. The systematic review was done according to the guidelines for reporting of systematic reviews and meta-analyses (PRISMA) and was registered in PROSPERO on April 29, 2020 (CRD42020155795). We conducted a search of Web of Science, MEDLINE, EMBASE, and PsycINFO (all 2000-2020) databases. We identified and extracted data from 17 published reports on 15 relevant studies.

RESULTS

Studies were heterogeneous and most had substantial methodological and technical limitations. Models varied widely in their performance. Published metrics were too heterogeneous to lend themselves to a meta-analysis. Four studies reported sensitivity (range: 0.21 - 0.95); and two reported specificity for prediction of mood episodes (range: 0.36 - 0.99). Two studies reported accuracy (range: 0.64 - 0.88) and four reported area under the curve (AUC; range: 0.52-0.95). Overall, models were better in predicting manic or hypomanic episodes, but their performance depended on feature type.

LIMITATIONS

Our conclusions are tempered by the lack of appropriate information impeding our ability to synthesize the available evidence.

CONCLUSIONS

Given the clinical variability in BD, predicting mood episodes remains a challenging task. Emerging e-monitoring technology for episode prediction in BD requires more development before it can be adopted clinically.

Terms and conditions apply: Critical issues for readability and jargon in mental health depression apps

BACKGROUND

Mental health services are turning to technology to ease the resource burden, but privacy policies are hard to understand potentially compromising consent for people with mental health problems. The FDA recommends a reading grade of 8.

OBJECTIVE

To investigate and improve the accessibility and acceptability of mental health depression app privacy policies.

METHODS

A mixed methods study using quantitative and qualitative data to improve the accessibility of app privacy policies. Service users completed assessments and focus groups to provide information on ways to improve privacy policy accessibility, including identifying and rewording jargon. This was supplemented by comparisons of mental health depression apps with social media, music and finance apps using readability analyses and examining whether GDPR affected accessibility.

RESULTS

Service users provided a detailed framework for increasing accessibility that emphasised having critical information for consent. Quantitatively, most app privacy policies were too long and complicated for ensuring informed consent (mental health apps mean reading grade = 13.1 (SD = 2.44)). Their reading grades were no different to those for other services. Only 3 mental health apps had a grade 8 or less and 99% contained service user identified jargon. Mental health app privacy policies produced for GDPR weren't more readable and were longer.

CONCLUSIONS

Apps specifically aimed at people with mental health difficulties are not accessible and even those that fulfilled the FDA's recommendation for reading grade contained jargon words. Developers and designers can increase accessibility by following a few rules and should, before launching, check whether the privacy policy can be understood.

User Perspectives on a Resilience-Building App (JoyPop): Qualitative Study

BACKGROUND

Resilience is the capability, resources, and processes that are available to a person or system to adapt successfully in the face of stress or adversity. Given that resilience can be enhanced, using advances in technology to deliver and evaluate the impact of resilience interventions is warranted. Evidence supports the effectiveness of the resilience-building JoyPop app in improving resilience-related outcomes after use; however, experiential data from users is also needed to provide a more comprehensive account of its utility.

OBJECTIVE

The aim of this study was to explore users' experiences with the JoyPop app and their perspectives on its utility.

METHODS

This qualitative description study involved a combination of group and one-on-one semistructured interviews with a subset of first-year undergraduate students who participated in a larger evaluation of the JoyPop app. Participants used the app for a 4-week period and were subsequently asked about their frequency of app use, most and least used features (and associated reasons), most and least helpful features (and associated reasons), barriers to use, facilitators of use and continuation, and recommendations for improvement. Data were coded and categorized through inductive content analysis.

RESULTS

The sample of 30 participants included 24 females and 6 males, with a mean age of 18.77 years (SD 2.30). App use ranged from 1 to 5 times daily (mean 2.11, SD 0.74), with the majority indicating that they used the app at least twice daily. The Rate My Mood, Journal, and SquareMoves features were reported to be used most often, while the Rate My Mood, Journal, and Breathing Exercises features were identified as the most helpful. A number of themes and subthemes pertaining to facilitators of app use (prompts, creating routine, self-monitoring opportunities, expressive opportunities), barriers to app use (editing, lack of variety, student lifestyle), outcomes of app use (increased awareness, checking in with oneself, helpful distraction, emotional control), and recommendations for app improvement (adding more features, enhancing existing features, enhancing tracking abilities, providing personalization) were identified.

CONCLUSIONS

This study provides insight into the aspects of the JoyPop app that motivated and benefitted users, as well as measures that can be taken to improve user experiences and promote longer-term uptake. Users were willing to engage with the app and incorporate it into their routine, and they valued the ability to self-monitor, express emotion, and engage in distraction.

Initial Training for Mental Health Peer Support Workers: Systematized Review and International Delphi Consultation

BACKGROUND

Initial training is essential for the mental health peer support worker (PSW) role. Training needs to incorporate recent advances in digital peer support and the increase of peer support work roles internationally. There is a lack of evidence on training topics that are important for initial peer support work training and on which training topics can be provided on the internet.

OBJECTIVE

The objective of this study is to establish consensus levels about the content of initial training for mental health PSWs and the extent to which each identified topic can be delivered over the internet.

METHODS

A systematized review was conducted to identify a preliminary list of training topics from existing training manuals. Three rounds of Delphi consultation were then conducted to establish the importance and web-based deliverability of each topic. In round 1, participants were asked to rate the training topics for importance, and the topic list was refined. In rounds 2 and 3, participants were asked to rate each topic for importance and the extent to which they could be delivered over the internet.

RESULTS

The systematized review identified 32 training manuals from 14 countries: Argentina, Australia, Brazil, Canada, Chile, Germany, Ireland, the Netherlands, Norway, Scotland, Sweden, Uganda, the United Kingdom, and the United States. These were synthesized to develop a preliminary list of 18 topics. The Delphi consultation involved 110 participants (49 PSWs, 36 managers, and 25 researchers) from 21 countries (14 high-income, 5 middle-income, and 2 low-income countries). After the Delphi consultation (round 1: n=110; round 2: n=89; and round 3: n=82), 20 training topics (18 universal and 2 context-specific) were identified. There was a strong consensus about the importance of five topics: lived experience as an asset, ethics, PSW well-being, and PSW role focus on recovery and communication, with a moderate consensus for all other topics apart from the knowledge of mental health. There was no clear pattern of differences among PSW, manager, and researcher ratings of importance or between responses from participants in countries with different resource levels. All training topics were identified with a strong consensus as being deliverable through blended web-based and face-to-face training (rating 1) or fully deliverable on the internet with moderation (rating 2), with none identified as only deliverable through face-to-face teaching (rating 0) or deliverable fully on the web as a stand-alone course without moderation (rating 3).

CONCLUSIONS

The 20 training topics identified can be recommended for inclusion in the curriculum of initial training programs for PSWs. Further research on web-based delivery of initial training is needed to understand the role of web-based moderation and whether web-based training better prepares recipients to deliver web-based peer support.

Using apps for bipolar disorder - An online survey of healthcare provider perspectives and practices

BACKGROUND

Smartphone apps have recognized potential for improving access to evidence-based care in the treatment of bipolar disorder (BD). Healthcare providers are well-positioned to play a role in guiding patients to access safe, evidence-supported, and trustworthy apps. However, little is known about whether and how clinicians use apps with people with BD: understanding practices and attitudes of healthcare providers is essential to support the implementation of mHealth interventions in a real-world context.

METHODS

A web-based survey was used to explore clinicians' attitudes towards, and use of apps when working with people with BD. Descriptive statistics were used to summarize quantitative findings. Free text responses were investigated using qualitative content analysis.

RESULTS

Eighty healthcare providers completed the survey. Approximately half of the respondents reported discussing or recommending apps in clinical practice with BD populations. Recommended apps were most commonly related to mood, sleep, and exercise. Barriers to discussing apps included a lack of healthcare provider knowledge/confidence, concerns about patients' ability to access apps, and beliefs that patients lacked interest in apps.

CONCLUSION

Although research suggests that people with BD are interested in using apps, uptake of such technology among clinicians is more limited. A lack of clinician knowledge regarding apps, combined with concerns about the digital divide and patient interest, may account for this relatively limited integration of apps into the management of BD. These findings emphasise the importance of considering the information needs of healthcare providers when planning dissemination strategies for app-based interventions for BD.

Postsecondary Student Engagement With a Mental Health App and Online Platform (Thought Spot): Qualitative Study of User Experience

BACKGROUND

There is growing interest in using mobile apps and online tools to support postsecondary student mental health, but most of these solutions have suboptimal user engagement in real-world settings. Poor engagement can limit long-term effectiveness and usefulness of these tools. Previous literature has proposed several theories that link factors such as low usability and poor user-centered design to app disengagement. However, few studies provide direct evidence showing what factors contribute to suboptimal user engagement in the context of mobile mental health apps for postsecondary students.

OBJECTIVE

This study focuses on understanding postsecondary students' attitudes and behaviors when using Thought Spot, a co-designed mental health app and online platform, to understand factors related to engagement and user experience.

METHODS

Students who were given access to Thought Spot for 6 months during a randomized trial of the intervention were invited to participate in one-on-one semistructured interviews. The interviews explored participants' overall experiences and perceptions of the app, along with factors that affected their usage of various features. All interviews were recorded, and template analysis was used to analyze transcripts.

RESULTS

User satisfaction was mixed among users of Thought Spot. The degree of engagement with the app appeared to be affected by factors that can be grouped into 5 themes: (1) Students valued detailed, inclusive, and relevant content; (2) Technical glitches and a lack of integration with other apps affected the overall user experience and satisfaction with the app; (3) Using the app to support peers or family can increase engagement; (4) Crowdsourced information from peers about mental health resources drove user engagement, but was difficult to obtain; and (5) Users often turned to the app when they had an immediate need for mental health information, rather than using it to track mental health information over time.

CONCLUSIONS

Content, user experience, user-centeredness, and peer support are important determinants of user engagement with mobile mental health apps among postsecondary students. In this study, participants disengaged when the app did not meet their expectations on these determinants. Future studies on user engagement should further explore the effectiveness of different features and the relative importance of various criteria for high-quality apps. Further focus on these issues may inform the creation of interventions that increase student engagement and align with their mental health needs.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03412461; https://clinicaltrials.gov/ct2/show/NCT03412461.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/resprot.6446.

Digital Interventions to Support Population Mental Health in Canada During the COVID-19 Pandemic: Rapid Review

BACKGROUND

The COVID-19 pandemic has resulted in a number of negative health related consequences, including impacts on mental health. More than 22% of Canadians reported that they had felt depressed in the last week, in response to a December 2020 national survey. Given the need to physically distance during the pandemic, and the increase in demand for mental health services, digital interventions that support mental health and wellness may be beneficial.

OBJECTIVE

The purpose of this research was to identify digital interventions that could be used to support the mental health of the Canadian general population during the COVID-19 pandemic. The objectives were to identify (1) the populations these interventions were developed for, inclusive of exploring areas of equity such as socioeconomic status, sex/gender, race/ethnicity and culture, and relevance to Indigenous peoples and communities; (2) the effect of the interventions; and (3) any barriers or facilitators to the use of the intervention.

METHODS

This study was completed using a Cochrane Rapid Review methodology. A search of Embase, PsycInfo, Medline, and Web of Science, along with Google, Million Short, and popular mobile app libraries, was conducted. Two screeners were involved in applying inclusion criteria using Covidence software. Academic articles and mobile apps identified were screened using the Standard Quality Assessment Criteria for Evaluating Primary Research Papers from a Variety of Fields resource, the American Psychiatric Association App Evaluation Framework, and the Mental Health Commission of Canada's guidance on app assessment and selection.

RESULTS

A total of 31 mobile apps and 114 web-based resources (eg, telemedicine, virtual peer support groups, discussion forums, etc) that could be used to support the mental health of the Canadian population during the pandemic were identified. These resources have been listed on a publicly available website along with search tags that may help an individual make a suitable selection. Variability exists in the populations that the interventions were developed for, and little assessment has been done with regard to areas of equity. The effect of the interventions was not reported for all those identified in this synthesis; however, for those that did report the effect, it was shown that they were effective in the context that they were used. A number of barriers and facilitators to using these interventions were identified, such as access, cost, and connectivity.

CONCLUSIONS

A number of digital interventions that could support population mental health in Canada during the global COVID-19 pandemic were identified, indicating that individuals have several options to choose from. These interventions vary in their purpose, approach, design, cost, and targeted user group. While some research and digital interventions addressed equity-related considerations, more research and focused attention should be given to this area.

Evaluation of Treatment Descriptions and Alignment With Clinical Guidance of Apps for Depression on App Stores: Systematic Search and Content Analysis

BACKGROUND

The use of apps for the treatment of depression shows great promise. However, there is uncertainty regarding the alignment of publicly available apps for depression with clinical guidance, their treatment fidelity and evidence base, and their overall safety.

OBJECTIVE

Built on previous analyses and reviews, this study aims to explore the treatment and safety issues of publicly available apps for depression.

METHODS

We conducted a content analysis of apps for depression in the 2 main UK app stores (Google Play and Apple App Store). App store listings were analyzed for intervention content, treatment fidelity, and fit with the National Institute for Health and Care Excellence (NICE) guidelines for the treatment of depression in adults.

RESULTS

A total of 353 apps for depression were included in the review. App descriptions reported the use of 20 treatment approaches and 37 treatment strategies. Many apps used transdiagnostic (155/353, 43.9%) and multitheoretical interventions to treat multiple disorders including depression. Although many interventions appeared to be evidence-informed, there were issues with treatment fidelity, research evidence, and fit with clinical guidelines. None of the apps fully aligned with the NICE guidelines for depression.

CONCLUSIONS

App developers have adopted many evidence-informed treatments in their interventions; however, more work is needed to improve clinical validity, treatment fidelity, and the safety of apps. We urge developers to consult relevant guidelines and standards, and to engage in reflective questioning on treatment and safety to address these issues and to improve treatment content and intervention design.

LionVu 2.0 Usability Assessment for Pennsylvania, United States

The Penn State Cancer Initiative implemented LionVu 1.0 (Penn State University, United States) in 2017 as a web-based mapping tool to educate and inform public health professionals about the cancer burden in Pennsylvania and 28 counties in central Pennsylvania, locally known as the catchment area. The purpose of its improvement, LionVu 2.0, was to assist investigators answer person-place-time questions related to cancer and its risk factors by examining several data variables simultaneously. The primary objective of this study was to conduct a usability assessment of a prototype of LionVu 2.0 which included area- and point-based data. The assessment was conducted through an online survey; 10 individuals, most of whom had a masters or doctorate degree, completed the survey. Although most participants had a favorable view of LionVu 2.0, many had little to no experience with web mapping. Therefore, it was not surprising to learn that participants wanted short 10-15-minute training videos to be available with future releases, and a simplified user-interface that removes advanced functionality. One unexpected finding was the suggestion of using LionVu 2.0 for teaching and grant proposals. The usability study of the prototype of LionVu 2.0 provided important feedback for its future development.

Connected Mental Health: Systematic Mapping Study

BACKGROUND

Although mental health issues constitute an increasing global burden affecting a large number of people, the mental health care industry is still facing several care delivery barriers such as stigma, education, and cost. Connected mental health (CMH), which refers to the use of information and communication technologies in mental health care, can assist in overcoming these barriers.

OBJECTIVE

The aim of this systematic mapping study is to provide an overview and a structured understanding of CMH literature available in the Scopus database.

METHODS

A total of 289 selected publications were analyzed based on 8 classification criteria: publication year, publication source, research type, contribution type, empirical type, mental health issues, targeted cohort groups, and countries where the empirically evaluated studies were conducted.

RESULTS

The results showed that there was an increasing interest in CMH publications; journals were the main publication channels of the selected papers; exploratory research was the dominant research type; advantages and challenges of the use of technology for mental health care were the most investigated subjects; most of the selected studies had not been evaluated empirically; depression and anxiety were the most addressed mental disorders; young people were the most targeted cohort groups in the selected publications; and Australia, followed by the United States, was the country where most empirically evaluated studies were conducted.

CONCLUSIONS

CMH is a promising research field to present novel approaches to assist in the management, treatment, and diagnosis of mental health issues that can help overcome existing mental health care delivery barriers. Future research should be shifted toward providing evidence-based studies to examine the effectiveness of CMH solutions and identify related issues.

Treating Psychological Trauma in the Midst of COVID-19: The Role of Smartphone Apps

With the COVID-19 pandemic confronting health systems worldwide, medical practitioners are treating a myriad of physical symptoms that have, sadly, killed many thousands of people. There are signs that the public is also experiencing psychological trauma as they attempt to navigate their way through the COVID-19 restrictions impinging on many aspects of society. With unprecedented demand for health professionals' time, people who are unable to access face-to-face assistance are turning to smartphone apps to help them deal with symptoms of trauma. However, the evidence for smartphone apps to treat trauma is limited, and clinicians need to be aware of the limitations and unresolved issues involved in using mental health apps.

The Mobile App Development and Assessment Guide (MAG): Delphi-Based Validity Study

Background

In recent years, there has been an exponential growth of mobile health (mHealth)–related apps. This has occurred in a somewhat unsupervised manner. Therefore, having a set of criteria that could be used by all stakeholders to guide the development process and the assessment of the quality of the apps is of most importance.

Objective

The aim of this paper is to study the validity of the Mobile App Development and Assessment Guide (MAG), a guide recently created to help stakeholders develop and assess mobile health apps.

Methods

To conduct a validation process of the MAG, we used the Delphi method to reach a consensus among participating stakeholders. We identified 158 potential participants: 45 patients as potential end users, 41 health care professionals, and 72 developers. We sent participants an online survey and asked them to rate how important they considered each item in the guide to be on a scale from 0 to 10. Two rounds were enough to reach consensus.

Results

In the first round, almost one-third (n=42) of those invited participated, and half of those (n=24) also participated in the second round. Most items in the guide were found to be important to a quality mHealth-related app; a total of 48 criteria were established as important. “Privacy,” “security,” and “usability” were the categories that included most of the important criteria.

Conclusions

The data supports the validity of the MAG. In addition, the findings identified the criteria that stakeholders consider to be most important. The MAG will help advance the field by providing developers, health care professionals, and end users with a valid guide so that they can develop and identify mHealth-related apps that are of quality.

Mobile health app usability and quality rating scales: a systematic review

PURPOSE

To review the rating scales used to evaluate usability and quality of mobile health applications, and to compare their purpose, content, and intended target users (i.e., patients, caregivers, or researchers).

MATERIAL AND METHODS

We conducted a systematic review of the literature in accordance with the PRISMA statement on Medline, CINAHL, PsycINFO, IEEE Explore databases, as well as a review of the grey literature to identify rating scales used to evaluate usability and quality of mobile health applications (m-health apps), between January 1, 2000 and July 31, 2018. Two researchers screened the titles and abstracts of articles that met inclusion criteria, and retrieved usability and quality rating scales from the articles.

RESULTS

We identified 24 usability scales and 25 quality rating scales in 87 peer-reviewed articles. We identified only one quality rating scale designed for non-expert users (i.e., patients or caregivers). None of the studies used a theoretical framework for app evaluation to support the scales. The validity of existing quality rating scales is yet to be investigated.

CONCLUSION

Existing usability and quality rating scales are targeted at professionals, not end users who are patients or caregivers. Rating scales that are usable by all end-users would make mobile health apps accessible and meaningful to consumers.Implications for rehabilitationThe number of mobile health applications on app stores that can be used for rehabilitation is increasing.Most healthcare providers lack the training to identify m-health apps with high quality to be used in rehabilitation.This study has reviewed the current rating scales that can help clinicians and care providers rate the quality of m-health apps and identify the ones that are most appropriate for their practice.

Stakeholder perspectives on evidence for digital mental health interventions: Implications for accreditation systems

Background

Digital mental health interventions can be effective for treating mental health problems, but uptake by consumers and clinicians is not optimal. The lack of an accreditation pathway for digital mental health interventions is a barrier to their uptake among clinicians and consumers. However, there are a number of factors that may contribute to whether a digital intervention is suitable for recommendation to the public. The aim of this study was to identify the types of evidence that would support the accreditation of digital interventions.

Method

An expert workshop was convened, including researcher, clinician, consumer (people with lived experience of a mental health condition) and policymaker representatives.

Results

Existing methods for assessing the evidence for digital mental health interventions were discussed by the stakeholders present at the workshop. Empirical evidence from randomised controlled trials was identified as a key component for evaluating digital interventions. However, information on the safety of users, data security, user ratings, and fidelity to clinical guidelines, along with data from routine care including adherence, engagement and clinical outcomes, were also identified as important considerations when evaluating an intervention. There are considerable challenges in weighing the evidence for a digital mental health intervention.

Conclusions

Empirical evidence should be the cornerstone of any accreditation system to identify appropriate digital mental health interventions. However, robust accreditation systems should also account for program and user safety, user engagement and experience, and fidelity to clinical treatment guidelines.

Mobile Apps for Medication Management: Review and Analysis

Background

Pharmacotherapy remains one of the major interventional strategies in medicine. However, patients from all age groups and conditions face challenges when taking medications, such as integrating them into the daily routine, understanding their effects and side effects, and monitoring outcomes. In this context, a reliable medication management tool adaptable to the patient’s needs becomes critical. As most people have a mobile phone, mobile apps offer a platform for such a personalized support tool available on the go.

Objective

This study aimed to provide an overview of available mobile apps, focusing on those that help patients understand and take their medications. We reviewed the existing apps and provided suggestions for future development based on the concept understand and manage, instead of the conventional adhere to medication. This concept aims to engage and empower patients to be in charge of their health, as well as see medication as part of a broader clinical approach, working simultaneously with other types of interventions or lifestyle changes, to achieve optimal outcomes.

Methods

We performed a Web search in the iOS Apple App Store and Android Google Play Store, using 4 search terms: medication management, pill reminder, medication health monitor, and medication helper. We extracted information from the app store descriptions for each eligible app and categorized into the following characteristics: features, author affiliation, specialty, user interface, cost, and user rating. In addition, we conducted Google searches to obtain more information about the author affiliation.

Results

A total of 328 apps (175 Android and 153 iOS) were categorized. The majority of the apps were developed by the software industry (73%, 11/15), a minority of them were codeveloped by health care professionals (15%, 3/20) or academia (2.1%; 7/328). The most prevalent specialty was diabetes (23 apps). Only 7 apps focused on mental health, but their content was highly comprehensive in terms of features and had the highest prevalence of the education component. The most prevalent features were reminder, symptom tracker, and ability to share data with a family member or doctor. In addition, we highlighted the features considered innovative and listed practical suggestions for future development and innovations.

Conclusions

We identified detailed characteristics of the existing apps, with the aim of informing future app development. Ultimately, the goal was to provide users with effective mobile health solutions, which can be expected to improve their engagement in the treatment process and long-term well-being. This study also highlighted the need for improved standards for reporting on app stores. Furthermore, it underlined the need for a platform to offer health app users an ongoing evaluation of apps by health professionals in addition to other users and to provide them with tools to easily select an appropriate and trustworthy app.

Why Reviewing Apps Is Not Enough: Transparency for Trust (T4T) Principles of Responsible Health App Marketplaces

The overselling of health apps that may provide little benefit and even harm needs the health community’s immediate attention. With little formal regulation, a light-touch approach to consumer protection is now warranted to give customers a modicum of information to help them choose from the vast array of so-called health apps. We suggest 4 guiding principles that should be adopted to provide the consumer with information that can guide their choice at the point of download. We call these the Transparency for Trust (T4T) principles, which are derived from experimental studies, systematic reviews, and reports of patient concerns. The T4T principles are (1) privacy and data security, (2) development characteristics, (3) feasibility data, and (4) benefits. All our questions are in a simple form so that all consumers can understand them. We suggest that app stores should take responsibility for providing this information and store it with any app marketed as a health app. Even the absence of information would provide consumers with some understanding and fuel their choice. This would also provide some commercial impetus for app developers to consider this requested information from the outset.

Understanding the quality, effectiveness and attributes of top-rated smartphone health apps

Objective

This study aimed to understand the attributes of popular apps for mental health and comorbid medical conditions, and how these qualities relate to consumer ratings, app quality and classification by the WHO health app classification framework.

Methods

We selected the 10 apps from the Apple iTunes store and the US Android Google Play store on 20 July 2018 from six disease states: depression, anxiety, schizophrenia, addiction, diabetes and hypertension. Each app was downloaded by two authors who provided information on the apps’ attributes, functionality, interventions, popularity, scientific backing and WHO app classification rating.

Results

A total of 120 apps were examined. Although none of these apps had Food and Drug Administration marketing approval, nearly 50% made claims that appeared medical. Most apps offered a similar type of services with 87.5% assigned WHO classification 1.4.2 ‘self-monitoring of health or diagnostic data by a client’ or 1.6.1 ‘client look-up of health information’. The ‘last updated’ attribute was highly correlated with a quality rating of the app although no apps features (eg, uses Global Positioning System, reminders and so on) were.

Conclusion

Due to the heterogeneity of the apps, we were unable to define a core set of features that would accurately assess app quality. The number of apps making unsupported claims combined with the number of apps offering questionable content warrants a cautious approach by both patients and clinicians in selecting safe and effective ones.

Clinical Implications

‘Days since last updated’ offers a useful and easy clinical screening test for health apps, regardless of the condition being examined.