Development and validation of the Italian version of the Mobile Application Rating Scale and its generalisability to apps targeting primary prevention

A Mindfulness-Based App Intervention for Pregnant Women: Protocol for a Pilot Feasibility Study

BACKGROUND

Pregnancy is a complex time characterized by major transformations in a woman, which impact her physical, mental, and social well-being. How a woman adapts to these changes can affect her quality of life and psychological well-being. The literature indicates that pregnant women commonly experience psychological symptoms, with anxiety, stress, and depression being among the most frequent. Hence, promoting a healthy lifestyle focused on women's psychological well-being is crucial. Recently developed digital solutions have assumed a crucial role in supporting psychological well-being in physiologically pregnant women. Therefore, the need becomes evident for the development and implementation of digital solutions, such as a virtual coach implemented in a smartphone, as a support for the psychological well-being of pregnant women who do not present psychological and psychiatric disorders.

OBJECTIVE

This study aims to assess the feasibility, acceptability, and utility of a mindfulness-based mobile app. The primary objective is to explore the feasibility of using a virtual coach, Maia, developed within the TreC Mamma app to promote women's psychological well-being during pregnancy through a psychoeducational module based on mindfulness. Finally, through the delivery of this module, the level of psychological well-being will be explored as a secondary objective.

METHODS

This is a proof-of-concept study in which a small sample (N=50) is sufficient to achieve the intended purposes. Recruitment will occur within the group of pregnant women belonging to the pregnancy care services of the Trento Azienda Provinciale per i Servizi Sanitari di Trento. The convenience sampling method will be used. Maia will interact with the participating women for 8 weeks, starting from weeks 24 and 26 of pregnancy. Specifically, there will be 2 sessions per week, which the woman can choose, to allow more flexibility toward her needs.

RESULTS

The psychoeducational pathway is expected to lead to significant results in terms of usability and engagement in women's interactions with Maia. Furthermore, it is anticipated that there will be improvements in psychological well-being and overall quality of life. The analysis of the data collected in this study will be mainly descriptive, orientated toward assessing the achievement of the study objectives.

CONCLUSIONS

Literature has shown that women preferred web-based support during the perinatal period, suggesting that implementing digital interventions can overcome barriers to social stigma and asking for help. Maia can be a valuable resource for regular psychoeducational support for women during pregnancy.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR1-10.2196/53890.

Support for Chronic Pain Management for Breast Cancer Survivors Through Novel Digital Health Ecosystems: Pilot Usability Study of the PainRELife Mobile App

BACKGROUND

Chronic pain is one of the most common and critical long-term effects of breast cancer. Digital health technologies enhance the management of chronic pain by monitoring physical and psychological health status and supporting pain self-management and patient treatment decisions throughout the clinical pathway.

OBJECTIVE

This pilot study aims to evaluate patients' experiences, including usability, with a novel digital integrated health ecosystem for chronic pain named PainRELife. The sample included patients with breast cancer during survivorship. The PainRELife ecosystem comprises a cloud technology platform interconnected with electronic health records and patients' devices to gather integrated health care data.

METHODS

We enrolled 25 patients with breast cancer (mean age 47.12 years) experiencing pain. They were instructed to use the PainRELife mobile app for 3 months consecutively. The Mobile Application Rating Scale (MARS) was used to evaluate usability. Furthermore, pain self-efficacy and participation in treatment decisions were evaluated. The study received ethical approval (R1597/21-IEO 1701) from the Ethical Committee of the European Institute of Oncology.

RESULTS

The MARS subscale scores were medium to high (range: 3.31-4.18), and the total app quality score was 3.90. Patients with breast cancer reported reduced pain intensity at 3 months, from a mean of 5 at T0 to a mean of 3.72 at T2 (P=.04). The total number of times the app was accessed was positively correlated with pain intensity at 3 months (P=.03). The engagement (P=.03), information (P=.04), and subjective quality (P=.007) subscales were positively correlated with shared decision-making. Furthermore, participants with a lower pain self-efficacy at T2 (mean 40.83) used the mobile app more than participants with a higher pain self-efficacy (mean 48.46; P=.057).

CONCLUSIONS

The data collected in this study highlight that digital health technologies, when developed using a patient-driven approach, might be valuable tools for increasing participation in clinical care by patients with breast cancer, permitting them to achieve a series of key clinical outcomes and improving quality of life. Digital integrated health ecosystems might be important tools for improving ongoing monitoring of physical status, psychological burden, and socioeconomic issues during the cancer survivorship trajectory.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/41216.

Assessment of healthcare application quality: Development of a standardized methods for healthcare professionals

The advancement of the mobile app market is reshaping healthcare, emphasizing the imperative for quality and efficacy in health applications. This methodology has been devised to assess mobile health applications, aiming to assist healthcare professionals in selecting apps for e-healthcare consumers. Key facets of this approach are: •A stringent selection process within mobile app stores•A standardized assessment using the Mobile App Rating Scale to achieve consistent and replicable evaluations, systematically organizing app evaluations•A comprehensive framework guiding healthcare practitioners in determining which apps to integrate into clinical practice and which to exclude Central to this method is the emphasis on distinguishing apps that enhance clinical practice from those that fall short in important areas such as the effectiveness of proposed health features, data accuracy, adherence to clinical guidelines, data security, and user privacy. With heightened attention to usability and accessibility, the methodology also addresses the common risks inherent in mHealth implementation, ensuring that selected apps not only meet technical criteria but also align with the broader healthcare ecosystem's needs and challenges.

The Most Popular Commercial Weight Management Apps in the Chinese App Store: Analysis of Quality, Features, and Behavior Change Techniques

BACKGROUND

Many smartphone apps designed to assist individuals in managing their weight are accessible, but the assessment of app quality and features has predominantly taken place in Western countries. Nevertheless, there is a scarcity of research evaluating weight management apps in China, which highlights the need for further investigation in this area.

OBJECTIVE

This study aims to conduct a comprehensive search for the most popular commercial Chinese smartphone apps focused on weight management and assess their quality, behavior change techniques (BCTs), and content-related features using appropriate evaluation scales. Additionally, the study sought to investigate the associations between the quality of various domains within weight management apps and the number of incorporated BCTs and app features.

METHODS

In April 2023, data on weight management apps from the iOS and Android app stores were downloaded from the Qimai Data platform. Subsequently, a total of 35 weight management apps were subjected to screening and analysis by 2 researchers. The features and quality of the apps were independently assessed by 6 professionals specializing in nutrition management and health behavioral change using the Mobile Application Rating Scale (MARS). Two registered dietitians, who had experience in app development and coding BCTs, applied the established 26-item BCT taxonomy to verify the presence of BCTs. Mean (SD) scores and their distributions were calculated for each section and item. Spearman correlations were used to assess the relationship between an app's quality and its technical features, as well as the number of incorporated BCTs.

RESULTS

The data set included a total of 35 apps, with 8 available in the Android Store, 10 in the Apple Store, and 17 in both. The overall quality, with a mean MARS score of 3.44 (SD 0.44), showed that functionality was the highest scoring domain (mean 4.18, SD 0.37), followed by aesthetics (mean 3.43, SD 0.42), engagement (mean 3.26, SD 0.64), and information (mean 2.91, SD 0.52), which had the lowest score. The mean number of BCTs in the analyzed apps was 9.17 (range 2-18 BCTs/app). The most common BCTs were "prompt review of behavioral goals" and "provide instruction," present in 31 apps (89%). This was followed by "prompt self-monitoring of behavior" in 30 apps (86%), "prompt specific goal setting" in 29 apps (83%), and "provide feedback on performance" in 27 apps (77%). The most prevalent features in the analyzed apps were the need for web access (35/35, 100%), monitoring/tracking (30/35, 86%), goal setting (29/35, 83%), and sending alerts (28/35, 80%). The study also revealed strong positive correlations among the number of BCTs incorporated, app quality, and app features. This suggests that apps with a higher number of BCTs tend to have better overall quality and more features.

CONCLUSIONS

The study found that the overall quality of weight management apps in China is moderate, with a particular weakness in the quality of information provided. The most prevalent BCTs in these apps were reviewing behavioral goals, providing guidance, self-monitoring of behavior, goal setting, and offering performance feedback. The most common features were the need for web access, monitoring and tracking, goal setting, and sending alerts. Notably, higher-quality weight management apps in China tended to incorporate more BCTs and features. These findings can be valuable for developers looking to improve weight management apps and enhance their potential to drive behavioral change in weight management.

Home-Based Treatment for Chronic Pain Combining Neuromodulation, Computer-Assisted Training, and Telemonitoring in Patients With Breast Cancer: Protocol for a Rehabilitative Study

BACKGROUND

Chronic pain is a disabling symptom frequently reported in patients with breast cancer with a prevalence ranging from 25% to 60%, representing a major health issue. It has negative consequences on health status, causing psychological distress and affecting quality of life. Furthermore, the clinical management of chronic pain is often inadequate, and many patients do not benefit from the administration of pharmacological treatments. Alternative therapeutic options have been implemented to improve the psychophysical well-being of patients, including neuromodulation and complementary interventions.

OBJECTIVE

We aimed to investigate the effectiveness of a home care strategy combining computerized rehabilitation, transcranial direct current stimulation (tDCS), and remote telemonitoring via a web-based platform in patients with breast cancer suffering for chronic pain.

METHODS

A web-based structured survey aimed at monitoring chronic pain and its effect on psychological functions will be delivered to patients with breast cancer through social media and email. In total, 42 patients with breast cancer affected by chronic pain will be recruited during the medical screening visit. The patients will be randomly divided into 3 treatment groups that will carry out either tDCS only, exercise therapy only, or a combination of both over a 3-week period. All the treatments will be delivered at the patients' home through the use of a system including a tablet, wearable inertial sensors, and a tDCS programmable medical device. Using web-based questionnaires, the perception of pain (based on the pain self-efficacy questionnaire, visual analogue scale, pain catastrophizing scale, and brief pain inventory) and psychological variables (based on the hospital and anxiety depression scale and 12-item short form survey) will be assessed at the beginning of treatment, 1 week after the start of treatment, at the end of treatment, 1 month after the start of treatment, and 3 months after the start of treatment. The system's usability (based on the mobile app rating scale and system usability scale) and its involvement in the decision-making process (based on the 9-item shared decision-making questionnaire) will be also evaluated. Finally, at the end of the treatment, a digital focus group will be conducted with the 42 patients to explore their unexpressed needs and preferences concerning treatment.

RESULTS

The study project is scheduled to start in June 2023, and it is expected to be completed by August 2025.

CONCLUSIONS

We expect that the combination of tDCS and telemedicine programs will reduce pain perceived by patients with breast cancer and improve their mental well-being more effectively than single interventions. Furthermore, we assume that this home-based approach will also improve patients' participation in routine clinical care, reducing disparities in accessing health care processes. This integrated home care strategy could be useful for patients with breast cancer who cannot find relief from chronic pain with pharmacological treatments or for those who have limited access to care due to poor mobility or geographical barriers, thus increasing the patients' empowerment and reducing health care costs.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/49508.

Mobile applications for cognitive training: Content analysis and quality review

Background

As the number of individuals suffering from cognitive diseases continues to rise, dealing with the diminished cognitive function that comes with age has become a serious public health concern. While the use of mobile applications (apps) as digital treatments for cognitive training shows promise, the analysis of their content and quality remains unclear.

Objective

The aim of this study was to systematically search and assess cognitive training apps using the multidimensional mobile app rating scale (MARS) to rate objective quality and identify critical points.

Methods

A search was conducted on the Google Play Store and Apple App Store in February 2022 using the terms "cognitive training" and "cognitive rehabilitation." The cognitive domains provided by each app were analyzed, and the frequency and percentage according to the apps were obtained. The MARS, a mHealth app quality rating tool including multidimensional measures, was used to analyze the quality of the apps. The relationship between the MARS score, the number of reviews, and 5-star ratings were examined.

Results

Of the 53 apps, 52 (98 %) included memory function, 48 (91 %) included attention function, 24 (45 %) included executive function, and 19 (36 %) included visuospatial function. The mean (SD) scores of MARS, 5-star ratings, and reviews of 53 apps were 3.09 (0.61), 4.33 (0.30), and 62,415.43 (121,578.77). From the between-section comparison, engagement (mean 2.97, SD 0.68) obtained lower scores than functionality (mean 3.18, SD 0.62), aesthetics (mean 3.13, SD 0.72), and information (mean 3.11, SD 0.54). The mean quality score and reviews showed a statistically significant association (r = 0.447 and P = .001*). As the number of domains increased, the mean quality score showed a statistically significant increasing trend (P = .002*).

Conclusions

Most apps provided training for the memory and attention domains, but few apps included executive function or visuospatial domains. The quality of the apps improved significantly when more domains were provided, and was positively associated with the number of reviews received. These results could be useful for the future development of mobile apps for cognitive training.

Chronic kidney disease and mobile health: quality of renal nutritional APPs in Italy

BACKGROUND AND AIM

Patient with chronic kidney disease (CKD) must adhere to a nutritional therapy characterized by a restrictive dietary scheme. Nutritional self-care can be enhanced through the use of nutritional apps. The purpose of this study is to evaluate the characteristics of specific nutritional apps for chronic renal failure available in Italy.

METHODS

A systematic search of mobile apps was conducted by two academic researchers in three Italian App stores: Google Play Store, Apple Store and Huawei AppGallery. Of the 1602 apps identified, 2 apps (Miku; MyFIR) were evaluated with the Italian version of a Mobile Application Rating Scale (MARS-ITA) by a multidisciplinary team of 20 professionals.

RESULTS

The study found that the two selected apps, available in the Google Play Store and Apple Store, aim to increase well-being through the acquisition of knowledge and behavioral change; but none identify health goals that should be achieved. The strategies used by the two applications are: information, education, monitoring and cognitive-behavioral challenge. The technical analysis showed adequate protection of personal data but only the most downloaded app (Miku) provides the possibility to share content, to send reminders and to browse when offline.

CONCLUSIONS

The main Italian app stores have two applications to monitor physical health and nutrition in patients with renal disease. Although these are free, easily accessible, navigable and valid in terms of health monitoring and information, greater availability of the offer would be desirable.

Conducting a systematic review and evaluation of commercially available mobile applications (apps) on a health-related topic: the TECH approach and a step-by-step methodological guide

OBJECTIVES

To provide an overview of the methodological considerations for conducting commercial smartphone health app reviews (mHealth reviews), with the aim of systematising the process and supporting high-quality evaluations of mHealth apps.

DESIGN

Synthesis of our research team's experiences of conducting and publishing various reviews of mHealth apps available on app stores and hand-searching the top medical informatics journals (eg, The Lancet Digital Health, npj Digital Medicine, Journal of Biomedical Informatics and the Journal of the American Medical Informatics Association) over the last five years (2018-2022) to identify other app reviews to contribute to the discussion of this method and supporting framework for developing a research (review) question and determining the eligibility criteria.

RESULTS

We present seven steps to support rigour in conducting reviews of health apps available on the app market: (1) writing a research question or aims, (2) conducting scoping searches and developing the protocol, (3) determining the eligibility criteria using the TECH framework, (4) conducting the final search and screening of health apps, (5) data extraction, (6) quality, functionality and other assessments and (7) analysis and synthesis of findings. We introduce the novel TECH approach to developing review questions and the eligibility criteria, which considers the Target user, Evaluation focus, Connectedness and the Health domain. Patient and public involvement and engagement opportunities are acknowledged, including co-developing the protocol and undertaking quality or usability assessments.

CONCLUSION

Commercial mHealth app reviews can provide important insights into the health app market, including the availability of apps and their quality and functionality. We have outlined seven key steps for conducting rigorous health app reviews in addition to the TECH acronym, which can support researchers in writing research questions and determining the eligibility criteria. Future work will include a collaborative effort to develop reporting guidelines and a quality appraisal tool to ensure transparency and quality in systematic app reviews.

Greek validation of the user version of the Mobile Application Rating Scale (uMARS)

OBJECTIVE

The original user version of the Mobile Application Rating Scale (uMARS) is an English-language questionnaire that was designed to allow non-expert app users to assess the quality of health apps. We aimed to translate into the Greek language and validate the uMARS.

METHODS

This was a qualitative prospective study. The World Health Organization translation process was followed and a readily available and free-of-charge app was used for the validation process. Internal consistency and reliability were tested twice within one month by 91 Greek medical students.

RESULTS

The total uMARS score showed excellent internal consistency (Cronbach's alpha = 0.86). The internal consistencies of its subscales were also very high (engagement alpha = 0.71; functionality alpha = 0.71; aesthetics alpha = 0.67; information alpha = 0.63), with the notable exception of the satisfaction alpha, which was 0.61. The uMARS total score demonstrated almost perfect agreement levels in most of the subscales according to the r_WG index from baseline to 1 month.

CONCLUSIONS

The Greek uMARS is a reliable and valid tool for assessing the quality of mobile apps.

Quality Assessment of the Road Traffic Health and Safety Apps with a Focus on the Five Rights of Information Management

Objective

The expansion of mobile applications as a tool for road traffic health and safety may develop several issues from the perspective of information management. Quality assessment of these apps, especially from an information system management perspective, appears inevitable, as their possible low quality may cause irreversible injury or fatal consequences. This study aimed to evaluate the quality of the apps in the three subcategories of road traffic safety apps (including Accident Record and Report (ARR), Distraction Management (DM), and Vehicle Operating, Fixing, and Maintenance (VOFM)) using the Mobile Application Rating Scale (MARS), which rates 23 evaluation criteria organized in five domains (Engagement, Esthetics, Information, and Subjective Quality) with particular attention to the five rights framework of health information system.

Method

The researchers retrieved road traffic health and safety mobile apps from Google Play. First, the domain expert panel (n= 7) (from disciplines of HIM and medical informatics) was formed. They scrutinized and discussed the MARS items and mapped them into the five rights framework of information quality. Moreover, the researchers assigned the apps to the information system or decision support system category. Two researchers independently reviewed the apps and conducted the qualitative content analysis to categorize them into ARR, DM, and VOFM classes. Finally, the quality of the apps was assessed using the MARS rating scale (max=5) in terms of 1) app classification category with a descriptive aim; 2) app subjective and objective quality categories comprised of engagement, functionality, esthetics, and information sections; and 3) an optional app-specific section. The mean scores for the subjective quality, objective quality, and app-specific sections were calculated separately for each mobile app. A score ≥ 3.0 was considered acceptable.

Results

A total number of 42 apps met the criteria for the assessment. The average objective quality scores were computed as 2.6, 2.2, and 3.0 for the ARR, DM, and VOFM apps, respectively. Therefore, the quality of the apps in the ARR and DM subgroups was not acceptable. Moreover, the quality of the apps in the VOFM subcategory was considered moderate. Furthermore, the subjective quality and app-specific sections of apps in the ARR and DM categories were less than moderate. Most apps had the potential of an information system or decision support system. Also, the criteria measured by MARS could be mapped to the five rights framework of information management.

Conclusion

The findings of this study revealed the existing gaps in three subcategories of road traffic safety apps. Considering the multiple criteria of the MARS and having in mind the framework of five rights, developers of the apps may develop better products in road traffic health and safety.

Evaluating and rating HIV/AIDS mobile apps using the feature-based application rating method and mobile app rating scale

Background

The purpose of this study was to evaluate HIV/AIDS mobile applications using the Mobile App Rating Scale (MARS) and rate the features of these applications using the new tool called the Feature-based Application Rating Method (FARM).

Methods

In this study, all available HIV/AIDS apps in Iran from Cafe Bazaar and Google Play Store due to inclusion criteria were studied. The evaluation of the quality of applications was done using the MARS tool and the FARM tool. The FARM, which was developed in this study, was applied to rank the features of the applications.

Results

In this study, 79 applications were included. The mean score of all apps using both tools was 3.58 (SD = 0.95) out of 5. The overall mean quality score based on the MARS was 3.14 (SD = 0.84), and the mean score of features based on FARM was 3.81 (SD = 1.23). This study showed a higher than moderate correlation between the scores assigned to apps based on the MARS and FARM tools (r > 0.4).

Conclusions

The HIV/AIDS mobile applications available in Iran had the "acceptable" ranking. Also, our study results showed that to evaluate mobile apps, using a single tool may not provide good insight to evaluators about the assessed apps. However, using more than one tool may provide more details about the evaluated apps. To improve the quality of mobile health apps and help users select the most desirable app, we suggested using tools like FARM for ranking apps based on the features of each app in the app stores.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-02029-8.

Mobile Apps for the Management of Gastrointestinal Diseases: Systematic Search and Evaluation Within App Stores

Background

Gastrointestinal diseases are associated with substantial cost in health care. In times of the COVID-19 pandemic and further digitalization of gastrointestinal tract health care, mobile health apps could complement routine health care. Many gastrointestinal health care apps are already available in the app stores, but the quality, data protection, and reliability often remain unclear.

Objective

This systematic review aimed to evaluate the quality characteristics as well as the privacy and security measures of mobile health apps for the management of gastrointestinal diseases.

Methods

A web crawler systematically searched for mobile health apps with a focus on gastrointestinal diseases. The identified mobile health apps were evaluated using the Mobile Application Rating Scale (MARS). Furthermore, app characteristics, data protection, and security measures were collected. Classic user star rating was correlated with overall mobile health app quality.

Results

The overall quality of the mobile health apps (N=109) was moderate (mean 2.90, SD 0.52; on a scale ranging from 1 to 5). The quality of the subscales ranged from low (mean 1.89, SD 0.66) to good (mean 4.08, SD 0.57). The security of data transfer was ensured only by 11 (10.1%) mobile health apps. None of the mobile health apps had an evidence base. The user star rating did not correlate with the MARS overall score or with the individual subdimensions of the MARS (all P>.05).

Conclusions

Mobile health apps might have a positive impact on diagnosis, therapy, and patient guidance in gastroenterology in the future. We conclude that, to date, data security and proof of efficacy are not yet given in currently available mobile health apps.

Development and validation of the Turkish version of the Mobile App Rating Scale - MARS-TR

BACKGROUND

The number of mobile health apps (MHAs) is growing rapidly. MHAs have great potential to improve health and health care. However, the quality of available MHAs remains unknown due to the lack of quality assessment regulations and standards for MHAs. The Mobile Application Rating Scale (MARS) is the most widely used instrument to assess the quality of MHAs, and available in English, Italian, Spanish, German, French, Arabic and Japanese. However, the scale is currently not available in the Turkish language.

OBJECTIVE

This study aimed to cross-culturally adapt the MARS into Turkish and evaluate the validity and reliability of the scale.

METHODS

The MARS was translated and adapted into Turkish according to the international guidelines. A total of 52 pregnancy tracking apps were evaluated by two independent raters. Internal consistency (Cronbach's alpha), inter-rater reliability (Intraclass Correlation Coefficient [ICC]), convergent validity and concurrent validity were explored. Regarding convergent validity, MARS-TR scores were compared with the ENLIGHT scale.

RESULTS

The MARS-TR was highly aligned with the original MARS. The MARS-TR showed excellent internal consistency (Cronbach's alpha of 0.95 for both raters) and excellent inter-rater reliability (ICC = 0.94; SEM = 0.02), with a smallest detectable change (95 % confidence level) of 0.05 points for the total score. Cronbach's alphas for the subscales ranged from 0.76 to 0.94 for the two raters. Correlations between the MARS-TR and ENLIGHT demonstrated adequate convergent validity (P < 0.05). No ceiling or floor effects were observed.

CONCLUSION

The results provide evidence that the Turkish version of MARS is a valid and reliable tool for researchers and experts to assess the quality of MHAs in Turkey.

“Usability testing of a new digital integrated health ecosystem PainRELife for the clinical man-agement of chronic pain in early breast cancer patients: Protocol for a Pilot Study” (Preprint)

BACKGROUND

Chronic pain (CP) and its management are critical issues in the care pathway of patients with breast cancer. Considering the complexity of CP experience in cancer, the international scientific community has advocated identifying cutting-edge approaches for CP management. Recent advances in the field of health technology enable the adoption of a novel approach to care management by developing integrated ecosystems and mobile health apps.

OBJECTIVE

The primary end point of this pilot study is to evaluate patients' usability experience at 3 months of a new digital and integrated technological ecosystem, PainRELife, for CP in a sample of patients with breast cancer. The PainRELife ecosystem is composed of 3 main technological assets integrated into a single digital ecosystem: Fast Healthcare Interoperability Resources-based cloud platform (Nu platform) that enables care pathway definition and data collection; a big data infrastructure connected to the Fast Healthcare Interoperability Resources server that analyzes data and implements dynamic dashboards for aggregate data visualization; and an ecosystem of personalized applications for patient-reported outcomes collection, digital delivery of interventions and tailored information, and decision support of patients and caregivers (PainRELife app).

METHODS

This is an observational, prospective pilot study. Twenty patients with early breast cancer and chronic pain will be enrolled at the European Institute of Oncology at the Division of Medical Senology and the Division of Pain Therapy and Palliative Care. Each patient will use the PainRELife mobile app for 3 months, during which data extracted from the questionnaires will be sent to the Nu Platform that health care professionals will manage. This pilot study is nested in a large-scale project named "PainRELife," which aims to develop a cloud technology platform to interoperate with institutional systems and patients' devices to collect integrated health care data. The study received approval from the Ethical Committee of the European Cancer Institute in December 2021 (number R1597/21-IEO 1701).

RESULTS

The recruitment process started in May 2022 and ended in October 2022.

CONCLUSIONS

The new integrated technological ecosystems might be considered an encouraging affordance to enhance a patient-centered approach to managing patients with cancer. This pilot study will inform about which features the health technological ecosystems should have to be used by cancer patients to manage CP.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/41216.

Development and validation of the Japanese version of the uMARS (user version of the mobile app rating system)

BACKGROUND

Although the global market of Mobile Health Apps (mHealth apps) continues to grow dramatically, most mHealth apps still not only lack evidence base but have even not been evaluated for the basic usability or functionality. The User Version of the Mobile App Rating Scale (uMARS) was developed to allow end users to assess mHealth apps objectively and subjectively. However, there is no Japanese version of uMARS to date.

OBJECTIVE

The purpose of this study is (1) to develop a validated Japanese version of uMARS and (2) to assess the translated version's reliability and validity in evaluating mHealth apps.

METHODS

The original uMARS was adapted for Japanese use by four specialists using universalist cross-cultural methods. Translation/back-translation was reviewed by the author of the original version of uMARS, and confirmed. Its reliability and validity were further evaluated as part of a prospective cohort study of postoperative patients using a new mHealth app.

RESULTS

Conceptual equivalence was analyzed and all items in all subcategories of the original uMARS were included in the Japanese version. Internal consistency was deemed acceptable for all subscales of objective and subjective quality with a Cronbach's alpha of 0.75-0.85. Test-retest reliability of all subscales was also acceptable with intraclass correlation coefficients (ICCs) of 0.57-0.88. Convergent/divergent validity and concurrent validity were also considered acceptable.

CONCLUSION

A Japanese version of uMARS was cross-culturally validated and found to be as reliable as the original uMARS. This Japanese version of uMARS is expected to become a standard tool in assessing the quality of mHealth apps in Japan.

The Quality of Indian Obesity-Related mHealth Apps: PRECEDE-PROCEED Model–Based Content Analysis

Background

The prevalence of obesity in India is increasing at an alarming rate. Obesity-related mHealth apps have proffered an exciting opportunity to remotely deliver obesity-related information. This opportunity raises the question of whether such apps are truly effective.

Objective

The aim of this study was to identify existing obesity-related mHealth apps in India and evaluate the potential of the apps’ contents to promote health behavior change. This study also aimed to discover the general quality of obesity-related mHealth apps.

Methods

A systematic search for obesity-related mHealth apps was conducted in both the Google Play Store and the Apple App Store. The features and quality of the sample apps were assessed using the Mobile Application Rating Scale (MARS) and the potential of the sample apps’ contents to promote health behavior change was assessed using the PRECEDE-PROCEED Model (PPM).

Results

A total of 13 apps (11 from the Google Play Store and 2 from the Apple App Store) were considered eligible for the study. The general quality of the 13 apps assessed using MARS resulted in mean scores ranging from 1.8 to 3.7. The bivariate Pearson correlation between the MARS rating and app user rating failed to establish statistically significant results. The multivariate regression analysis result indicated that the PPM factors are significant determinants of health behavior change (F3,9=63.186; P<.001) and 95.5% of the variance (R2=0.955; P<.001) in the dependent variable (health behavior change) can be explained by the independent variables (PPM factors).

Conclusions

In general, mHealth apps are found to be more effective when they are based on theory. The presence of PPM factors in an mHealth app can greatly influence the likelihood of health behavior change among users. So, we suggest mHealth app developers consider this to develop efficient apps. Also, mHealth app developers should consider providing health information from credible sources and indicating the sources of the information, which will increase the perceived credibility of the apps among the users. We strongly recommend health professionals and health organizations be involved in the development of mHealth apps. Future research should include mHealth app users to understand better the apps’ effectiveness in bringing about health behavior change.

Japanese Version of the Mobile App Rating Scale (MARS): Development and Validation

Background

The number of mobile health (mHealth) apps continues to rise each year. Widespread use of the Mobile App Rating Scale (MARS) has allowed objective and multidimensional evaluation of the quality of these apps. However, no Japanese version of MARS has been made available to date.

Objective

The purposes of this study were (1) to develop a Japanese version of MARS and (2) to assess the translated version’s reliability and validity in evaluating mHealth apps.

Methods

To develop the Japanese version of MARS, cross-cultural adaptation was used using a universalist approach. A total of 50 mental health apps were evaluated by 2 independent raters. Internal consistency and interrater reliability were then calculated. Convergent and divergent validity were assessed using multitrait scaling analysis and concurrent validity.

Results

After cross-cultural adaptation, all 23 items from the original MARS were included in the Japanese version. Following translation, back-translation, and review by the author of the original MARS, a Japanese version of MARS was finalized. Internal consistency was acceptable by all subscales of objective and subjective quality (Cronbach α=.78-.89). Interrater reliability was deemed acceptable, with the intraclass correlation coefficient (ICC) ranging from 0.61 to 0.79 for all subscales, except for “functionality,” which had an ICC of 0.40. Convergent/divergent validity and concurrent validity were also considered acceptable. The rate of missing responses was high in several items in the “information” subscale.

Conclusions

A Japanese version of MARS was developed and shown to be reliable and valid to a degree that was comparable to the original MARS. This Japanese version of MARS can be used as a standard to evaluate the quality and credibility of mHealth apps.

Review of Mobile Apps for Women With Anxiety in Pregnancy: Maternity Care Professionals' Guide to Locating and Assessing Anxiety Apps

Background

Mental health and pregnancy apps are widely available and have the potential to improve health outcomes and enhance women’s experience of pregnancy. Women frequently access digital information throughout their pregnancy. However, health care providers and women have little information to guide them toward potentially helpful or effective apps.

Objective

This review aimed to evaluate a methodology for systematically searching and reviewing commercially available apps that support pregnant women with symptoms of anxiety in order to assist maternity care professionals in identifying resources that they could recommend for these women.

Methods

A stepwise systematic approach was used to identify, select, describe, and assess the most popular and highly user-rated apps available in the United Kingdom from January to March 2021. This included developing a script-based search strategy and search process, writing evaluation criteria, and conducting a narrative description and evaluation of the selected apps.

Results

Useful search terms were identified, which included nonclinical, aspirational, and problem-based phrases. There were 39 apps selected for inclusion in the review. No apps specifically targeted women with anxiety in pregnancy. Of the 39 apps included in the review, 33 (85%) focused solely on mind-body techniques to promote relaxation, stress reduction, and psychological well-being. Only 8 of the 39 (21%) apps included in the review reported that health care professionals had contributed to app development and only 1/39 (3%) provided empirical evidence on the effectiveness and acceptability of the app. The top 12/39 (31%) apps were evaluated by 2 independent reviewers using the developed criteria and scores. There was a small negative correlation between the reviewers’ scores and app user rating scores, with higher user rating scores associated with lower reviewer scores.

Conclusions

App developers, publishers, and maternity care professionals should seek advice from women with lived experience of anxiety symptoms in pregnancy to locate, promote, and optimize the visibility of apps for pregnant women. There is a lack of resources that provide coping strategies based on current evidence for the treatment of anxiety in pregnancy. Maternity care providers are limited in their ability to locate and recommend acceptable and trustworthy apps because of the lack of information on the evidence base, development, and testing of apps. Maternity care professionals and women need access to libraries of trusted apps that have been evaluated against relevant and established criteria.

Digital Medication Adherence Support: Could Healthcare Providers Recommend Mobile Health Apps?

Adherence to prescribed medication is suboptimal in 50% of the chronic population, resulting in negative medical and economic outcomes. With the widespread use of mobile phones worldwide, medication adherence apps for mobile phones become promising medication adherence aids thanks to simplicity, user-friendliness, and accessibility for the public. Yet, until today, there is insufficient evidence in favor of using mobile health (mHealth) apps to increase medication adherence. This study aims to develop a methodology for scientific and end-user (patient) mHealth evaluation (a) to identify medication adherence apps search terms, (b) to evaluate identified apps based on scientific criteria, and (c) to report best smartphone apps evaluated by patients. Search terms were identified via literature review and expertise. Firstly, an online questionnaire was developed to identify frequently used search terms by recruited patients. Related medication adherence apps were identified and selected using predefined inclusion criteria. Secondly, identified apps were evaluated thanks to a scientific evaluation method and a created online questionnaire for patient feedback. Recruited patients were invited to test and evaluate the selected apps. Out of 1,833 free-of-charge and 307 paid apps identified, only four free-of-charge and three paid apps remained included in the study after eligibility criteria. None of the selected app reached a high score. Looking at the overall scores, Medisafe (59%), MyTherapy (56%), and Meds on time (44%) received the highest scores in the scientific app evaluation. In the patient evaluation, Dosecast (3.83 out of five points), Medisafe (3.62), and SwissMeds (3.50) received the highest scores. None of the apps in this research has undergone a process for certification, for example, CE marking, through a notified body. Security and data protection aspects of existing apps highly contribute to these low evaluation scores through little information on patient's data processing and storage. This might be corrected through the introduction of General Data Protection Regulation (GDPR) in the European Economic Area (EEA) and more scrutiny through regulatory bodies in the EU/EEA and the USA. None of the applications should be recommended by healthcare providers. In addition, clinical studies with chronic patients are necessary to measure long-term app impacts.

Adapting an evidence-based e-learning CBT program into a mobile app for people experiencing gambling-related problems: A Formative Study

Background

Many people who experience harm and problems from gambling do not seek treatment from gambling treatment services because of personal and resource barriers. Mobile health (mHealth) interventions are widely used across diverse health care areas and populations. However, there are few in the gambling harm field, despite their potential as an additional modality for delivering treatment and support.

Objective

This study aims to understand the needs, preferences, and priorities of people experiencing gambling harms and who are potential end users of a cognitive behavioral therapy mHealth intervention to inform design, features, and functions.

Methods

Drawing on a mixed methods approach, we used creators and domain experts to review the GAMBLINGLESS web-based program and convert it into an mHealth prototype. Each module was reviewed against the original evidence base to maintain its intended fidelity and conceptual integrity. Early wireframes, design ideas (look, feel, and function), and content examples were developed to initiate discussions with end users. Using a cocreation process with a young adult, a Māori, and a Pasifika peoples group, all with experiences of problem or harmful gambling, we undertook 6 focus groups: 2 cycles per group. In each focus group, participants identified preferences, features, and functions for inclusion in the final design and content of the mHealth intervention.

Results

Over 3 months, the GAMBLINGLESS web-based intervention was reviewed and remapped from 4 modules to 6. This revised program is based on the principles underpinning the transtheoretical model, in which it is recognized that some end users will be more ready to change than others. Change is a process that unfolds over time, and a nonlinear progression is common. Different intervention pathways were identified to reflect the end users’ stage of change. In all, 2 cycles of focus groups were then conducted, with 30 unique participants (13 Māori, 9 Pasifika, and 8 young adults) in the first session and 18 participants (7 Māori, 6 Pasifika, and 5 young adults) in the second session. Prototype examples demonstrably reflected the focus group discussions and ideas, and the features, functions, and designs of the Manaaki app were finalized. Attributes such as personalization, cultural relevance, and positive framing were identified as the key. Congruence of the final app attributes with the conceptual frameworks of the original program was also confirmed.

Conclusions

Those who experience gambling harms may not seek help. Developing and demonstrating the effectiveness of new modalities to provide treatment and support are required. mHealth has the potential to deliver interventions directly to the end user. Weaving the underpinning theory and existing evidence of effective treatment with end-user input into the design and development of mHealth interventions does not guarantee success. However, it provides a foundation for framing the intervention’s mechanism, context, and content, and arguably provides a greater chance of demonstrating effectiveness.

Validation of a Korean version of mobile app rating scale (MARS) for apps targeting disease management

The mobile app rating scale (MARS) is a widely used instrument for evaluating smartphone app quality. We aimed to examine the reliability and validity of the Korean version of MARS (MARS-K). Two independent raters performed the assessment using the translated 23-item questionnaire. We applied intraclass correlation coefficient analysis (ICC) to examine inter-rater reliability, Omega, and item-total correlation for internal consistency, and Pearson's r for test-retest reliability and correlation between subscales and the total score of MARS-K. Most items showed moderate to good ICC (0.447-1.000). The MARS-K showed excellent internal consistency and all subscales exceeded the acceptable level of omega. Results indicated MARS-K to be a valid and reliable instrument for evaluating disease management apps offered in the Korean app store. However, upgrades are recommended to further improve MARS-K's rating accuracy and reliability.

A review of the quality and content of mobile apps to support lifestyle modifications following a transient ischaemic attack or 'minor' stroke

Objective

Secondary prevention is recommended to reduce cardiovascular risk after transient ischaemic attack (TIA) or ‘minor’ stroke. Mobile health interventions can provide accessible, cost-effective approaches to address modifiable risk factors, such as physical inactivity, hypertension and being overweight. The objective of this study was to evaluate the quality of apps for supporting lifestyle change following a TIA or ‘minor’ stroke.

Methods

Systematic searches of Google Play and the Apple Store were carried out to identify mobile apps released between 1 November 2019 and 1 October 2021. Keywords were used including stroke, TIA, lifestyle, prevention and recovery. Quality was assessed using the Mobile Application Rating Scale (MARS). Common components were identified with the Behaviour Change Technique (BCT) Taxonomy. Descriptive statistics were used to summarize the performance results for each app.

Results

Searches identified 2545 potential apps. Thirty remained after removing duplicates and screening titles and descriptions. Six were eligible after full review of their content. All apps included at least one BCT (range: 1–16 BCTs). The most frequent BCTs included ‘information about health consequences’ (n = 5/6), ‘verbal or visual communication from a credible source’ (n = 4/6) and ‘action planning’ (n = 4/6). The mean MARS score was 2.57/5 (SD: 0.51; range: 1.78–3.36). No apps were of ‘good’ overall quality (scoring more than 4/5).

Conclusions

This is the first review of mobile health interventions for this population. Only a small number of apps were available. None were targeted specifically at people with a TIA or ‘minor’ stroke. Overall quality was low. Further work is needed to develop and test accessible, user designed, and evidence-informed digital interventions in this population.

Spanish adaptation and validation of the User Version of the Mobile Application Rating Scale (uMARS)

OBJECTIVE

While the professional version of the Mobile App Rating Scale (MARS) has already been translated, and validated into the Spanish language, its user-centered counterpart has not yet been adapted. Furthermore, no other similar tools exist in the Spanish language. The aim of this paper is to adapt and validate User Version of the MARS (uMARS) into the Spanish language.

MATERIALS AND METHODS

Cross-cultural adaptation, translation, and metric evaluation. The internal consistency and test-retest reliability of the Spanish version of the uMARS were evaluated using the RadarCovid app. Two hundred and sixteen participants rated the app using the translated scale. The app was then rated again 2 weeks later by 21 of these participants to measure test-retest reliability.

RESULTS

No major differences were observed between the uMARS original and the Spanish version. Discrimination indices (item-scale correlation) obtained appropriate results for both raters. The Spanish uMARS presented with excellent internal consistency, α = .89 and .67 for objective and subjective quality, respectively, and temporal stability (r > 0.82 for all items and subscales).

DISCUSSION

The Spanish uMARS is a useful tool for health professionals to recommend high-quality mobile apps to their patients based on the user's perspective and for researchers and app developers to use end-user feedback and evaluation, to help them identify highly appraised and valued components, as well as areas for further development, to continue ensuring the increasing quality and prominence of the area of mHealth.

CONCLUSION

uMARS Spanish version is an instrument with adequate metric properties to assess the quality of health apps from the user perspective.

Promoting Health via mHealth Applications Using a French Version of the Mobile App Rating Scale: Adaptation and Validation Study

Background

In the recent decades, the number of apps promoting health behaviors and health-related strategies and interventions has increased alongside the number of smartphone users. Nevertheless, the validity process for measuring and reporting app quality remains unsatisfactory for health professionals and end users and represents a public health concern. The Mobile Application Rating Scale (MARS) is a tool validated and widely used in the scientific literature to evaluate and compare mHealth app functionalities. However, MARS is not adapted to the French culture nor to the language.

Objective

This study aims to translate, adapt, and validate the equivalent French version of MARS (ie, MARS-F).

Methods

The original MARS was first translated to French by two independent bilingual scientists, and their common version was blind back-translated twice by two native English speakers, culminating in a final well-established MARS-F. Its comprehensibility was then evaluated by 6 individuals (3 researchers and 3 nonacademics), and the final MARS-F version was created. Two bilingual raters independently completed the evaluation of 63 apps using MARS and MARS-F. Interrater reliability was assessed using intraclass correlation coefficients. In addition, internal consistency and validity of both scales were assessed. Mokken scale analysis was used to investigate the scalability of both MARS and MARS-F.

Results

MARS-F had a good alignment with the original MARS, with properties comparable between the two scales. The correlation coefficients (r) between the corresponding dimensions of MARS and MARS-F ranged from 0.97 to 0.99. The internal consistencies of the MARS-F dimensions engagement (ω=0.79), functionality (ω=0.79), esthetics (ω=0.78), and information quality (ω=0.61) were acceptable and that for the overall MARS score (ω=0.86) was good. Mokken scale analysis revealed a strong scalability for MARS (Loevinger H=0.37) and a good scalability for MARS-F (H=0.35).

Conclusions

MARS-F is a valid tool, and it would serve as a crucial aid for researchers, health care professionals, public health authorities, and interested third parties, to assess the quality of mHealth apps in French-speaking countries.

What do people want in a smoking cessation app? An analysis of user reviews and app quality

INTRODUCTION

Mobile smoking cessation (mCessation) apps have the potential to complement and enhance existing interventions, but many are of low quality. Exploring app reviews can provide a broader understanding of user experiences and engagement, to enhance the quality, acceptability and effectiveness of future developments.

METHODS

Publicly available user reviews and ratings of smoking cessation apps were mined from Google Play TM and the App Store ® via a targeted two-stage search strategy. English language smoking cessation apps with at least 20 consumer reviews between 2011 and 2020 were included. User reviews were thematically analysed using Braun & Clarke's framework. Apps were independently scored using the Mobile Apps Rating Scale (MARS) and compared to average user star ratings.

RESULTS

Forty-eight versions of 42 apps, encompassing 1,414 associated reviews, met eligibility criteria. Inductive coding of reviews produced 1,084 coding references including reviews coded across multiple nodes. Themes generated included: (1) supportive characteristics/tools; (2) useability; (3) influence on smoking behaviour; (4) benefits of quitting; and (5) role as a supplementary tool for quitting. The mean MARS score of 36 free and accessible apps was 3.10 (SD 0.71) with mean scores ranging from 2.00 to 4.47. An inverse relationship between MARS scores and average user star ratings was observed .

CONCLUSIONS

App personalisation, relationality, functionality and credibility were important to users, and should be considered as key design components for future apps. Differences between user star ratings and MARS scores may illustrate competing priorities of consumers and researchers, and the importance of a co-design development method.

IMPLICATIONS

This is the first study to use unsolicited user reviews from a large population to understand the general mCessation user experience in relation to making a quit attempt. Our findings highlight specific features favoured and disliked by users, including their influence on engagement, and supports previous findings that mCessation applications need to be highly tailorable, functional, credible and supportive. We recommend a consumer-driven, co-design approach for future mCessation app developments to optimise user acceptability and engagement.

Translation and validation of the Italian version of the user version of the Mobile Application Rating Scale (uMARS)

Background

Health sciences are steadily developing apps to help people to adopt correct lifestyles and to help physicians to monitor patients with chronic diseases. However, a properly validated tool that can evaluate patients' perception of apps is still lacking in many languages. In English, a validated questionnaire, called User Version of the Mobile Application Rating Scale (uMARS), is currently available. We translated the uMARS into Italian and validated our version.

Methods

The uMARS questionnaire was translated from English to Italian by an official translator, and then administered to 100 smartphone users in order to evaluate the same app at times 1 and 2 (after 2 weeks). Paired t-test, Pearson Correlation Coefficient, Intraclass Correlation Coefficient (ICCs) and Cronbach's Alpha were used to evaluate the reliability and validity of the Italian uMARS.

Results

We recruited 100 subjects, 52 males (52%) and 48 females (48%), with a mean age of 22.8 (SD: 3.4). All subjects answered all questions both at time 1 and at time 2. Paired t-test showed no statistically significant difference in each answer or group of answers between times 1 and 2 (P > 0.05). Cronbach's alpha was 0.945, as all subjects answered all questions. Each question was further assessed through the Pearson correlation coefficient, which demonstrated high reliability, with significant P (< 0.05) and Pearson Coefficients higher than 0.7. Similarly, ICCs were always higher than 0.750.

Conclusions

Our results validated the Italian version of uMARS, which may become a reliable and useful tool for evaluating health apps.

Evaluating the quality and safety of health-related apps and e-tools: Adapting the Mobile App Rating Scale and developing a quality assurance protocol

BACKGROUND

Whilst apps and e-tools have tremendous potential as low-cost, scalable mental health intervention and prevention tools, it is essential that consumers and health professionals have a means by which to evaluate their quality and safety.

OBJECTIVE

This study aimed to: 1) adapt the original Mobile App Rating Scale (MARS) in order to be appropriate for the evaluation of both mobile phone applications as well as e-tools; 2) test the reliability of the revised scale; and 3) develop a quality assurance protocol for identifying and rating new apps and e-tools to determine appropriateness for use in clinical practice.

METHODS

The MARS was adapted to include items specific to health-related apps and e-tools, such as the availability of resources, strategies for self-management, and quality information. The 41 apps and e-tools in the standard youth configuration of the InnoWell Platform, a digital tool designed to support or enhance mental health service delivery, were independently rated by two expert raters using the A-MARS. Cronbach's alpha was used to calculate the internal consistency and interclass correlation coefficients were used to calculate interrater reliability.

RESULTS

The A-MARS was shown to be a reliable scale with acceptable to excellent internal consistency and moderate to excellent interrater reliability across the subscales. Given the ever-increasing number of health information technologies on the market, a protocol to identify and rate new apps and e-tools for potential clinical use is presented.

CONCLUSIONS

Whilst the A-MARS is a useful tool to guide health professionals as they explore available apps and e-tools for potential clinical use, the training, time, and skill required to use it effectively may be prohibitive. As such, health professionals and services are likely to benefit from including a digital navigator as part of the care team to assist in selecting and rating apps and e-tools, increasing the usability of the data, and technology troubleshooting. When selecting, evaluating and/or recommending apps and e-tools to consumers, it is important to consider: 1) the availability of explicit strategies to set, monitor and review SMART goals; 2) the accessibility of credible, user friendly information and resources from reputable sources; 3) evidence of effectiveness; and 4) interoperability with other health information technologies.

Evaluation of M-Health Applications Use in Epilepsy: A Systematic Review

Background:

Epilepsy is a neurological disorder characterized by seizures and recurrent attacks. Self-management leads to seizure control and maximizes the quality of life in epileptic patients. The purpose of this study was to evaluate the quality of applications available in the epileptic google play store based on the rating features of MARS (Mobile Applications Rating Scale).

Methods:

The search was conducted systematically using the keywords “epilepsy”, “seizure”, “mobile health” at the Android google play store. Data were extracted and analyzed from Feb 2018 to Apr 2019.

Results:

Accordingly, 45 apps were identified potentially relevant of which 20 met inclusion criteria. Twenty-five apps were excluded because they were unrelated to epilepsy self-management, or not Development for people with epilepsy, not in English language or were not free and available. The total mean MARS score was 3.21 out of 5, and more than half of apps (17, 85%) had a minimum acceptability score of 3.0. The mean of apps’ items were 3.27 in Engagement, 3.96 in function, 3.30 in Aesthetics, 2.96 in Information and 2.73 in subjective quality items.

Conclusion:

Few apps meet prespecified criteria for quality, content, and functionality for epilepsy self-management. Despite the rapid evolution of self-management apps, lack of validation studies is a significant concern that limits the clinical value of these apps. Moreover, having a guideline and benchmarking in the field of mobile application development, in epilepsy management, can help analyze the content of established applications.

Interdisciplinary Mobile Health Model to Improve Clinical Care After Heart Transplantation: Implementation Strategy Study

Background

Solid organ transplantation could be the only life-saving treatment for end-stage heart failure. Nevertheless, multimorbidity and polypharmacy remain major problems after heart transplant. A technology-based behavioral intervention model was established to improve clinical practice in a heart transplant outpatient setting. To support the new strategy, the mHeart app, a mobile health (mHealth) tool, was developed for use by patients and providers.

Objective

The primary objective of this study was to describe the implementation of the mHeart model and to outline the main facilitators identified when conceiving an mHealth approach. The secondary objectives were to evaluate the barriers, benefits, and willingness to use mHealth services reported by heart transplant recipients and cardiology providers.

Methods

This was an implementation strategy study directed by a multidisciplinary cardiology team conducted in four stages: design of the model and the software, development of the mHeart tool, interoperability among systems, and quality and security requirements. A mixed methods study design was applied combining a literature review, several surveys, interviews, and focus groups. The approach involved merging engineering and behavioral theory science. Participants were chronic-stage heart transplant recipients, patient associations, health providers, stakeholders, and diverse experts from the legal, data protection, and interoperability fields.

Results

An interdisciplinary and patient-centered process was applied to obtain a comprehensive care model. The heart transplant recipients (N=135) included in the study confirmed they had access to smartphones (132/135, 97.7%) and were willing to use the mHeart system (132/135, 97.7%). Based on stakeholder agreement (>75%, N=26), the major priorities identified of the mHealth approach were to improve therapy management, patient empowerment, and patient-provider interactions. Stakeholder agreement on the barriers to implementing the system was weak (<75%). Establishing the new model posed several challenges to the multidisciplinary team in charge. The main factors that needed to be overcome were ensuring data confidentiality, reducing workload, minimizing the digital divide, and increasing interoperability. Experts from various fields, scientific societies, and patient associations were essential to meet the quality requirements and the model scalability.

Conclusions

The mHeart model will be applicable in distinct clinical and research contexts, and may inspire other cardiology health providers to create innovative ways to deal with therapeutic complexity and multimorbidity through health care systems. Professionals and patients are willing to use such innovative mHealth programs. The facilitators and key strategies described were needed for success in the implementation of the new holistic theory–based mHealth strategy.

Validation of the Mobile Application Rating Scale (MARS)

BACKGROUND

Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity.

OBJECTIVE

This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS.

METHODS

Data was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Non-centrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation.

RESULTS

In total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05).

CONCLUSION

The metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

Mobile Phone Apps for Food Allergies or Intolerances in App Stores: Systematic Search and Quality Assessment Using the Mobile App Rating Scale (MARS)

Background

Food allergies and intolerances are increasing worldwide, and mobile phone apps could be a promising tool for self-management of these issues.

Objective

This study aimed to systemically search and assess food allergy or intolerance apps in app stores using the multidimensional Mobile App Rating Scale (MARS) to rate the objective and subjective quality and to identify critical points for future improvements.

Methods

This systematic search identified apps through the keywords “food allergy,” “food intolerance,” and “allergens” in English, Spanish, and Italian in the Apple App Store (iOS) and Google Play Store (Android). The inclusion criteria were a user star rating of ≥3 (of 5 stars) to limit the selection to the most highly rated apps; ≥1000 reviews as an indicator of reliability; and the most recent update performed up to 2017. Then, the apps were divided according to their purpose (searching for allergen-free “food products,” “restaurants,” or recipes in “meal planners”) and evaluated on a scale of 1 to 5 points using the MARS in terms of (1) app classification category with a descriptive aim; (2) app subjective and objective quality categories comprised of engagement, functionality, esthetics, and information sections (Medline was searched for eligible apps to check whether they had been tested in trials); and (3) an optional app-specific section. Furthermore, the output and input features were evaluated. Differences between MARS sections and between app purposes and correlations among MARS sections, star ratings, and numbers of reviews were evaluated.

Results

Of the 1376 apps identified, 14 were included: 12 related to food allergies and intolerances that detect 2-16 food allergens and 2 related only to gluten intolerance. The mean (SD) MARS scores (maximum 5 points) were 3.8 (SD 0.4) for objective quality, highlighting whether any app had been tested in trials; 3.5 (SD 0.6) for subjective quality; and 3.6 (SD 0.7) for the app-specific section. Therefore, a rating ≥3 points indicated overall acceptable quality. From the between-section comparison, engagement (mean 3.5, SD 0.6) obtained significantly lower scores than functionality (mean 4.1, SD 0.6), esthetics (mean 4, SD 0.5), and information (mean 3.8, SD 0.4). However, when the apps were compared by purpose, critical points were identified: meal planner apps showed significantly higher engagement (mean 4.1, SD 0.4) than food product (mean 3.0, SD 0.6; P=.05) and restaurant (mean 3.2, SD 0.3; P=.02) apps.

Conclusions

In this systematic search of food allergy or intolerance apps, acceptable MARS quality was identified, although the engagement section for food product and restaurant purpose apps should be improved and the included apps should be tested in trials. The critical points identified in this systematic search can help improve the innovativeness and applicability of future food allergy and intolerance apps.

The Design and Rationale of a Pilot Study: A COmmunity and Tech-Based ApproaCh for Hypertension Self-MANagement (COACHMAN)

African-Americans with hypertension continue to demonstrate poor blood pressure (BP) control and have markedly lower rates of hypertension self-management compared to non-African-Americans. Innovative and practical solutions such as mHealth technology are promising and can be leveraged to promote self-management of hypertension. Substantial evidence has demonstrated the importance of community support in improving patients' management of chronic illnesses. Unfortunately, such programs do not offer technology-based interventions (TBI) as a delivery method. Thus, this paper describes the design and rationale of an ongoing pilot study that incorporates TBI using a community-based participatory approach.

The German Version of the Mobile App Rating Scale (MARS-G): Development and Validation Study

BACKGROUND

The number of mobile health apps (MHAs), which are developed to promote healthy behaviors, prevent disease onset, manage and cure diseases, or assist with rehabilitation measures, has exploded. App store star ratings and descriptions usually provide insufficient or even false information about app quality, although they are popular among end users. A rigorous systematic approach to establish and evaluate the quality of MHAs is urgently needed. The Mobile App Rating Scale (MARS) is an assessment tool that facilitates the objective and systematic evaluation of the quality of MHAs. However, a German MARS is currently not available.

OBJECTIVE

The aim of this study was to translate and validate a German version of the MARS (MARS-G).

METHODS

The original 19-item MARS was forward and backward translated twice, and the MARS-G was created. App description items were extended, and 104 MHAs were rated twice by eight independent bilingual researchers, using the MARS-G and MARS. The internal consistency, validity, and reliability of both scales were assessed. Mokken scale analysis was used to investigate the scalability of the overall scores.

RESULTS

The retranslated scale showed excellent alignment with the original MARS. Additionally, the properties of the MARS-G were comparable to those of the original MARS. The internal consistency was good for all subscales (ie, omega ranged from 0.72 to 0.91). The correlation coefficients (r) between the dimensions of the MARS-G and MARS ranged from 0.93 to 0.98. The scalability of the MARS (H=0.50) and MARS-G (H=0.48) were good.

CONCLUSIONS

The MARS-G is a reliable and valid tool for experts and stakeholders to assess the quality of health apps in German-speaking populations. The overall score is a reliable quality indicator. However, further studies are needed to assess the factorial structure of the MARS and MARS-G.

The Arabic Version of the Mobile App Rating Scale: Development and Validation Study

Background

With thousands of health apps in app stores globally, it is crucial to systemically and thoroughly evaluate the quality of these apps due to their potential influence on health decisions and outcomes. The Mobile App Rating Scale (MARS) is the only currently available tool that provides a comprehensive, multidimensional evaluation of app quality, which has been used to compare medical apps from American and European app stores in various areas, available in English, Italian, Spanish, and German. However, this tool is not available in Arabic.

Objective

This study aimed to translate and adapt MARS to Arabic and validate the tool with a sample of health apps aimed at managing or preventing obesity and associated disorders.

Methods

We followed a well-established and defined “universalist” process of cross-cultural adaptation using a mixed methods approach. Early translations of the tool, accompanied by confirmation of the contents by two rounds of separate discussions, were included and culminated in a final version, which was then back-translated into English. Two trained researchers piloted the MARS in Arabic (MARS-Ar) with a sample of 10 weight management apps obtained from Google Play and the App Store. Interrater reliability was established using intraclass correlation coefficients (ICCs). After reliability was ascertained, the two researchers independently evaluated a set of additional 56 apps.

Results

MARS-Ar was highly aligned with the original English version. The ICCs for MARS-Ar (0.836, 95% CI 0.817-0.853) and MARS English (0.838, 95% CI 0.819-0.855) were good. The MARS-Ar subscales were highly correlated with the original counterparts (P<.001). The lowest correlation was observed in the area of usability (r=0.685), followed by aesthetics (r=0.827), information quality (r=0.854), engagement (r=0.894), and total app quality (r=0.897). Subjective quality was also highly correlated (r=0.820).

Conclusions

MARS-Ar is a valid instrument to assess app quality among trained Arabic-speaking users of health and fitness apps. Researchers and public health professionals in the Arab world can use the overall MARS score and its subscales to reliably evaluate the quality of weight management apps. Further research is necessary to test the MARS-Ar on apps addressing various health issues, such as attention or anxiety prevention, or sexual and reproductive health.

A Community and Technology-Based Approach for Hypertension Self-Management (COACHMAN) to Improve Blood Pressure Control in African Americans: Results from a Pilot Study

PURPOSE

Adoption of technology has increased to support self-managing chronic diseases. However, behavioral interventions evaluating such technology have been understudied in African Americans with hypertension. The aim of this study was to explore a community and technology-based intervention for hypertension self-management (COACHMAN) intervention on blood pressure (BP) control and health-related quality of life (HRQoL) in African Americans with hypertension.

METHODS

Sixty African Americans (mean age 60; 75% females) who were prescribed antihypertensive medications and owning a smartphone were randomized to the COACHMAN (n = 30) or enhanced usual care (n = 30) group for 12 weeks. COACHMAN is comprised of four components: web-based education, home BP monitoring, medication management application, and nurse counseling. Hypertension knowledge, self-efficacy, technology adoption/use, medication adherence, BP, and HRQoL scores were assessed.

RESULTS

Mean systolic and diastolic BP at baseline was 150.49 (SD = 13.89) and 86.80 (SD = 13.39), respectively. After completing the 3-month intervention to improve hypertension self-management, the groups did not significantly differ in BP control and HRQoL. Clinically relevant BP reduction was observed in the intervention group. Paired t-test showed that mean medication-taking adherence scores significantly improved in the intervention group (P = 0.023) compared to the control group (P = 0.075).

CONCLUSION

Using technology may have a positive impact on supporting hypertension self-management, particularly in medication-taking adherence. Further research is warranted in a larger sample and should include standardization of medication management to isolate the effects of behavioral interventions on changes in BP.

CLINICALTRIALSGOV IDENTIFIER

NCT03722667.

Criteria for assessing the quality of mHealth apps: a systematic review

Objective

Review the existing studies including an assessment tool/method to assess the quality of mHealth apps; extract their criteria; and provide a classification of the collected criteria.

Methods

In accordance with the PRISMA statement, a literature search was conducted in MEDLINE, EMBase, ISI and Scopus for English language citations published from January 1, 2008 to December 22, 2016 for studies including tools or methods for quality assessment of mHealth apps. Two researchers screened the titles and abstracts of all retrieved citations against the inclusion and exclusion criteria. The full text of relevant papers was then individually examined by the same researchers. A senior researcher resolved eventual disagreements and confirmed the relevance of all included papers. The authors, date of publication, subject fields of target mHealth apps, development method, and assessment criteria were extracted from each paper. The extracted assessment criteria were then reviewed, compared, and classified by an expert panel of two medical informatics specialists and two health information management specialists.

Results

Twenty-three papers were included in the review. Thirty-eight main classes of assessment criteria were identified. These were reorganized by expert panel into 7 main classes (Design, Information/Content, Usability, Functionality, Ethical Issues, Security and Privacy, and User-perceived value) with 37 sub-classes of criteria.

Conclusions

There is a wide heterogeneity in assessment criteria for mHealth apps. It is necessary to define the exact meanings and degree of distinctness of each criterion. This will help to improve the existing tools and may lead to achieve a better comprehensive mHealth app assessment tool.

Spanish adaptation and validation of the Mobile Application Rating Scale questionnaire

INTRODUCTION AND AIM

In recent years, numerous health-related apps have appeared on the market, and assessing their quality has become crucial. A very popular tool worldwide, created especially for this purpose, is the Mobile Application Rating Scale (MARS). However, there are no similar tools in Spanish. Therefore, the aim of this paper is to adapt MARS to the Spanish language and validate the resultant version.

METHOD

The design consists of three processes: cross-cultural adaptation, translation, and metric evaluation. 46 mobile applications, 23 of which were for Apple telephones and 23 of which were for telephones running Android, were included in the study. The main objective of these applications was to promote physical activity. The internal factor structure and reliability of MARS were examined.

RESULTS

No major differences were observed in the two Spanish translations, which were carried out independently. The blind back-translation, reviewed by the original author of MARS, suggested minor edits. Discrimination indices (item-scale correlation) obtained appropriate results for both raters. The reliability of the scores was found to be appropriate both in terms of internal consistency (α > 0.77), temporal stability (r > 0.72), and inter-rater reliability (IC > 0.76). The correlations between the subscales have shown high values with range between 0.47 and 0.83.

CONCLUSION

The Spanish version of MARS was shown to have appropriate metric properties to assess the quality of health apps.

The Characteristics and Quality of Mobile Phone Apps Targeted at Men Who Have Sex With Men in China: A Window of Opportunity for Health Information Dissemination?

BACKGROUND

A number of mobile phone apps has been developed specifically for men who have sex with men (MSM). We will call these mobile phone apps MSM apps for simplicity. At present, the characteristics and quality and purpose of these MSM apps are unclear.

OBJECTIVE

The aim of this study was to objectively and comprehensively evaluate the characteristics and quality of the MSM apps to assess whether they disseminated health information among the MSM in China.

METHODS

We searched 2 dominant mobile phone app stores (Apple Store and Android Market) to obtain MSM apps using the keywords MSM, gay, lesbian, gay, bisexual, and transgender (LGBT), ，，，，, and . Apps were excluded if they did not have a Chinese language interface or if their target population was not MSM. Basic information about the eligible apps for this study, such as app name, app store category, and date of last update was gathered from the specified app stores. The quality of apps was rated by 2 independent raters using Mobile App Rating Scale (MARS). The intraclass correlation coefficient (ICC) between raters was computed as a measure for interrater reliability of the MARS. All analyses were conducted using SPSS version 20.0 (SPSS Inc).

RESULTS

A total of 575 apps were reviewed between September 15, 2018 and September 30, 2018, out of which, 532 apps were excluded. Finally, 43 apps were included. Of the 43 apps, 16 were from the Apple Store, 10 were from Android Market, and 17 were available in both app stores. In addition, 39 out of 43 apps were for social and sexual networking, whereas 10 contained sexual health information, for example, HIV/sexually transmitted diseases knowledge, HIV test, and condom use. The average rating was 4 stars. The number of downloads for 21 apps exceeded 10,000. A total of 31 apps had acceptable quality (as defined by a MARS score of >3), with functionality as the highest scoring domain, followed by information quality, esthetics, and engagement. Interrater reliability was excellent for the overall mean app quality scores (ICC=.946; 95% CI 0.904-0.970) and the subjective quality scores (ICC=.910; 95% CI 0.841-0.950).

CONCLUSIONS

By reviewing the available apps, we found that MSM apps are popular. The majority of MSM apps are for dating, whereas few of them contain HIV prevention and health information. The overall quality of the apps is acceptable. The utilization of mobile phone technologies is a promising way for delivering HIV prevention messages to MSM. We recommend that researchers and app developers should work together to disseminate health information for MSM via mobile technologies.

Assessing the Quality of Mobile Phone Apps for Weight Management: User-Centered Study With Employees From a Lebanese University

Background

Evaluating the quality of mobile health apps for weight loss and weight management is important to understand whether these can be used for obesity prevention and treatment. Recent reviews call for more research on multidimensional aspects of app quality, especially involving end users, as there are already many expert reviews on this domain. However, no quantitative study has investigated how laypersons see popular apps for weight management and perceive different dimensions of app quality.

Objective

This study aimed to explore how laypersons evaluate the quality of 6 free weight management apps (My Diet Coach, SparkPeople, Lark, MyFitnessPal, MyPlate, and My Diet Diary), which achieved the highest quality ratings in a related and recent expert review.

Methods

A user-centered study was conducted with 36 employees of a Lebanese university. Participants enrolled in the study on a rolling basis between October 2016 and March 2017. Participants were randomly assigned an app to use for 2 weeks. App quality was evaluated at the end of the trial period using the Mobile App Rating Scale user version (uMARS). uMARS assesses the dimensions of engagement, functionality, aesthetics, information, and subjective quality on 5-point scales. Internal consistency and interrater agreement were examined. The associations between uMARS scores and users’ demographic characteristics were also explored using nonparametric tests. Analyses were completed in November 2017.

Results

Overall, the 6 apps were of moderately good quality (median uMARS score 3.6, interquartile range [IQR] 0.3). The highest total uMARS scores were achieved by Lark (mean 4.0 [SD 0.5]) and MyPlate (mean 3.8 [SD 0.4]), which also achieved the highest subjective quality scores (Lark: mean 3.3 [SD 1.4]; MyPlate: mean 3.3 [SD 0.8]). Functionality was the domain with the highest rating (median 3.9, IQR 0.3), followed by aesthetics (median 3.7, IQR 0.5), information (median 3.7, IQR 0.1), and engagement (median 3.3, IQR 0.2). Subjective quality was judged low (median 2.5, IQR 0.9). Overall, subjective quality was strongly and positively related (P<.001) with total uMARS score (ρ=.75), engagement (ρ=.68), information, and aesthetics (ρ=.60) but not functionality (ρ=.40; P=.02). Higher engagement scores were reported among healthy (P=.003) and obese individuals (P=.03), who also showed higher total uMARS (P=.04) and subjective quality (P=.05) scores.

Conclusions

Although the apps were considered highly functional, they were relatively weak in engagement and subjective quality scores, indicating a low propensity of using the apps in the future. As engagement was the subdomain most strongly associated with subjective quality, app developers and researchers should focus on creating engaging apps, holding constant the functionality, aesthetics, and information quality. The tested apps (in particular Lark and MyPlate) were perceived as more engaging and of higher quality among healthy, obese individuals, making them a promising mode of delivery for self-directed interventions promoting weight control among the sampled population or in similar and comparable settings.

Consumer Mobile Apps for Potential Drug-Drug Interaction Check: Systematic Review and Content Analysis Using the Mobile App Rating Scale (MARS)

BACKGROUND

General consumers can now easily access drug information and quickly check for potential drug-drug interactions (PDDIs) through mobile health (mHealth) apps. With aging population in Canada, more people have chronic diseases and comorbidities leading to increasing numbers of medications. The use of mHealth apps for checking PDDIs can be helpful in ensuring patient safety and empowerment.

OBJECTIVE

The aim of this study was to review the characteristics and quality of publicly available mHealth apps that check for PDDIs.

METHODS

Apple App Store and Google Play were searched to identify apps with PDDI functionality. The apps' general and feature characteristics were extracted. The Mobile App Rating Scale (MARS) was used to assess the quality.

RESULTS

A total of 23 apps were included for the review-12 from Apple App Store and 11 from Google Play. Only 5 of these were paid apps, with an average price of $7.19 CAD. The mean MARS score was 3.23 out of 5 (interquartile range 1.34). The mean MARS scores for the apps from Google Play and Apple App Store were not statistically different (P=.84). The information dimension was associated with the highest score (3.63), whereas the engagement dimension resulted in the lowest score (2.75). The total number of features per app, average rating, and price were significantly associated with the total MARS score.

CONCLUSIONS

Some apps provided accurate and comprehensive information about potential adverse drug effects from PDDIs. Given the potentially severe consequences of incorrect drug information, there is a need for oversight to eliminate low quality and potentially harmful apps. Because managing PDDIs is complex in the absence of complete information, secondary features such as medication reminder, refill reminder, medication history tracking, and pill identification could help enhance the effectiveness of PDDI apps.

Evaluation of a Mobile Phone-Based Intervention to Increase Parents' Knowledge About the Measles-Mumps-Rubella Vaccination and Their Psychological Empowerment: Mixed-Method Approach

BACKGROUND

There is mixed evidence on the effectiveness of vaccination-related interventions. A major limitation of most intervention studies is that they do not apply randomized controlled trials (RCTs), the method that, over the last 2 decades, has increasingly been considered as the only method to provide proof of the effectiveness of an intervention and, consequently, as the most important instrument in deciding whether to adopt an intervention or not. This study, however, holds that methods other than RCTs also can produce meaningful results.

OBJECTIVE

The aim of this study was to evaluate 2 mobile phone-based interventions aimed at increasing parents' knowledge of the measles-mumps-rubella (MMR) vaccination (through elements of gamification) and their psychological empowerment (through the use of narratives), respectively. The 2 interventions were part of an RCT.

METHODS

We conducted 2 studies with the RCT participants: a Web-based survey aimed at assessing their rating of the tool regarding a number of qualities such as usability and usefulness (N=140), and qualitative telephonic interviews to explore participants' experiences with the app (N=60).

RESULTS

The results of the survey showed that participants receiving the knowledge intervention (alone or together with the empowerment intervention) liked the app significantly better compared with the group that only received the empowerment intervention (F_2,137=15.335; P<.001). Parents who were exposed to the empowerment intervention complained that they did not receive useful information but were only invited to make an informed, autonomous MMR vaccination decision.

CONCLUSIONS

The results suggest that efforts to empower patients should always be accompanied by the provision of factual information. Using a narrative format that promotes parents' identification can be an appropriate strategy, but it should be employed together with the presentation of more points of views and notions regarding, for instance, the risks and benefits of the vaccination at the same time.

TRIAL REGISTRATION

International Standard Randomized Controlled Trial Number 30768813; http://www.isrctn.com/ ISRCTN30768813 (Archived by WebCite at http://www.webcitation.org/6xOQSJ3w8).

Current status of postnatal depression smartphone applications available on application stores: an information quality analysis

OBJECTIVES

It is the aim of the current research to identify some common functionalities of postnatal application, and to determine the quality of the information content of postnatal depression application using validated scales that have been applied for applications in other specialties.

SETTINGS AND PARTICIPANTS

To determine the information quality of the postnatal depression smartphone applications, the two most widely used smartphone application stores, namely Apple iTunes as well as Google Android Play store, were searched between 20May and 31 May. No participants were involved. The inclusion criteria for the application were that it must have been searchable using the keywords 'postnatal', 'pregnancy', 'perinatal', 'postpartum' and 'depression', and must be in English language.

INTERVENTION

The Silberg Scale was used in the assessment of the information quality of the smartphone applications.

PRIMARY AND SECONDARY OUTCOMES MEASURE

The information quality score was the primary outcome measure.

RESULTS

Our current results highlighted that while there is currently a myriad of applications, only 14 applications are specifically focused on postnatal depression. In addition, the majority of the currently available applications on the store have only disclosed their last date of modification as well as ownership. There remain very limited disclosures about the information of the authors, as well as the references for the information included in the application itself. The average score for the Silberg Scale for the postnatal applications we have analysed is 3.0.

CONCLUSIONS

There remains a need for healthcare professionals and developers to jointly conceptualise new applications with better information quality and evidence base.

Effectiveness of a smartphone app to increase parents' knowledge and empowerment in the MMR vaccination decision: A randomized controlled trial

Researchers are trying to build evidence for mhealth effectiveness in various fields. However, no evidence yet is showing the effectiveness of mhealth on parents' attitudes and behavior with regard to recommended vaccination of their children. The aim of this study was to look into the effects of 2 smartphone-based interventions targeting MMR vaccination knowledge and psychological empowerment respectively. The interventions used gamification features and videos in combination with text messages. We conducted a 2x2 between-subject factorial randomized controlled trial (absence/presence of knowledge intervention X absence/presence of empowerment intervention) with parents of young children in Italy. We randomly allocated 201 eligible participants to one of the 4 conditions. Data were collected by questionnaires at baseline and posttest. Primary outcomes were MMR vaccination knowledge, psychological empowerment, risk perception, and preferred decisional role; secondary outcomes included MMR vaccination intention, attitude, confidence, and recommendation intention. A significant gain in vaccination knowledge was reported by all experimental groups compared with the control (F(3,179) = 48.58, p < .000), while only those receiving both interventions reported a significant increase in their psychological empowerment (t(179) = -2.79, p = .006). Participants receiving the intervention targeting knowledge reported significantly higher intention to vaccinate (t(179) = 2.111; p = .03) and higher confidence in the decision (t(179) = 2.76; p = .006) compared with the control group. Parent-centered, gamified mobile interventions aimed at providing parents with vaccination-related information can be used to increase their knowledge, their intention to vaccinate as well as their confidence in the vaccination decision.

An eHealth Project on Invasive Pneumococcal Disease: Comprehensive Evaluation of a Promotional Campaign

BACKGROUND

The recently launched Pneumo Rischio eHealth project, which consists of an app, a website, and social networking activity, is aimed at increasing public awareness of invasive pneumococcal disease (IPD). The launch of this project was prompted by the inadequate awareness of IPD among both laypeople and health care workers, the heavy socioeconomic burden of IPD, and the far from optimal vaccination coverage in Italy, despite the availability of safe and effective vaccines.

OBJECTIVE

The objectives of our study were to analyze trends in Pneumo Rischio usage before and after a promotional campaign, to characterize its end users, and to assess its user-rated quality.

METHODS

At 7 months after launching Pneumo Rischio, we established a 4-month marketing campaign to promote the project. This intervention used various approaches and channels, including both traditional and digital marketing strategies. To highlight usage trends, we used different techniques of time series analysis and modeling, including a modified Mann-Kendall test, change-point detection, and segmented negative binomial regression of interrupted time series. Users were characterized in terms of demographics and IPD risk categories. Customer-rated quality was evaluated by means of a standardized tool in a sample of app users.

RESULTS

Over 1 year, the app was accessed by 9295 users and the website was accessed by 143,993 users, while the project's Facebook page had 1216 fans. The promotional intervention was highly effective in increasing the daily number of users. In particular, the Mann-Kendall trend test revealed a significant (P ≤.01) increasing trend in both app and website users, while change-point detection analysis showed that the first significant change corresponded to the start of the promotional campaign. Regression analysis showed a significant immediate effect of the intervention, with a mean increase in daily numbers of users of 1562% (95% CI 456%-4870%) for the app and 620% (95% CI 176%-1777%) for the website. Similarly, the postintervention daily trend in the number of users was positive, with a relative increase of 0.9% (95% CI 0.0%-1.8%) for the app and 1.4% (95% CI 0.7%-2.1%) for the website. Demographics differed between app and website users and Facebook fans. A total of 69.15% (10,793/15,608) of users could be defined as being at risk of IPD, while 4729 users expressed intentions to ask their doctor for further information on IPD. The mean app quality score assigned by end users was approximately 79.5% (397/500).

CONCLUSIONS

Despite its specific topic, Pneumo Rischio was accessed by a considerable number of users, who ranked it as a high-quality project. In order to reach their target populations, however, such projects should be promoted.