The most used questionnaires for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health

Successful Electronic Consultation Service Initiative in Quebec, Canada With Primary Care Physicians' and Specialists' Experiences on Acceptance and Use of Technological Innovation: Cross-Sectional Exploratory Study

BACKGROUND

Electronic consultation (eConsult) is an eHealth service that allows primary care providers (PCPs) to electronically consult specialists regarding their patients' medical issues. Many studies have demonstrated that eConsult services improve timely access to specialist care; prevent unnecessary referrals; improve PCPs', specialists', and patients' satisfaction; and therefore have a large impact on costs. However, no studies have evaluated PCPs' and specialists' acceptance of eConsult services in Quebec, Canada, and worldwide.

OBJECTIVE

This exploratory study aims to identify factors affecting eConsult service acceptance by PCPs and specialists in urban and rural primary care clinics across 3 regions in the province of Quebec, Canada, by integrating the Unified Theory of Acceptance and Use of Technology and Task-Technology Fit (TTF) models and user satisfaction. This research was designed to broaden and assist in scaling up this effective eHealth service innovation across the province.

METHODS

A cross-sectional web-based survey was sent to all PCPs (n=263) and specialists (n=62) who used the eConsult Quebec Service between July 2017 and May 2021. We proposed a unified model integrating the Unified Theory of Acceptance and Use of Technology model and TTF model and user satisfaction by endorsing 11 hypotheses. The partial least squares was used to investigate factors influencing the acceptance of the eConsult Quebec Service.

RESULTS

Of the 325 end users, 136 (41.8%) users responded (PCPs: 101/263, 38.4%; specialists: 35/62, 57%). The results of the analysis with partial least squares method indicate that 9 of our 11 hypotheses are supported. The direct relationships uniting the various constructs of the model highlighted the importance of several key constructs and predominant correlations. The results suggest that satisfaction is the key driver behind the use of the eConsult Quebec Service. Performance expectancy (P<.001) and effort expectancy (P=.03) can have a positive impact on behavioral intention (BI), and BI (P<.001) can impact adoption. TTF has an influence on performance expectancy (P<.001), adoption (P=.02), and satisfaction (P<.001). However, the results show that there is no direct effect between social influence (P=.38) and BI or between facilitating conditions (P=.17) and adoption.

CONCLUSIONS

This study provides a better understanding of the factors influencing PCPs' and specialists' intention to adopt the eConsult Quebec Service. Furthermore, this study tests a research model and a technology that have never been explored in Quebec until now. On the basis of the results, the service is a good fit to meet the users' need to improve access to specialized medical advice. Therefore, the results of our study have made a valuable contribution to the implementation of the service by policy makers in order to maximize acceptance, use, adoption, and success across the province of Quebec. Moreover, after 4 successful years, the eConsult Quebec pilot project is now the Conseil Numérique digital consultation service.

Quality assessment of mHealth apps: a scoping review

Introduction

The number of mHealth apps has increased rapidly during recent years. Literature suggests a number of problems and barriers to the adoption of mHealth apps, including issues such as validity, usability, as well as data privacy and security. Continuous quality assessment and assurance systems might help to overcome these barriers. Aim of this scoping review was to collate literature on quality assessment tools and quality assurance systems for mHealth apps, compile the components of the tools, and derive overarching quality dimensions, which are potentially relevant for the continuous quality assessment of mHealth apps.

Methods

Literature searches were performed in Medline, EMBASE and PsycInfo. Articles in English or German language were included if they contained information on development, application, or validation of generic concepts of quality assessment or quality assurance of mHealth apps. Screening and extraction were carried out by two researchers independently. Identified quality criteria and aspects were extracted and clustered into quality dimensions.

Results

A total of 70 publications met inclusion criteria. Included publications contain information on five quality assurance systems and further 24 quality assessment tools for mHealth apps. Of these 29 systems/tools, 8 were developed for the assessment of mHealth apps for specific diseases, 16 for assessing mHealth apps for all fields of health and another five are not restricted to health apps. Identified quality criteria and aspects were extracted and grouped into a total of 14 quality dimensions, namely "information and transparency", "validity and (added) value", "(medical) safety", "interoperability and compatibility", "actuality", "engagement", "data privacy and data security", "usability and design", "technology", "organizational aspects", "social aspects", "legal aspects", "equity and equality", and "cost(-effectiveness)".

Discussion

This scoping review provides a broad overview of existing quality assessment and assurance systems. Many of the tools included cover only a few dimensions and aspects and therefore do not allow for a comprehensive quality assessment or quality assurance. Our findings can contribute to the development of continuous quality assessment and assurance systems for mHealth apps.

Systematic Review Registration

https://www.researchprotocols.org/2022/7/e36974/, International Registered Report Identifier, IRRID (DERR1-10.2196/36974).

Bounded rationality in healthcare: unraveling the psychological factors behind patient satisfaction in China

Introduction

Patient satisfaction is a crucial metric to gauge the quality of medical services, but the psychological factors influencing patient satisfaction remain insufficiently explored.

Methods

This study examines these psychological factors by applying the theory of bounded rationality to 1,442 inpatients in Hangzhou, China, whose data were collected using a questionnaire. One-way ANOVA, correlation analysis, and hierarchical regression were used to analyze patient satisfaction and its associated factors. Additionally, the path analysis of the structural equation model revealed the mechanisms behind the key psychological factors that influenced patient satisfaction.

Results

Medical risk perception, the social cognition of the medical environment, and social desirability bias had significant positive impacts on patient satisfaction. By contrast, negative emotions had a significant negative impact on patient satisfaction. Notably, patients' negative emotions had both a suppressive effect and a positive moderating effect on the relationship between medical risk perception and patient satisfaction. Similarly, social desirability bias had a suppressive effect on the correlation between the social cognition of the medical environment and patient satisfaction, albeit with a negative moderating effect.

Discussion

These results suggest that when evaluating and improving patient satisfaction, accounting only for the factors that directly influence medical service quality is insufficient, as the indirect and moderating effects of patients' negative emotions and the social cognition of the medical environment must also be considered. Medical service providers should thus address patients' negative emotions, establish good doctor-patient relationships, optimize service environments, provide managers with medical risk education and training on negative emotions, and prioritize patient-centered care. Additionally, the government and relevant health departments should optimize medical policies, enhance fairness and accessibility, and create a positive social cognitive environment through public education and awareness campaigns.

Telehealth-Delivered Program and Accompanying Patients to Enhance the Clinical Condition of Patients Throughout a Liver Transplant: Protocol for a Mixed Methods Study

BACKGROUND

Liver transplantation (LT) is indicated in patients with severe acute or chronic liver failure for which no other therapy is available. With the increasing number of LTs in recent years, liver centers worldwide must manage their patients according to their clinical situation and the expected waiting time for transplantation. The LT clinic at the Centre hospitalier de l'Université de Montréal (CHUM) is developing a new health care model across the entire continuum of pre-, peri-, and posttransplant care that features patient monitoring by an interdisciplinary team, including an accompanying patient; a digital platform to host a clinical plan; a learning program; and data collection from connected objects.

OBJECTIVE

This study aims to (1) evaluate the outcomes following the implementation of a patient platform with connected devices and an accompanying patient, (2) identify implementation barriers and facilitators, (3) describe service outcomes in terms of health outcomes and the rates and nature of contact with the accompanying patient, (4) describe patient outcomes, and (5) assess the intervention's cost-effectiveness.

METHODS

Six types of participants will be included in the study: (1) patients who received transplants and reached 1 year after transplantation before September 2023 (historical cohort or control group), (2) patients who will receive an LT between December 2023 and November 2024 (prospective cohort/intervention group), (3) relatives of those patients, (4) accompanying patients who have received an LT and are interested in supporting patients who will receive an LT, (5) health care professionals, and (6) decision makers. To describe the study sample and collect data to achieve all the objectives, a series of validated questionnaires, accompanying patient logbooks, transcripts of interviews and focus groups, and clinical indicators will be collected throughout the study.

RESULTS

In total, 5 (steering, education, clinical-technological, nurse prescription, and accompanying patient) working committees have been established for the study. Recruitment of patients is expected to start in November 2023. All questionnaires and technological platforms have been prepared, and the clinicians, stakeholders, and accompanying patient personnel have been recruited.

CONCLUSIONS

The implementation of this model in the trajectory of LT recipients at the CHUM may allow for better monitoring and health of patients undergoing transplantation, ultimately reducing the average length of hospital stay and promoting better use of medical resources. In the event of positive results, this model could be transposed to all transplant units at the CHUM and across Quebec (potentially affecting 888 patients per year) but could also be applied more widely to the monitoring of patients with other chronic diseases. The lessons learned from this project will be shared with decision makers and will serve as a model for other initiatives involving accompanying patients, connected objects, or digital platforms.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54440.

Usability and Acceptability of a Conversational Agent Health Education App (Nthabi) for Young Women in Lesotho: Quantitative Study

BACKGROUND

Young women in Lesotho face myriad sexual and reproductive health problems. There is little time to provide health education to women in low-resource settings with critical shortages of human resources for health.

OBJECTIVE

This study aims to determine the acceptability and usability of a conversational agent system, the Nthabi health promotion app, which was culturally adapted for use in Lesotho.

METHODS

We conducted a descriptive quantitative study, using a 22-item Likert scale survey to assess the perceptions of the usability and acceptability of 172 young women aged 18-28 years in rural districts of Lesotho, who used the system on either smartphones or tablets for up to 6 weeks. Descriptive statistics were used to calculate the averages and frequencies of the variables. χ2 tests were used to determine any associations among variables.

RESULTS

A total of 138 participants were enrolled and completed the survey. The mean age was 22 years, most were unmarried, 56 (40.6%) participants had completed high school, 39 (28.3%) participants were unemployed, and 88 (63.8%) participants were students. Respondents believed the app was helpful, with 134 (97.1%) participants strongly agreeing or agreeing that the app was "effective in helping them make decisions" and "could quickly improve health education and counselling." In addition, 136 (98.5%) participants strongly agreed or agreed that the app was "simple to use," 130 (94.2 %) participants reported that Nthabi could "easily repeat words that were not well understood," and 128 (92.7%) participants reported that the app "could quickly load the information on the screen." Respondents were generally satisfied with the app, with 132 (95.6%) participants strongly agreeing or agreeing that the health education content delivered by the app was "well organised and delivered in a timely way," while 133 (96.4%) participants "enjoyed using the interface." They were satisfied with the cultural adaptation, with 133 (96.4%) participants strongly agreeing or agreeing that the app was "culturally appropriate and that it could be easily shared with a family or community members." They also reported that Nthabi was worthwhile, with 127 (92%) participants reporting that they strongly agreed or agreed that they were "satisfied with the application and intended to continue using it," while 135 (97.8%) participants would "encourage others to use it." Participants aged 18-24 years (vs those aged 25-28 years) agreed that the "Nthabi app was simple to use" (106/106, 100% vs 30/32, 98.8%; P=.01), and agreed that "the educational content was well organised and delivered in a timely way" (104/106, 98.1% vs 28/32, 87.5%; P=.01).

CONCLUSIONS

These results support further study of conversational agent systems as alternatives to traditional face-to-face provision of health education services in Lesotho, where there are critical shortages of human resources for health.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04354168; https://www.clinicaltrials.gov/study/NCT04354168.

A Comparison of Patients' and Neurologists' Assessments of their Teleneurology Encounter: A Cross-Sectional Analysis

Background and Objectives: To better understand patients' and neurologists' assessments of their experiences regarding effectiveness of teleneurology encounters. Methods: Following an audio-video telehealth visit, neurologists asked patients to participate in a survey-based research study about the encounter, and then, the neurologists also recorded their own evaluations. Data were analyzed using standard quantitative and qualitative techniques for dichotomous and ordered-category survey responses in this cross-sectional analysis. Results: The study included unique encounters between 187 patients and 11 general neurologists. The mean patient age was 49 ± 17.5 years. Two thirds of the patients (66.8%, 125/187) were female. One third (33.2%; 62) were patients new to the NYU Langone Health neurology practices. The most common patient chief complaints were headache (69/187, 36.9%), focal and generalized numbness or tingling (21, 11.2%), memory difficulty (15, 8%), spine-related symptoms (12, 6.4%), and vertigo (11, 5.9%). Most patients (94.7%, 177/187) reported that the teleneurology encounter satisfied their needs. Patients and their neurologists agreed that the experience was effective in 91% (162/178) of encounters, regardless of whether the visit was for a new or established patient visit. Discussion: More than 90% of new and established patients and their neurologists agreed that teleneurology encounters were effective despite some limitations of the examination, the occasional need for patient assistance, and technical difficulties. Our results provide further evidence to justify and to expand the clinical use of teleneurology.

"Internet+Nursing Service" Mobile Apps in China App Stores: Functionality and Quality Assessment Study

Background

As the Chinese society ages and the concern for health and quality of life grows, the demand for care services in China is increasing. The widespread use of internet technology has greatly improved the convenience and efficiency of web-based services. As a result, the Chinese government has been implementing "Internet+Nursing Services" since 2019, with mobile apps being the primary tools for users to access these services. The quality of these apps is closely related to user experience and the smooth use of services.

Objective

This study aims to evaluate the functionality, services, and quality of "Internet+Nursing Service" apps; identify weaknesses; and provide suggestions for improving service programs and the research, development, improvement, and maintenance of similar apps.

Methods

In December 2022, two researchers searched for "Internet+Nursing Service" apps by applying the search criteria on the Kuchuan mobile app monitoring platform. After identifying the apps to be included based on ranking criteria, they collected information such as the app developer, app size, version number, number of downloads, user ratings, and number and names of services. Afterward, 5 trained researchers independently evaluated the quality of the apps by using the Chinese version of the user version of the Mobile App Rating Scale (uMARS-C). The total uMARS-C score was based on the average of the five evaluators' ratings.

Results

A total of 17 "Internet+Nursing Service" apps were included. Among these, 12 (71%) had been downloaded more than 10,000 times, 11 (65%) had user ratings of 4 or higher, the median app size was 62.67 (range 22.71-103; IQR 37.51-73.47) MB, 16 (94%) apps provided surgical wound dressing change services, 4 (24%) covered first-tier cities, and only 1 (6%) covered fourth-tier cities. The median total uMARS-C score was 3.88 (range 1.92-4.92; IQR 3.71-4.05), which did not correlate with app store user ratings (r=0.003; P=.99). The quality of most apps (11/17, 65%) was average. Most apps (12/17, 71%) were rated as "good" or above (≥4 points) in terms of information quality, layout, graphics, performance, and ease of use; however, the vast majority of apps were rated as "fair" or even "poor" (<4 points) in terms of credibility (14/17, 82%) and demand (16/17, 94%).

Conclusions

"Internet+Nursing Service" apps need to broaden their service coverage, increase service variety, and further optimize their service structure. The overall quality of these apps is generally poor. App developers should collaborate with medical professionals and communicate with target users before launching their products to ensure accurate content, complete functionality, and good operation that meets user needs.

Factors associated with usability of the EMPOWER-SUSTAIN Self-management mobile app© among individuals with cardiovascular risk factors in primary care

Objective

This study aimed to determine the usability of the EMPOWER-SUSTAIN Self-Management Mobile App© and evaluate the factors associated with its usability among patients with cardiovascular risk factors in primary care.

Methodology

This was a cross-sectional study, conducted among patients aged ≥ 18 years with cardiovascular risk factors attending a university primary care clinic. Patients were given the app to use for at least three months. Those who fulfilled the eligibility criteria were recruited. Data gathered were on sociodemographic, clinical characteristics, self-management support by doctors, utilisation of the app at home and social support in using the app. The previously translated and validated Malay version of the mHealth App Usability Questionnaire was used to measure usability. The mean usability score was calculated and linear regressions analysis was conducted to determine the factors associated with the usability of the app.

Results

A total of 247 patients with at least one cardiovascular risk factor(s) were recruited. The mean age was 60.2 (±8.2). The majority were Malays (86.2%) and half of them were males (52.2%). The total mean (±SD) usability score was 5.26 (±0.67) indicating a high usability of the app. Usability of the app declined with increasing age in the simple linear regressions analysis. The multiple linear regressions yielded that being Malay (b = 0.31, 95% CI 0.08,0.54), using the app at home to understand their medications (b = 0.33, 95% CI 0.12,0.53) and having social support from family members and friends (b = 0.28, 95% CI 0.07,0.49) were significantly associated with higher usability of the app.

Conclusion

The usability of the EMPOWER-SUSTAIN Self-Management Mobile App© was high among patients with cardiovascular risk factors in our primary care clinic. This finding supports the widespread use of this app among our patients. Involvement of family members and friends should be encouraged to improve the usability of the app.

The most used questionnaires for evaluating the usability of robots and smart wearables: A scoping review

Background

As the field of robotics and smart wearables continues to advance rapidly, the evaluation of their usability becomes paramount. Researchers may encounter difficulty in finding a suitable questionnaire for evaluating the usability of robotics and smart wearables. Therefore, the aim of this study is to identify the most commonly utilized questionnaires for assessing the usability of robots and smart wearables.

Methods

A comprehensive search of databases, including PubMed, Web of Science, and Scopus, was conducted for this scoping review. Two authors performed the selection of articles and data extraction using a 10-field data extraction form. In cases of disagreements, a third author was consulted to reach a consensus. The inclusions were English-language original research articles that utilized validated questionnaires to assess the usability of healthcare robots and smart wearables. The exclusions comprised review articles, non-English publications, studies not focused on usability, those assessing clinical outcomes, articles lacking questionnaire details, and those using non-validated or researcher-made questionnaires. Descriptive statistics methods (frequency and percentage), were employed to analyze the data.

Results

A total of 314 articles were obtained, and after eliminating irrelevant and duplicate articles, a final selection of 50 articles was included in this review. A total of 17 questionnaires were identified to evaluate the usability of robots and smart wearables, with 10 questionnaires specifically for wearables and 7 questionnaires for robots. The System Usability Scale (50%) and Post-Study System Usability Questionnaire (19.44%) were the predominant questionnaires utilized to assess the usability of smart wearables. Moreover, the most commonly used questionnaires for evaluating the usability of robots were the System Usability Scale (56.66%), User Experience Questionnaire (16.66%), and Quebec User Evaluation of Satisfaction with Assistive Technology (10%).

Conclusion

Commonly employed questionnaires serve as valuable tools in assessing the usability of robots and smart wearables, aiding in the refinement and optimization of these technologies for enhanced user experiences. By incorporating user feedback and insights, designers can strive towards creating more intuitive and effective robotic and wearable solutions.

The German version of the mHealth App Usability Questionnaire (GER-MAUQ): Translation and validation study in patients with cardiovascular disease

Objective

In Germany, only a few standardized evaluation tools for assessing the usability of mobile Health apps exist so far. This study aimed to translate and validate the English patient version for standalone apps of the mHealth App Usability Questionnaire (MAUQ) into a German version.

Methods

Following scientific guidelines for translation and cross-cultural adaptation, the patient version for standalone apps was forward and back-translated from English into German by an expert panel. In total, 53 participants who were recruited as part of the beta testing process of the recently developed mHealth app HerzFit, answered the questions of the German version of the MAUQ (GER-MAUQ) and the System Usability Scale. Subsequently, a descriptive as well as a psychometric analysis was performed to test validity and reliability.

Results

After conducting three cognitive interviews, five items were modified. The values for Cronbach alpha for the entire questionnaire and the three subscales (0.966, 0.814, 0.910, and 0.909) indicate strong internal consistency. The correlation analysis revealed that the scores of the GER-MAUQ, the subscales and the SUS were strongly correlated with each other. The correlation coefficient of the SUS and the GER-MAUQ overall score was r = 0.854, P < 0.001 and the coefficients of the subscales and the SUS were r = 0.642, P < 0.001; r = 0.866, P < 0.001 and r = 0.643, P < 0.001.

Conclusions

We have developed a novel German version of the MAUQ and demonstrated it as a reliable and valid measurement tool for assessing the usability of standalone mHealth apps from the patients' perspective. The GER-MAUQ allows a new form of standardized assessment of usability of mHealth apps for patients with cardiovascular disease in Germany. Further research with a larger sample and other samples is recommended.

Refinement and Usability Analysis of an eHealth App for Ankylosing Spondylitis as a Complementary Treatment to Physical Therapy: Development and Usability Study

BACKGROUND

Mobile eHealth apps have been used as a complementary treatment to increase the quality of life of patients and provide new opportunities for the management of rheumatic diseases. Telemedicine, particularly in the areas of prevention, diagnostics, and therapy, has become an essential cornerstone in the care of patients with rheumatic diseases.

OBJECTIVE

This study aims to improve the design and technology of YogiTherapy and evaluate its usability and quality.

METHODS

We newly implemented the mobile eHealth app YogiTherapy with a modern design, the option to change language, and easy navigation to improve the app's usability and quality for patients. After refinement, we evaluated the app by conducting a study with 16 patients with AS (4 female and 12 male; mean age 48.1, SD 16.8 y). We assessed the usability of YogiTherapy with a task performance test (TPT) with a think-aloud protocol and the quality with the German version of the Mobile App Rating Scale (MARS).

RESULTS

In the TPT, the participants had to solve 6 tasks that should be performed on the app. The overall task completion rate in the TPT was high (84/96, 88% completed tasks). Filtering for videos and navigating to perform an assessment test caused the largest issues during the TPT, while registering in the app and watching a yoga video were highly intuitive. Additionally, 12 (75%) of the 16 participants completed the German version of MARS. The quality of YogiTherapy was rated with an average MARS score of 3.79 (SD 0.51) from a maximum score of 5. Furthermore, results from the MARS questionnaire demonstrated a positive evaluation regarding functionality and aesthetics.

CONCLUSIONS

The refined and tested YogiTherapy app showed promising results among most participants. In the future, the app could serve its function as a complementary treatment for patients with AS. For this purpose, surveys with a larger number of patients should still be conducted. As a substantial advancement, we made the app free and openly available on the iOS App and Google Play stores.

Institute for Clinical and Economic Review - Peterson Health Technology Institute value assessment framework for digital health technologies

Digital health technologies (DHTs) are a broad and rapidly innovating class of interventions with distinctive pathways for development, regulatory approval, uptake and reimbursement. Given the unique nature of DHTs, existing value assessment frameworks and evidence standards for health technologies such as drugs and devices are not directly applicable. The value assessment framework presented here describes a conceptual model and associated methods to guide assessments of DHTs. The framework seeks to accomplish two goals: to set evidence standards that guide technology developers to generate robust evidence on their products; and to provide reviews that help organizations adopt high-impact DHTs with the strongest evidence for delivering improved clinical outcomes and cost savings. This assessment framework will serve as the roadmap for future evaluations of DHTs by the Institute for Clinical and Economic Review (ICER) and the Peterson Health Technology Institute (PHTI). We believe that all stakeholders will benefit from comprehensive and explicit standards of evidence on the different dimensions necessary to understand the value of DHTs.

The impact of eHealth on relationships and trust in primary care: a review of reviews

BACKGROUND

Given the increasing integration of digital health technologies in team-based primary care, this review aimed at understanding the impact of eHealth on patient-provider and provider-provider relationships.

METHODS

A review of reviews was conducted on three databases to identify papers published in English from 2008 onwards. The impact of different types of eHealth on relationships and trust and the factors influencing the impact were thematically analyzed.

RESULTS

A total of 79 reviews were included. Patient-provider relationships were discussed more frequently as compared to provider-provider relationships. Communication systems like telemedicine were the most discussed type of technology. eHealth was found to have both positive and negative impacts on relationships and/or trust. This impact was influenced by a range of patient-related, provider-related, technology-related, and organizational factors, such as patient sociodemographics, provider communication skills, technology design, and organizational technology implementation, respectively.

CONCLUSIONS

Recommendations are provided for effective and equitable technology selection, application, and training to optimize the impact of eHealth on relationships and trust. The review findings can inform providers' and policymakers' decision-making around the use of eHealth in primary care delivery to facilitate relationship-building.

Evaluation of a telehealth program for high-risk pregnancy in a health service provider institution

INTRODUCTION

eHealth offers a solution to current challenges in maternal health. Telemonitoring can rethink antenatal care, achieving more personalized medicine, especially in high-obstetric risk conditions. A home care model mediated by the development of a mobile and web application that seeks support in clinical decision-making was created in our institution. The software architecture consists of an information system (HCIS, Health Care Information System), a database, an authentication server, and an interoperability bus.

METHODS

The usability of the software by patients was assessed using the Information Systems Usability Questionnaire (CSUQ) version 3, as well as the satisfaction of the care model designed through a survey in a prospective observational study in 62 patients with a high-risk pregnancy. Participants were monitored in real-time through blood glucose, blood pressure, and weight measurements, and attended telemedicine appointments with an obstetrician from admission to six weeks postpartum.

RESULTS

The CSUQ results show a good acceptance of the use of the application by patients both in the global score and the different domains, Global, System utility (SYSUSE), Information Quality (INFOQUAL), and Interface quality (INTERQUAL). The satisfaction survey also shows good results in general and by domains.

CONCLUSION

The findings of this study provide some suggestions for the implementation of a technological development and a care model from the patient's perspective.

German Version of the mHealth App Usability Questionnaire in a Cohort of Patients With Cancer: Translation and Validation Study

BACKGROUND

Good usability is important for the adoption and continued use of mobile health (mHealth) apps. In particular, high usability can support intuitive use by patients, which improves compliance and increases the app's effectiveness. However, many usability studies do not use adequate tools to measure perceived usability. The mHealth App Usability Questionnaire (MAUQ) was developed specifically for end users in a medical context. MAUQ is a relatively new but increasingly used questionnaire to evaluate mHealth apps, but it is not yet available in German.

OBJECTIVE

This study aims to translate MAUQ into German and determine its internal consistency, reliability, and construct validity.

METHODS

This validation study was conducted as part of a usability evaluation project for an mHealth app used as a therapy support tool during breast cancer chemotherapy. MAUQ was translated into German through a rigorous forward-backward translation process, ensuring semantic and conceptual equivalence. Patient responses to MAUQ and System Usability Scale (SUS) were analyzed for validation. Descriptive analysis was performed for the MAUQ subscales and SUS standard scores. Significance tests and correlation coefficients assessed the relationship between the SUS and MAUQ results, confirming construct validity. Internal consistency was assessed for item reliability and consistency in measuring the target construct. Free-text questions assessed translation comprehensibility, with responses analyzed descriptively and qualitatively using content analysis.

RESULTS

In this study, 133 participants responded to the questionnaire, and the validation analysis showed substantially positive correlations between the overall MAUQ score and its subscales: ease of use (r=0.56), interface and satisfaction (r=0.75), and usefulness (r=0.83). These findings support the construct validity of MAUQ and emphasize the importance of these subscales in assessing the usability of the Enable app. The correlation coefficients ranging from 0.39 to 0.68 for the items further validate the questionnaire by aligning with the overall score and capturing the intended concept. The high internal consistency reliability of MAUQ (Cronbach α=.81) and its subscales further enhances the instrument's robustness in accurately evaluating the usability of mHealth apps.

CONCLUSIONS

We successfully validated the German translation of the MAUQ for stand-alone apps using a standardized approach in a cohort of patients with breast cancer. In our validation study, MAUQ exhibited strong internal consistency reliability (Cronbach α=.81) across its subscales, indicating reliable and consistent measurement. Furthermore, a significant positive correlation (P<.001) was found between the subscales and the overall score, supporting their consistent measurement of the intended construct. Therefore, MAUQ can be considered a reliable instrument for assessing the usability of mHealth apps among German-speaking adults. The availability of the German version of MAUQ will help other researchers in conducting usability studies of mHealth apps in German-speaking cohorts and allow for international comparability of their results.

["Connected device for monitoring patients treated by prostatectomy: Implementation and qualitative assessment"]

INTRODUCTION

Support care aims to improve the experience of patients. m-health is one of the tools recently developed to promote patient empowerment. The objective of this study was to evaluate the appreciation of an m-health application to enhance prostatectomy path for patients suffering from prostate cancer.

METHOD

A prospective monocentric study was conducted in the urology department of the University Hospital of Rennes from February to April 2023. MyCHU application was optimized by integrating information sheets in the postoperative period after prostatectomy on sphincter rehabilitation exercises, erectile dysfunction and urinary incontinence. The questionnaire used to evaluate the usability of "MyCHU" application was the System Usability Scale (SUS). Semi-structured interviews explored the patients' feelings about the content of the information sheets and the impact on their empowerment regarding sexual disorders.

RESULTS

Twelve patients participated in this study and 7 agreed to complete an interview The average SUS score was 75.58, which indicate an high usability. Patients appreciated the fact that the application structured their healthcare pathway by centralizing information. The information sheets were clear and accurate. The impact on their empowerment was positive, with a gain in their ability to take ownership of the therapies.

CONCLUSION

The role of digital technology in health care has been growing in recent years. Our study has shown the interest that mobile application can bring to the patient who undergoes prostatectomy. It increases his empowerment and favor the dialogue with his surgeon.

LEVEL OF EVIDENCE: 3

Methods used to evaluate usability of mobile clinical decision support systems for healthcare emergencies: a systematic review and qualitative synthesis

Objective

The aim of this study was to determine the methods and metrics used to evaluate the usability of mobile application Clinical Decision Support Systems (CDSSs) used in healthcare emergencies. Secondary aims were to describe the characteristics and usability of evaluated CDSSs.

Materials and Methods

A systematic literature review was conducted using Pubmed/Medline, Embase, Scopus, and IEEE Xplore databases. Quantitative data were descriptively analyzed, and qualitative data were described and synthesized using inductive thematic analysis.

Results

Twenty-three studies were included in the analysis. The usability metrics most frequently evaluated were efficiency and usefulness, followed by user errors, satisfaction, learnability, effectiveness, and memorability. Methods used to assess usability included questionnaires in 20 (87%) studies, user trials in 17 (74%), interviews in 6 (26%), and heuristic evaluations in 3 (13%). Most CDSS inputs consisted of manual input (18, 78%) rather than automatic input (2, 9%). Most CDSS outputs comprised a recommendation (18, 78%), with a minority advising a specific treatment (6, 26%), or a score, risk level or likelihood of diagnosis (6, 26%). Interviews and heuristic evaluations identified more usability-related barriers and facilitators to adoption than did questionnaires and user testing studies.

Discussion

A wide range of metrics and methods are used to evaluate the usability of mobile CDSS in medical emergencies. Input of information into CDSS was predominantly manual, impeding usability. Studies employing both qualitative and quantitative methods to evaluate usability yielded more thorough results.

Conclusion

When planning CDSS projects, developers should consider multiple methods to comprehensively evaluate usability.

User Acceptability and Perceived Impact of a Mobile Interactive Education and Support Group Intervention to Improve Postnatal Health Care in Northern India: A Qualitative Study

Background

Postnatal care, crucial for preventing and assessing complications after birth, remains low in India. An interactive mHealth community-based postnatal intervention was implemented to promote healthy maternal behaviors through knowledge and social support in rural Northern India. However, there is limited information on how virtual health interventions in resource-constrained settings are perceived by the users and which elements influence their engagement and sustained participation.

Objective

We explored the user perceptions of acceptability and impact of a virtual interactive maternal and child health intervention pilot tested in Punjab State, India, including their perspectives on barriers and facilitators to engage with this intervention.

Methods

This qualitative study was embedded within extensive mixed-method research, and oriented by the Realist Evaluation approach. Sixteen participants were recruited from the parent study. They were identified by purposive sampling to cover diverse levels of attendance and engagement with the intervention. In-depth interviews were conducted by phone. Following translation, a framework analysis was completed to search for the main themes. Feedback was requested from intervention moderators during the process to prioritize local interpretation.

Results

Study participants reported overall satisfaction with the intervention. The mothers appreciated the educational material provided and the communication with other participants and health professionals. Across context, intervention, and actor domains, the barriers most commented on were network and connectivity challenges, lack of time due to household responsibilities, and feeling uncomfortable sharing personal experiences. Family buy-in and support were fundamental for overcoming the high domestic workload and baby care. Another facilitator mentioned was moderators' guidance on using the different intervention modalities. Regarding perceived impact, participants shared that MeSSSSage increased their capability and motivation to breastfeed, seek care as needed, and use contraception according to their preferences. Finally, participants suggested adding more topics to the educational content and adjusting the dynamics within the group calls to improve the intervention.

Conclusions

This study identifies the high acceptability and perceived impact of a novel postnatal care program in a rural setting, including the users' perceived barriers to engaging with the intervention and possible solutions to overcome them. These findings enable refinement of the ongoing intervention, providing a more robust framing for its scalability and long-term sustainability. On a larger scale, conclusions from this research provide new insights and encouragement to global stakeholders who aspire to improve maternal and neonatal outcomes in low-income and middle-income countries through mHealth.

A Tablet-Based App to Support Nursing Home Staff in Delivering an Individualized Cognitive and Physical Exercise Program for Individuals With Dementia: Mixed Methods Usability Study

BACKGROUND

The promotion of physical activity in individuals with dementia living in nursing homes is crucial for preserving physical and cognitive functions and the associated quality of life. Nevertheless, the implementation of physical activity programs in this setting is challenging, as the time and expertise of nursing home staff are limited. This situation was further exacerbated by the COVID-19 pandemic. Mobile health apps may be a sustainable approach to overcome these challenges in the long term. Therefore, the Individualized Cognitive and Physical Exercise-App (the InCoPE-App) was developed to support nursing home staff in delivering and implementing tailored cognitive and physical exercise training for individuals with dementia.

OBJECTIVE

This study aims to assess the usability of the InCoPE-App in terms of user performance and user perception in a laboratory setting using a mixed methods approach.

METHODS

Nursing home staff were encouraged to perform 5 basic tasks within the InCoPE-App. Their thoughts while using the app were captured by implementing a think aloud protocol. Then, participants completed the System Usability Scale questionnaire. The think aloud transcripts were qualitatively evaluated to unveil usability issues. All identified issues were rated in terms of their necessity to be fixed. Task completion (ie, success rate and time) and perceived usability were evaluated descriptively.

RESULTS

A total of 14 nursing home employees (mean age 53.7, SD 10.6 years; n=13, 93% women) participated in the study. The perceived usability of the InCoPE-App, as assessed by the System Usability Scale questionnaire, can be rated as "good." The main usability issues concerned navigation logic and comprehensibility of app content.

CONCLUSIONS

The InCoPE-App is a user-friendly app that enables nursing home staff to deliver and implement cognitive and physical exercise training for individuals with dementia in nursing homes. The InCoPE-App can be used with little training, even by people aged ≥50 years, who may have low digital literacy. To achieve sustainable use and high user satisfaction of the InCoPE-App in the long term, it should be implemented and evaluated in a field study.

A novel smartphone application for the informal caregivers of cancer patients: Usability study

Informal caregivers are a critical source of support for cancer patients. However, their perspectives are not routinely collected, despite health impacts related to the burden of caregiving. We created the TOGETHERCare smartphone application (app) to collect observer-reported outcomes regarding the cancer patient's health and caregiver's perceptions of their own mental and physical health, and to provide tips and resources for self-care and patient care. We enrolled 54 caregivers between October 2020 and March 2021 from Kaiser Permanente Northern California (KPNC), an integrated healthcare system. Fifty caregivers used the app for approximately 28 days. Usability and acceptability were assessed using questions from the Mobile App Rating Scale (MARS), the System Usability Scale (SUS), the Net Promoter Score (NPS), and semi-structured interviews. The caregivers' mean age was 54.4 years, 38% were female and 36% were non-White. The SUS total mean score was 83.4 (SD = 14.2), for a percentile rank of 90-95 ("excellent"). Median MARS responses to the functionality questions were also high. The NPS score of 30 at the end of the study indicated that most caregivers would recommend the app. Themes from semi-structured interviews were consistent across the study period and indicated that the app was easy to use and helpful. Caregivers indicated a need for feedback from the app, suggested some changes to the wording of questions, the app's visuals, and timing of notifications. This study demonstrated that caregivers are willing to complete frequent surveys about themselves and their patients. The app is unique because it provides a remote method to collect caregivers' observations about the patient that may be useful for clinical care. To our knowledge, TOGETHERCare is the first mobile app developed specifically to capture adult cancer patient symptoms from the informal caregiver's perspective. Future research will examine whether use of this app can help improve patient outcomes.

User testing to modify the MyHealthyGut digital health application for inflammatory bowel disease

Introduction

Inflammatory bowel disease, characterized by chronic intestinal inflammation, can be subcategorized into Crohn's disease and ulcerative colitis. The treatment for these conditions is unique to each patient and may include lifestyle changes, pharmaceutical intervention, and surgery. Lifestyle changes, such as dietary intervention, are a cornerstone of inflammatory bowel disease symptom management. Given the daily burden of this disease, self-management is paramount in coping with and/or minimizing symptoms. The MyHealthyGut application, successfully proven to be a self-management tool for celiac disease, shows promise for use in an inflammatory bowel disease patient population.

Objective

To conduct user testing to gather valuable insights for the development of an IBD-focused version of the existing MyHealthyGut app.

Methods

Participants included inflammatory bowel disease patients and healthcare practitioners. Participants used the application for a 2-week period, followed by participation in a focus group or individual interview to provide feedback. Qualitative questionnaires were administered verbally and feedback was recorded. Thematic analysis techniques were used for data quantification and analysis.

Results

15 participants were recruited and enrolled. Of these, 14 participants took part in the focus group and/or individual interviews. The feedback suggested changes related to clinical uses, food and symptom tracking, ease of use, and educational content. All (100%) participants reported that they would either use the application themselves or recommend it to patients, once their suggestions were implemented.

Conclusion

Through user testing and feedback collection, priorities for app modification were identified. Areas for modification in the app functions and features, ease of use, and content were identified. Once updated to meet the needs of inflammatory bowel disease patients, the MyHealthyGut app may be a useful tool for IBD self-management.

Usability Evaluation of Dashboards: A Systematic Literature Review of Tools

Introduction

In recent years, the use of dashboards in healthcare has been considered an effective approach for the visual presentation of information to support clinical and administrative decisions. Effective and efficient use of dashboards in clinical and managerial processes requires a framework for the design and development of tools based on usability principles.

Objectives

The present study is aimed at investigating the existing questionnaires used for the usability evaluation framework of dashboards and at presenting more specific usability criteria for evaluating dashboards.

Methods

This systematic review was conducted using PubMed, Web of Science, and Scopus, without any time restrictions. The final search of articles was performed on September 2, 2022. Data collection was performed using a data extraction form, and the content of selected studies was analyzed based on the dashboard usability criteria.

Results

After reviewing the full text of relevant articles, a total of 29 studies were selected according to the inclusion criteria. Regarding the questionnaires used in the selected studies, researcher-made questionnaires were used in five studies, while 25 studies applied previously used questionnaires. The most widely used questionnaires were the System Usability Scale (SUS), Technology Acceptance Model (TAM), Situation Awareness Rating Technique (SART), Questionnaire for User Interaction Satisfaction (QUIS), Unified Theory of Acceptance and Use of Technology (UTAUT), and Health Information Technology Usability Evaluation Scale (Health-ITUES), respectively. Finally, dashboard evaluation criteria, including usefulness, operability, learnability, ease of use, suitability for tasks, improvement of situational awareness, satisfaction, user interface, content, and system capabilities, were suggested.

Conclusion

General questionnaires that were not specifically designed for dashboard evaluation were mainly used in reviewed studies. The current study suggested specific criteria for measuring the usability of dashboards. When selecting the usability evaluation criteria for dashboards, it is important to pay attention to the evaluation objectives, dashboard features and capabilities, and context of use.

An Investigation on Chinese Public Acceptance of COVID-19 Prevention Measures

China has basically succeeded in bringing the COVID-19 epidemic under control, thanks to a timely series of effective prevention and control measures taken by the Chinese government. In this study, a public acceptance questionnaire of epidemic prevention measures was designed to investigate the influencing factors of public acceptance. A total of 2062 samples were collected from 8 March 2020 to 9 April 2020, and Independent-Samples T-Test and One-way ANOVA were used to analyze the data collected in the questionnaire in SPSS version 22.0. The results show that age and educational level have a significant influence on public acceptance. With the development of the epidemic, the acceptability grew generally higher. The public acceptance of traffic measures is the highest. This study summarises China’s scientific experience in the fight against COVID-19 and the differences in public acceptance. It can provide a positive reference for the development of epidemic prevention in other countries.

Design and evaluation of a mobile-based nutrition education application for infertile women in Iran

Background

The prevalence of infertility in Iran is higher than the world average. Furthermorte, education and nutrition are among the effective factors in improving the quality of life of women with infertility. According to the significant role of smartphones in people's lives as well as health education, the present study aimed to design and evaluate a mobile-based nutrition education application for infertile women.

Methods

This quantitative research was conducted in two stages. Initially, the educational contents were determined based on a review of the literature. Later, the obtained contents were given to 10 nutritionists and five infertility specialists to determine the necessity of each item. In the next stage, the application prototype was designed based on the results of the first stage and distributed among 220 infertile women. After two months, the Questionnaire of User Interface Satisfaction was administrated to assess the usability of the developed application. The results were analyzed via SPSS software version 20.

Results

According to the nutritionists and infertility specialists, the contents determined for the nutrition educational application were categorized under three general sections of user's demographic data, educational contents, required capabilities. The users' mean score of the application usability was calculated as 7.44 out of 9 indicating a good level of satisfaction.

Conclusions

Nutrition education of women with infertility problems can play a significant role in improving their awareness and treatment outcomes. Due to the increasing use of smartphones, designing a mobile-based nutrition educational application can be of great benefit for women with infertility according to the cultural conditions and characteristics of each community.

Development of a Healthcare Information System for Community Care of Older Adults and Evaluation of Its Acceptance and Usability

Objective

The need for health and social care for community-dwelling elderly is on the rise as the population ages. Through the provision of comprehensive services by multiple professionals in local communities, elderly people can receive continual care in a non-medical setting, which is favorable for early detection and intervention of potential problems. However, the lack of digitalization in primary care affects the effectiveness of the services and precludes full exploitation of the data. This study proposed an information system dedicated to caring for community-dwelling elderly people and investigated its acceptance and usability.

Methods

An information system was designed for elderly care centers in the community, where data generated during care delivery, involving socio-demographic data, bio-measurements and health assessments and questionnaires, were digitized and stored for information management and exchange. A study was conducted to evaluate the acceptance and usability of the system after routine use of 6 months. The users of the system at an elderly care center were recruited to respond to a technology acceptance questionnaire and a system usability questionnaire.

Results

The mean scores of the acceptance and usability questionnaires reached 5.1 out of the highest possible score of 7. The constructs of the acceptance questionnaire had good reliability. The social influence and facilitating conditions constructs had a significant correlation with the behavioral intention construct.

Conclusions

The proposed information system demonstrated good acceptance and usability, which supported the feasibility of implementing it in community care centers for older adults. Further research will be conducted to address the limitation of sample size by extending the system to other elderly care centers, forming a large user base for a more in-depth and comprehensive performance evaluation.