System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis

Usability and usefulness of (electronic) patient identification systems-A cross-sectional evaluation in German-speaking radiation oncology departments

PURPOSE

Patient misidentification in radiation oncology (RO) is a significant concern due to the potential harm to patient health and the burden on healthcare systems. Electronic patient identification systems (ePIS) are increasingly being used as an alternative or supplement to organizational systems (oPIS). The objective of this study was to assess the usability and usefulness of ePIS and oPIS in German-speaking countries.

METHODS

A cross-sectional survey was designed by a group of experts from various professional backgrounds in RO. The survey consisted of 38 questions encompassing quantitative and qualitative data on usability, user experience, and usefulness of PIS. It was available between August and October 2022.

RESULTS

Of 118 eligible participants, 37% had implemented some kind of ePIS. Overall, 22% of participants who use an oPIS vs. 10% of participants who use an ePIS reported adverse events in terms of patients' misidentification in the past 5 years. Frequent or very frequent drop-outs of electronic systems were reported by 31% of ePIS users. Users of ePIS significantly more often affirmed a positive cost-benefit ratio of ePIS as well as an improvement of workflow, whereas users of oPIS more frequently apprehended a decrease in staffs' attention through ePIS. The response rate was 8%.

CONCLUSION

The implementation of ePIS can contribute to efficient PI and improved processes. Apprehensions by oPIS users and assessments of ePIS users differ significantly in aspects of the perceived usefulness of ePIS. However, technical problems need to be addressed to ensure the reliability of ePIS. Further research is needed to assess the impact of different PIS on patient safety in RO.

Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study

Background

The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result.

Objective

This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements.

Methods

A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach.

Results

Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified.

Conclusions

The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Telemedicine home-based management in patients with chronic heart failure and diabetes type II: study protocol for a randomized controlled trial

BACKGROUND

Heart failure and type 2 diabetes are prevalent public health issues in Europe. These complex chronic conditions require extensive pharmacological management, ongoing self-care, and behavioral changes. Despite the known benefits of lifestyle changes, such as regular exercise and better control of blood sugar levels, patients may need help implementing the recommended changes. This study aims to assess the effectiveness of a telemedicine program for managing heart failure and type 2 diabetes at home. The program focuses on promoting lifestyle changes.

METHODS AND ANALYSIS

During scheduled outpatient cardiology evaluations, eligible patients are recruited and randomly assigned to either an intervention or control group in a 1:1 ratio. The intervention group receives support from a nursing case manager through a structured home-based teleassistance program and a trainer for daily physical activity stimulation. They also have access to teleconsultations with cardiologists and diabetes specialists as needed, telemonitoring of vital signs, and daily step tracking. An app records and monitors daily drug treatment, glycemia, blood pressure, heart rate, and other clinical parameters. Patients can also self-report symptoms and communicate via a chat and videoconference system with a Nurse Case Manager. The control group receives routine care. Data collection occurs before intervention and 6 months after baseline during a new outpatient cardiology evaluation. The primary outcome is to measure the difference in the distance walked during a 6-min walk test between baseline and after 6 months. The key secondary outcomes include improving the disease status and physical activity profile. Data will be analyzed according to the intention-to-treat principles.

DISCUSSION

This study will provide evidence on the efficacy of a telemedicine home-based management model to maintain correct lifestyles in patients with both heart failure and type 2 diabetes, improving self-management, their empowerment on the diseases, and increasing their knowledge and ability to recognize symptoms early.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05633784. Registered on November 30, 2022.

Co-design and usability of an interactive web-based fertility decision aid for transgender youth and young adults

OBJECTIVE

To develop a patient- and family-centered Aid For Fertility-Related Medical Decisions (AFFRMED) interactive website targeted for transgender and nonbinary (TNB) youth/young adults and their parents to facilitate shared decision-making about fertility preservation interventions through user-centered participatory design.

METHOD

TNB youth/young adults interested in or currently receiving pubertal suppression or gender-affirming hormone treatment and parents of eligible TNB youth/young adults were recruited to participate in a series of iterative human-centered co-design sessions to develop an initial AFFRMED prototype. Subsequently, TNB youth/young adults and parents of TNB youth/young adults were recruited for usability testing interviews, involving measures of usability (i.e., After Scenario Questionnaire, Net Promotor Score, System Usability Scale).

RESULTS

Twenty-seven participants completed 18 iterative co-design sessions and provided feedback on 10 versions of AFFRMED (16 TNB youth/young adults and 11 parents). Nine TNB youth/young adults and six parents completed individual usability testing interviews. Overall, participants rated AFFRMED highly on measures of acceptability, appropriateness, usability, and satisfaction. However, scores varied by treatment cohort, with TNB youth interested in or currently receiving pubertal suppression treatment reporting the lowest usability scores.

CONCLUSIONS

We co-created a youth- and family-centered fertility decision aid prototype that provides education and decision support in an online, interactive format. Future directions include testing the efficacy of the decision aid in improving fertility and fertility preservation knowledge, decisional self-efficacy, and decision satisfaction.

Moniz Game: Usability and User Experience Evaluation of a Musical Game for Motor Coordination

Purpose: The objective of this research was to develop a musical digital game for rehabilitation of upper limb and to verify its usability and user experience with professionals in the field (physical therapists). Materials and Methods: Thirty working professionals were recruited to evaluate the system. The usability was evaluated with the System Usability Scale (SUS) and the user experience was verified with the Game Flow scale. Results: The overall score of the SUS scale was 88.67 (±9.129); this score is interpreted as "Best Imaginable" (86-100). The user experience rating had most of its domains equal or higher than 4, which indicates that all the requirements for a good user experience were present in the game. Conclusions: The Moniz Game proved to be a game with good usability and can be a tool for application in clinical practice regarding motor coordination. However, further studies are needed to evaluate the effect of the Moniz Game on motor coordination in patients with neurological dysfunctions.

Randomized Controlled Trial of Clinical Guidelines Versus Interactive Decision-Support for Improving Medical Trainees' Confidence with Latent Tuberculosis Care

BACKGROUND

In order to eliminate tuberculosis (TB) in the USA, primary care providers must take on an expanded role in the diagnosis and management of latent tuberculosis infection (LTBI). Clinical practice guidelines and recommendations exist for LTBI management, but there is a need for innovative tools to improve medical students' and residents' knowledge of evidence-based practices for LTBI testing and treatment.

OBJECTIVE

To assess the impact of LTBI-ASSIST, a free online decision support aid, as a novel educational tool and mechanism of delivering clinical practice guidelines for medical trainees.

DESIGN

A single site, randomized controlled trial of trainees delivered by electronic survey.

INTERVENTIONS

Medical students and Internal Medicine residents at the Johns Hopkins University School of Medicine.

PARTICIPANTS

Participants were randomized in 1:1 ratio to receive the US clinical practice guidelines and recommendations for Latent TB management (control arm) or the guidelines plus an introduction to LTBI-ASSIST (LTBI-ASSIST arm) as they completed a case-based knowledge assessment and reported confidence with domains of LTBI care.

MAIN MEASURES

(1) Proportion of questions answered correctly on a case-based knowledge assessment; (2) change in reported confidence with domains of LTBI care.

KEY RESULTS

One hundred and thirty participants completed the knowledge assessment. Those randomized to receive the LTBI-ASSIST Tool performed better on the case-based knowledge assessment with a mean score of 75.9% (95% CI: 70.6-81.1), compared to 57.4% (52.8-62.0) in the group that received the guidelines only (p <0.001). Similarly, the LTBI-ASSIST group reported a higher change in confidence (measured as post-assessment confidence minus pre-assessment confidence), compared to the control group, in six of the seven domains of LTBI care.

CONCLUSIONS

LTBI-ASSIST can be an effective supplement to existing guidelines in educating medical trainees and helping providers find evidence-based, guideline-supported answers for questions encountered in clinical practice.

TRIAL REGISTRATION

NIH Clinical Trial Registry No. NCT05772065.

Feasibility study of rehabilitation for cardiac patients aided by an artificial intelligence web-based programme: a randomised controlled trial (RECAP trial)-a study protocol

INTRODUCTION

Cardiac rehabilitation (CR) delivered by rehabilitation specialists in a healthcare setting is effective in improving functional capacity and reducing readmission rates after cardiac surgery. It is also associated with a reduction in cardiac mortality and recurrent myocardial infarction. This trial assesses the feasibility of a home-based CR programme delivered using a mobile application (app).

METHODS

The Rehabilitation through Exercise prescription for Cardiac patients using an Artificial intelligence web-based Programme (RECAP) randomised controlled feasibility trial is a single-centre prospective study, in which patients will be allocated on a 1:1 ratio to a home-based CR programme delivered using a mobile app with accelerometers or standard hospital-based rehabilitation classes. The home-based CR programme will employ artificial intelligence to prescribe exercise goals to the participants on a weekly basis. The trial will recruit 70 patients in total. The primary objectives are to evaluate participant recruitment and dropout rates, assess the feasibility of randomisation, determine acceptability to participants and staff, assess the rates of potential outcome measures and determine hospital resource allocation to inform the design of a larger randomised controlled trial for clinical efficacy and health economic evaluation. Secondary objectives include evaluation of health-related quality of life and 6 minute walk distance.

ETHICS AND DISSEMINATION

RECAP trial received a favourable outcome from the Berkshire research ethics committee in September 2022 (IRAS 315483).Trial results will be made available through publication in peer-reviewed journals and presented at relevant scientific meetings.

TRIAL REGISTRATION NUMBER

ISRCTN97352737.

Identifying Factors of User Acceptance of a Drone-Based Medication Delivery: User-Centered Design Approach

BACKGROUND

The use of drones in the health care sector is increasingly being discussed against the background of the aging population and the growing shortage of skilled workers. In particular, the use of drones to provide medication in rural areas could bring advantages for the care of people with and without a need for care. However, there are hardly any data available that focus on the interaction between humans and drones.

OBJECTIVE

This study aims to disclose and analyze factors associated with user acceptance of drone-based medication delivery to derive practice-relevant guidance points for participatory technology development (for apps and drones).

METHODS

A controlled mixed methods study was conducted that supports the technical development process of an app design for drone-assisted drug delivery based on a participatory research design. For the quantitative analysis, established and standardized survey instruments to capture technology acceptance, such as the System Usability Scale; Technology Usage Inventory (TUI); and the Motivation, Engagement, and Thriving in User Experience model, were used. To avoid possible biasing effects from a continuous user development (eg, response shifts and learning effects), an ad hoc group was formed at each of the 3 iterative development steps and was subsequently compared with the consisting core group, which went through all 3 iterations.

RESULTS

The study found a positive correlation between the usability of a pharmacy drone app and participants' willingness to use it (r=0.833). Participants' perception of usefulness positively influenced their willingness to use the app (r=0.487; TUI). Skepticism had a negative impact on perceived usability and willingness to use it (r=-0.542; System Usability Scale and r=-0.446; TUI). The study found that usefulness, skepticism, and curiosity explained most of the intention to use the app (F3,17=21.12; P<.001; R2=0.788; adjusted R2=0.751). The core group showed higher ratings on the intention to use the pharmacy drone app than the ad hoc groups. Results of the 2-tailed t tests showed a higher rating on usability for the third iteration of the core group compared with the first iteration.

CONCLUSIONS

With the help of the participatory design, important aspects of acceptance could be revealed by the people involved in relation to drone-assisted drug delivery. For example, the length of time spent using the technology is an important factor for the intention to use the app. Technology-specific factors such as user-friendliness or curiosity are directly related to the use acceptance of the drone app. Results of this study showed that the more participants perceived their own competence in handling the app, the more they were willing to use the technology and the more they rated the app as usable.

MyTrack+: Human-centered design of an mHealth app to support long-term weight loss maintenance

A growing body of research has focused on the utility of adaptive intervention models for promoting long-term weight loss maintenance; however, evaluation of these interventions often requires customized smartphone applications. Building such an app from scratch can be resource-intensive. To support a novel clinical trial of an adaptive intervention for weight loss maintenance, we developed a companion app, MyTrack+, to pair with a main commercial app, FatSecret (FS), leveraging a user-centered design process for rapid prototyping and reducing software engineering efforts. MyTrack+ seamlessly integrates data from FS and the BodyTrace smart scale, enabling participants to log and self-monitor their health data, while also incorporating customized questionnaires and timestamps to enhance data collection for the trial. We iteratively refined the app by first developing initial mockups and incorporating feedback from a usability study with 17 university students. We further improved the app based on an in-the-wild pilot study with 33 participants in the target population, emphasizing acceptance, simplicity, customization options, and dual app usage. Our work highlights the potential of using an iterative human-centered design process to build a companion app that complements a commercial app for rapid prototyping, reducing costs, and enabling efficient research progress.

ChatGPT-3.5 System Usability Scale early assessment among Healthcare Workers: Horizons of adoption in medical practice

Artificial intelligence (AI) chatbots, such as ChatGPT, have widely invaded all domains of human life. They have the potential to transform healthcare future. However, their effective implementation hinges on healthcare workers' (HCWs) adoption and perceptions. This study aimed to evaluate HCWs usability of ChatGPT three months post-launch in Saudi Arabia using the System Usability Scale (SUS). A total of 194 HCWs participated in the survey. Forty-seven percent were satisfied with their usage, 57 % expressed moderate to high trust in its ability to generate medical decisions. 58 % expected ChatGPT would improve patients' outcomes, even though 84 % were optimistic of its potential to improve the future of healthcare practice. They expressed possible concerns like recommending harmful medical decisions and medicolegal implications. The overall mean SUS score was 64.52, equivalent to 50 % percentile rank, indicating high marginal acceptability of the system. The strongest positive predictors of high SUS scores were participants' belief in AI chatbot's benefits in medical research, self-rated familiarity with ChatGPT and self-rated computer skills proficiency. Participants' learnability and ease of use score correlated positively but weakly. On the other hand, medical students and interns had significantly high learnability scores compared to others, while ease of use scores correlated very strongly with participants' perception of positive impact of ChatGPT on the future of healthcare practice. Our findings highlight the HCWs' perceived marginal acceptance of ChatGPT at the current stage and their optimism of its potential in supporting them in future practice, especially in the research domain, in addition to humble ambition of its potential to improve patients' outcomes particularly in regard of medical decisions. On the other end, it underscores the need for ongoing efforts to build trust and address ethical and legal concerns of AI implications in healthcare. The study contributes to the growing body of literature on AI chatbots in healthcare, especially addressing its future improvement strategies and provides insights for policymakers and healthcare providers about the potential benefits and challenges of implementing them in their practice.

Evaluating the Usability of an mHealth App for Empowering Cancer Survivors With Disabilities: Heuristic Evaluation and Usability Testing

BACKGROUND

More than 18 million cancer survivors are living in the United States. The effects of cancer and its treatments can have cognitive, psychological, physical, and social consequences that many survivors find incredibly disabling. Posttreatment support is often unavailable or underused, especially for survivors living with disabilities. This leaves them to deal with new obstacles and struggles on their own, oftentimes feeling lost during this transition. Mobile health (mHealth) interventions have been shown to effectively aid cancer survivors in dealing with many of the aftereffects of cancer and its treatments; these interventions hold immense potential for survivors living with disabilities. We developed a prototype for WeCanManage, an mHealth-delivered self-management intervention to empower cancer survivors living with disabilities through problem-solving, mindfulness, and self-advocacy training.

OBJECTIVE

Our study conducted a heuristic evaluation of the WeCanManage high-fidelity prototype and assessed its usability among cancer survivors with known disabilities.

METHODS

We evaluated the prototype using Nielsen's 10 principles of heuristic evaluation with 22 human-computer interaction university students. On the basis of the heuristic evaluation findings, we modified the prototype and conducted usability testing on 10 cancer survivors with a variety of known disabilities, examining effectiveness, efficiency, usability, and satisfaction, including a completion of the modified System Usability Scale (SUS).

RESULTS

The findings from the heuristic evaluation were mostly favorable, highlighting the need for a help guide, addressing accessibility concerns, and enhancing the navigation experience. After usability testing, the average SUS score was 81, indicating a good-excellent design. The participants in the usability testing sample expressed positive reactions toward the app's design, educational content and videos, and the available means of connecting with others. They identified areas for improvement, such as improving accessibility, simplifying navigation within the community forums, and providing a more convenient method to access the help guide.

CONCLUSIONS

Overall, usability testing showed positive results for the design of WeCanManage. The course content and features helped participants feel heard, understood, and less alone.

Patient-Centered Chronic Wound Care Mobile Apps: Systematic Identification, Analysis, and Assessment

Background

The prevalence of chronic wounds is predicted to increase within the aging populations in industrialized countries. Patients experience significant distress due to pain, wound secretions, and the resulting immobilization. As the number of wounds continues to rise, their adequate care becomes increasingly costly in terms of health care resources worldwide. eHealth support systems are being increasingly integrated into patient care. However, to date, no systematic analysis of such apps for chronic wounds has been published.

Objective

The aims of this study were to systematically identify and subjectively assess publicly available German- or English-language mobile apps for patients with chronic wounds, with quality assessments performed by both patients and physicians.

Methods

Two reviewers independently conducted a systematic search and assessment of German- or English-language mobile apps for patients with chronic wounds that were available in the Google Play Store and Apple App Store from April 2022 to May 2022. In total, 3 apps met the inclusion and exclusion criteria and were reviewed independently by 10 physicians using the German Mobile App Rating Scale (MARS) and the System Usability Scale (SUS). The app with the highest mean MARS score was subsequently reviewed by 11 patients with chronic wounds using the German user version of the MARS (uMARS) and the SUS. Additionally, Affinity for Technology Interaction (ATI) scale scores were collected from both patients and physicians.

Results

This study assessed mobile apps for patients with chronic wounds that were selected from a pool of 118 identified apps. Of the 73 apps available in both app stores, 10 were patient oriented. After excluding apps with advertisements or costs, 3 apps were evaluated by 10 physicians. Mean MARS scores ranged from 2.64 (SD 0.65) to 3.88 (SD 0.65) out of 5, and mean SUS scores ranged from 50.75 (SD 27) to 80.5 (SD 17.7) out of 100. WUND APP received the highest mean MARS score (mean 3.88, SD 0.65 out of 5) among physicians. Hence, it was subsequently assessed by 11 patients and achieved a similar rating (uMARS score: mean 3.89, SD 0.4 out of 5). Technical affinity, as measured with the ATI scale, was slightly lower in patients (score: mean 3.62, SD 1.35 out of 6) compared to physicians (score: mean 3.88, SD 1.03 out 6).

Conclusions

The quality ratings from physicians and patients were comparable and indicated mediocre app quality. Technical affinity, as assessed by using the ATI scale, was slightly lower for patients. Adequate apps for patients with chronic wounds remain limited, emphasizing the need for improved app development to meet patient needs. The ATI scale proved valuable for assessing technical affinity among different user groups.

WeChat mini program in laboratory biosafety education among medical students at Guangzhou Medical University: a mixed method study of feasibility and usability

BACKGROUND

Laboratory biosafety should be a priority in all healthcare institutions. In traditional laboratory safety teaching students typically receive knowledge passively from their teachers without active involvement. The combination of experiential learning and mobile learning may provide students with greater engagement, retention, and application of knowledge. To address this issue, we developed and conducted a convergent mixed methods study to assess the feasibility and usability of a WeChat mini program (WMP) named WeMed for laboratory biosafety education for medical laboratory students at Guangzhou Medical University (GMU).

METHODS

The study was conducted between November 2022 and October 2023 among second-year undergraduate students at GMU. It involved the concurrent collection, analysis, and interpretation of both qualitative and quantitative data to assess feasibility and usability. In the quantitative strand, two evaluations were conducted via online surveys from students (n = 67) after a four-week study period. The System Usability Scale (SUS) was used to evaluate usability, while self-developed questions were used to assess feasibility. Additionally, a knowledge test was administered 6 months after the program completion. In the qualitative strand, fourteen semi-structured interviews were conducted, whereby a reflexive thematic analysis was utilized to analyze the interview data.

RESULTS

The overall SUS score is adequate (M = 68.17, SD = 14.39). The acceptability of the WeMed program is in the marginal high range. Most students agreed that WeMed was useful for learning biosafety knowledge and skills (13/14, 93%), while 79% (11/14) agreed it was easy to use and they intended to continue using it. After 6 months, a significant difference in the knowledge test scores was observed between the WeMed group (n = 67; 2nd year students) and the traditional training group (n = 90; 3rd year students). However, the results should be interpreted cautiously due to the absence of a pretest.

CONCLUSION

The combination of experiential learning and mobile learning with WMP is a feasible tool for providing laboratory biosafety knowledge and skills. Ongoing improvements should be made in order to increase long-term acceptance.

Evaluating the impact of movement tracking feedback on engagement with home exercise programmes of children with cerebral palsy using a new therapy app: a protocol for a mixed-methods single-case experimental design with alternating treatments

INTRODUCTION

Children with cerebral palsy (CP) are prescribed home exercise programmes (HEPs) to increase the frequency of movement practice, yet adherence to HEPs can be low. This paper outlines the protocol for a single-case experimental design (SCED) with alternating treatments, using a new home therapy exercise application, Bootle Boot Camp (BBCamp), offered with and without movement tracking feedback. This study will explore the impact of feedback on engagement, movement quality, lower limb function and family experiences to help understand how technology-supported HEPs should be translated and the added value, if any, of movement tracking technology.

METHODS AND ANALYSIS

In this explanatory sequential mixed-methods study using a SCED, 16 children with CP (aged 6-12 years, Gross Motor Function Classification System levels I-II) will set lower limb goals and be prescribed an individualised HEP by their physiotherapist to complete using BBCamp on their home television equipped with a three-dimensional camera-computer system. Children will complete four weekly exercise sessions over 6 weeks. Children will be randomised to 1 of 16 alternating treatment schedules where BBCamp will provide or withhold feedback during the first 4 weeks. The version of BBCamp that results in the most therapeutic benefit will be continued for 2 final weeks. Goals will be re-evaluated and families interviewed. The primary outcome is adherence (proportion of prescribed exercise repetitions attempted) as a measure of behavioural engagement. Secondary outcomes are affective and cognitive engagement (smiley face ratings), exercise fidelity, lower limb function, goal achievement and participant experiences. SCED data will be analysed using visual and statistical methods. Quantitative and qualitative data will be integrated using joint displays.

ETHICS AND DISSEMINATION

Ethical approval was obtained from the Research Ethics Boards at Bloorview Research Institute and the University of Toronto. Results will be distributed through peer-reviewed journals and scientific conferences.

TRIAL REGISTRATION NUMBER

NCT05998239; pre-results.

Designing an App to Support Measurement-Based Peer Supervision of Frontline Health Workers Delivering Brief Psychosocial Interventions in Texas: Multimethod Study

BACKGROUND

The unmet need for mental health care affects millions of Americans. A growing body of evidence in implementation science supports the effectiveness of task sharing in the delivery of brief psychosocial interventions. The digitization of training and processes supporting supervision can rapidly scale up task-shared interventions and enable frontline health workers (FLWs) to learn, master, and deliver interventions with quality and support.

OBJECTIVE

We aimed to assess the perceived feasibility and acceptability of a novel mobile and web app designed and adapted to support the supervision, training, and quality assurance of FLWs delivering brief psychosocial interventions.

METHODS

We followed human-centered design principles to adapt a prototype app for FLWs delivering brief psychosocial interventions for depression, drawing from an app previously designed for use in rural India. Using a multimethod approach, we conducted focus group sessions comprising usability testing and group interviews with FLWs recruited from a large health system in Texas to assess the feasibility and acceptability of the app. The positive System Usability Scale was used to determine the app's overall usability. We also calculated the participants' likelihood of recommending the app to others using ratings of 0 to 10 from least to most likely (net promoter score). Focus group transcripts were coded and analyzed thematically, and recommendations were summarized across 4 key domains.

RESULTS

A total of 18 FLWs varying in role and experience with client care participated in the study. Participants found the app to be usable, with an average System Usability Scale score of 72.5 (SD 18.1), consistent with the industry benchmark of 68. Participants' likelihood of recommending the app ranged from 5 to 10, yielding a net promoter score of 0, indicating medium acceptability. Overall impressions of the app from participants were positive. Most participants (15/18, 83%) found the app easy to access and navigate. The app was considered important to support FLWs in delivering high-quality mental health care services. Participants felt that the app could provide more structure to FLW training and supervision processes through the systematic collection and facilitation of performance-related feedback. Key concerns included privacy-related and time constraints regarding implementing a separate peer supervision mechanism that may add to FLWs' workloads.

CONCLUSIONS

We designed, built, and tested a usable, functional mobile and web app prototype that supports FLW-delivered psychosocial interventions in the United States through a structured supervision mechanism and systematic collection and review of performance measures. The app has the potential to scale the work of FLWs tasked with delivering these interventions to the hardest-to-reach communities they serve. The results of this project will inform future work to evaluate the app's use and efficacy in real-world settings to support task-shared mental health programs across the United States.

A cancer personalised activity and lifestyle tool (CAN-PAL): A codesign study with patients and healthcare professionals

AIMS

To codesign a cancer personalised activity and lifestyle tool (CAN-PAL) based on an existing tool. To help cancer care workers support people affected by cancer to plan and integrate physical activity into lifestyles.

DESIGN

Mixed-methods codesign study.

METHODS

Phase 1: Focus groups with people affected by cancer (n = 10) or interviews (n = 2) to discuss suitable physical activities and adaptation of the existing tool. Data were recorded, transcribed and analysed thematically. Themes informed the design of the prototype CAN-PAL and user guide. Phase 2: Healthcare professionals considered the potential use of the CAN-PAL prototype and completed an online survey including the system usability scale and free text responses.

RESULTS

Phase 1: Identified suitable physical activities and four themes were identified including: Capability, benefits, barriers and resources which informed the prototype CAN-PAL and user guide. Phase 2: The user survey was completed by 12 healthcare professionals. Median (range) system usability scale was 80 (50-95) (best score 100), scores >68 indicate good or better usability. Themes from the free text comments included strengths, amendments, considerations and limitations. Results were used to finalise CAN-PAL and the user guide.

CONCLUSION

The codesigned CAN-PAL tool had good usability. Further work is needed to evaluate the impact of CAN-PAL on activity levels and behaviour in people affected by cancer.

RELEVANCE TO CLINICAL PRACTICE

People affected by cancer need support to undertake physical activity. The purpose of CAN-PAL is to assist cancer care workers to support people affected by cancer to plan and integrate physical activity into lifestyles.

PATIENT OR PUBLIC CONTRIBUTION

Public partners considered the findings from Phase 1 and 2 and informed the design of the prototype, final CAN-PAL and user guide and coauthored the paper.

REPORTING METHOD

The study adhered to relevant EQUATOR guidelines; the study was reported according to the COREQ checklist.

A pilot study of trauma-sensitive yoga and Breathe2Relax among service members in an intensive outpatient program

Emerging research indicates that yoga is a promising adjunct to psychological trauma treatment. The current pilot study examined the associations between psychophysiological stress, diaphragmatic breathing (DB), and a trauma-sensitive yoga (TSY) regimen developed specifically for trauma-exposed service members in alignment with recent calls for precision in reporting therapeutic yoga protocols. Participants were 31 service members enrolled in a trauma-focused intensive outpatient program (IOP). Service members participated in a brief diaphragmatic breathing (DB) session using the Breathe2Relax (B2R) app followed by the TSY session. Heart rate (HR) and perceived stress were measured at baseline and after both the DB practice and the TSY session. We assessed Yoga and DB expectancies at baseline and post TSY. Participants also rated the acceptability and usability of the B2R app. Results of linear mixed effects regression models showed decreases in HR and perceived stress, compared to baseline, following DB (HR, b = -8.68, CI 95% = -13.34, -4.02; perceived stress, b = -1.77, CI 95% = -2.35, -1.18) and TSY (HR, b = -12.44, CI 95% = -17.15, -7.73; perceived stress b = -3.69, CI 95% = -4.29, -3.08). Higher levels of expectancies, compared to lower levels, related to stronger decreases in HR and perceived stress, particularly after TSY. Overall, participants rated the B2R usability as high; virtually all participants reported that "most would learn to use the app quickly," and 76.6% reported that they would use it frequently.

Evaluation of an application for the self-assessment of lifestyle behaviour in cardiac patients

BACKGROUND

Currently, no uniform, well-validated and comprehensive lifestyle behaviour self-assessment instrument exists for patients with cardiovascular disease.

PURPOSE

To evaluate the usability of a novel mobile application (LifeStyleScore) based on validated instruments for the assessment of cardiovascular risk behaviours. Secondly, the application's acceptance by healthcare professionals (HCPs) and its association with improved patient activation and lifestyle behaviour was evaluated.

METHODS

In this single-centre, non-randomised observational pilot study, patients with coronary artery disease or atrial fibrillation entering cardiac rehabilitation (CR) completed the LifeStyleScore application, the Patient Activation Measure (PAM-13®), and the System Usability Scale (SUS) during the CR intake and after CR completion. A focus group interview was performed with the HCPs involved.

RESULTS

We analysed 20 participants, 3 of whom were women, with a mean age of 61.9 ± 6.7 years. The LifeStyleScore application was rated with a SUS score above average (> 68) before (69.6 ± 13.4) and after CR (68.6 ± 15.1). All HCPs (n = 8) found the application usable. Patient activation did not increase significantly after CR compared with baseline (62.0 ± 8.6 versus 59.2 ± 9.5, respectively, p = 0.28) and only physical activity levels improved significantly (2.4 ± 0.7 (standardised score) at baseline, 2.8 ± 0.4 after CR, p = 0.04).

CONCLUSION

The LifeStyleScore application was found to be usable for patients receiving CR. Its use did not result in increased patient activation, and of the lifestyle behaviours only physical activity levels improved. Further research is needed to evaluate how such applications can be optimally incorporated in CR programmes.

Usability and feasibility of PreventS-MD web app for stroke prevention

BACKGROUND

Most strokes and cardiovascular diseases (CVDs) are potentially preventable if their risk factors are identified and well controlled. Digital platforms, such as the PreventS-MD web app (PreventS-MD) may aid health care professionals (HCPs) in assessing and managing risk factors and promoting lifestyle changes for their patients.

METHODS

This is a mixed-methods cross-sectional two-phase survey using a largely positivist (quantitative and qualitative) framework. During Phase 1, a prototype of PreventS-MD was tested internationally by 59 of 69 consenting HCPs of different backgrounds, age, sex, working experience, and specialties using hypothetical data. Collected comments/suggestions from the study HCPs in Phase 1 were reviewed and implemented. In Phase 2, a near-final version of PreventS-MD was developed and tested by 58 of 72 consenting HCPs using both hypothetical and real patient (n = 10) data. Qualitative semi-structured interviews with real patients (n = 10) were conducted, and 1 month adherence to the preventive recommendations was assessed by self-reporting. The four System Usability Scale (SUS) groups of scores (0-50 unacceptable; 51-68 poor; 68-80.3 good; >80.3 excellent) were used to determine usability of PreventS-MD.

FINDINGS

Ninety-nine HCPs from 27 countries (45% from low- to middle-income countries) participated in the study, and out of them, 10 HCPs were involved in the development of PreventS before the study, and therefore were not involved in the survey. Of the remaining 89 HCPs, 69 consented to the first phase of the survey, and 59 of them completed the first phase of the survey (response rate 86%), and 58 completed the second phase of the survey (response rate 84%). The SUS scores supported good usability of the prototype (mean score = 80.2; 95% CI [77.0-84.0]) and excellent usability of the final version of PreventS-MD (mean score = 81.7; 95% CI [79.1-84.3]) in the field. Scores were not affected by the age, sex, working experience, or specialty of the HCPs. One-month follow-up of the patients confirmed the high level of satisfaction/acceptability of PreventS-MD and (100%) adherence to the recommendations.

INTERPRETATION

The PreventS-MD web app has a high level of usability, feasibility, and satisfaction by HCPs and individuals at risk of stroke/CVD. Individuals at risk of stroke/CVD demonstrated a high level of confidence and motivation in following and adhering to preventive recommendations generated by PreventS-MD.

The most used questionnaires for evaluating the usability of robots and smart wearables: A scoping review

Background

As the field of robotics and smart wearables continues to advance rapidly, the evaluation of their usability becomes paramount. Researchers may encounter difficulty in finding a suitable questionnaire for evaluating the usability of robotics and smart wearables. Therefore, the aim of this study is to identify the most commonly utilized questionnaires for assessing the usability of robots and smart wearables.

Methods

A comprehensive search of databases, including PubMed, Web of Science, and Scopus, was conducted for this scoping review. Two authors performed the selection of articles and data extraction using a 10-field data extraction form. In cases of disagreements, a third author was consulted to reach a consensus. The inclusions were English-language original research articles that utilized validated questionnaires to assess the usability of healthcare robots and smart wearables. The exclusions comprised review articles, non-English publications, studies not focused on usability, those assessing clinical outcomes, articles lacking questionnaire details, and those using non-validated or researcher-made questionnaires. Descriptive statistics methods (frequency and percentage), were employed to analyze the data.

Results

A total of 314 articles were obtained, and after eliminating irrelevant and duplicate articles, a final selection of 50 articles was included in this review. A total of 17 questionnaires were identified to evaluate the usability of robots and smart wearables, with 10 questionnaires specifically for wearables and 7 questionnaires for robots. The System Usability Scale (50%) and Post-Study System Usability Questionnaire (19.44%) were the predominant questionnaires utilized to assess the usability of smart wearables. Moreover, the most commonly used questionnaires for evaluating the usability of robots were the System Usability Scale (56.66%), User Experience Questionnaire (16.66%), and Quebec User Evaluation of Satisfaction with Assistive Technology (10%).

Conclusion

Commonly employed questionnaires serve as valuable tools in assessing the usability of robots and smart wearables, aiding in the refinement and optimization of these technologies for enhanced user experiences. By incorporating user feedback and insights, designers can strive towards creating more intuitive and effective robotic and wearable solutions.

Development of the INpaTiEnt Rehabilitation App Compliance QuesTionnaire [INTERACT]

BACKGROUND

The elements of previously designed questionnaires do not take into consideration the burdens encountered in an inpatient hospital setting. The purpose of this study is to validate elements of a non-compliance survey in an in-hospital setting and discuss aspects of compliance with telerehabilitative physiotherapy in the early postoperative period.

METHODS

A literature search was conducted to identify elements that prevent patients from performing their prescribed physical therapy exercises. These items were then evaluated by the expert review technique as described by Ikart. Afterwards, the synthesized survey was handed out to patients for the assessment of the quality of its items.

RESULTS

The results of the expert review technique identified some conceptual and grammatical problems. This led to the adjustment of some of the elements. The quality of the resulting questionnaire was deemed to be good, as patients were able to fully understand the concepts and answer accordingly. A statistical analysis was conducted to evaluate the responses.

CONCLUSIONS

The items of this second questionnaire have proven to be reliable to assess the compliance of patients in an in-hospital setting. These items will be added to the cross-culturally adapted items of a previous questionnaire.

Quality of Digital Health Interventions Across Different Health Care Domains: Secondary Data Analysis Study

BACKGROUND

There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards.

OBJECTIVE

The objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP).

METHODS

We collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis.

RESULTS

We found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI's health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score.

CONCLUSIONS

There is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women's health, sexual health, and dental care may be needed.

Feasibility and Usability of the Job Adjustment Mobile App for Pregnant Women: Longitudinal Observational Study

BACKGROUND

Working pregnant women often need to adjust their physically demanding jobs for a healthy pregnancy. However, uncertainty about the extent of these adjustments can hinder their effectiveness. To address this, we developed the Job Adjustment mobile app, which allows users to input job and health details to generate a variety of personalized action plans. As this is the first version of the app, assessing its feasibility and usability is crucial.

OBJECTIVE

This study aims to verify the feasibility and usability of the Job Adjustment mobile app.

METHODS

A longitudinal observational study was conducted on pregnant Japanese women who were allowed to use the app anytime from 12 to 34 weeks of gestation; they received reminder emails every 2 weeks encouraging app use. A questionnaire was administered before app use and at 20 and 32 weeks of gestation. Feasibility was evaluated across 4 domains: implementation, demand, acceptability, and adverse events. Implementation was evaluated based on 3 parameters: dropout rate, initial reminder email receipt rate, and adherence rate (measured as pregnant women who used the app at intervals of 2.5 weeks or less). Demand was measured by intervals between use and intervals between log-in, and participants answered 15 questions to assess acceptability. Adverse events were assessed by analyzing the degree of anxiety related to work. Demographic data were analyzed to determine any statistically significant differences in intervals between uses. Usability was evaluated using the System Usability Scale.

RESULTS

The analysis included 66 pregnant women, and 61% (n=40) of them were multipara. The dropout rate, adherence rate, and initial reminder email receipt rate were 18% (13/71), 44% (29/66), and 79% (52/66) respectively. The median intervals between use and intervals between log-in were 2.94 (IQR 2.00-5.13) weeks and 2.28 (IQR 1.81-4.00) weeks, respectively. Overall, 60% (35/58) to 90% (52/58) of the participants responded positively to all 15 questions assessing acceptability, and no anxiety regarding work was recorded. The mean System Usability Scale score was 66.1 points. Multipara women had significantly longer intervals between app use compared to primipara women (P=.01).

CONCLUSIONS

The results demonstrated acceptable levels of feasibility and usability of the app. However, the low adherence rates, especially among multipara women, suggest the need for modifications to reduce the time burden of the app. Further research should explore more effective and acceptable intervals between use and timing, involving a larger sample and accounting for diverse characteristics of pregnant women.

TRIAL REGISTRATION

UMIN Clinical Trials Registry UMIN000042943; https://tinyurl.com/ydrchfas.

Digital health behaviour change interventions in severe mental illness: a systematic review

The use of digital technologies as a method of delivering health behaviour change (HBC) interventions is rapidly increasing across the general population. However, the role in severe mental illness (SMI) remains overlooked. In this study, we aimed to systematically identify and evaluate all of the existing evidence around digital HBC interventions in people with an SMI. A systematic search of online electronic databases was conducted. Data on adherence, feasibility, and outcomes of studies on digital HBC interventions in SMI were extracted. Our combined search identified 2196 titles and abstracts, of which 1934 remained after removing duplicates. Full-text screening was performed for 107 articles, leaving 36 studies to be included. From these, 14 focused on physical activity and/or cardio-metabolic health, 19 focused on smoking cessation, and three concerned other health behaviours. The outcomes measured varied considerably across studies. Although over 90% of studies measuring behavioural changes reported positive changes in behaviour/attitudes, there were too few studies collecting data on mental health to determine effects on psychiatric outcomes. Digital HBC interventions are acceptable to people with an SMI, and could present a promising option for addressing behavioural health in these populations. Feedback indicated that additional human support may be useful for promoting adherence/engagement, and the content of such interventions may benefit from more tailoring to specific needs. While the literature does not yet allow for conclusions regarding efficacy for mental health, the available evidence to date does support their potential to change behaviour across various domains.

Mobile health supported multi-domain recovery trajectories after major arthroplasty or spine surgery: a pilot feasibility and usability study

BACKGROUND

Recovery after surgery intersects physical, psychological, and social domains. In this study we aim to assess the feasibility and usability of a mobile health application called PositiveTrends to track recovery in these domains amongst participants undergoing hip, knee arthroplasty or spine surgery. Our secondary aim was to generate procedure-specific, recovery trajectories within the pain and medication, psycho-social and patient-reported outcomes domain.

METHODS

Prospective, observational study in participants greater than eighteen years of age. Data was collected prior to and up to one hundred and eighty days after completion of surgery within the three domains using PositiveTrends. Feasibility was assessed using participant response rates from the PositiveTrends app. Usability was assessed quantitatively using the System Usability Scale. Heat maps and effect plots were used to visualize multi-domain recovery trajectories. Generalized linear mixed effects models were used to estimate the change in the outcomes over time.

RESULTS

Forty-two participants were enrolled over a four-month recruitment period. Proportion of app responses was highest for participants who underwent spine surgery (median = 78, range = 36-100), followed by those who underwent knee arthroplasty (median = 72, range = 12-100), and hip arthroplasty (median = 62, range = 12-98). System Usability Scale mean score was 82 ± 16 at 180 days postoperatively. Function improved by 8 and 6.4 points per month after hip and knee arthroplasty, respectively. In spine participants, the Oswestry Disability Index decreased by 1.4 points per month. Mood improved in all three cohorts, however stress levels remained elevated in spine participants. Pain decreased by 0.16 (95% Confidence Interval: 0.13-0.20, p < 0.001), 0.25 (95% CI: 0.21-0.28, p < 0.001) and 0.14 (95% CI: 0.12-0.15, p < 0.001) points per month in hip, knee, and spine cohorts respectively. There was a 10.9-to-40.3-fold increase in the probability of using no medication for each month postoperatively.

CONCLUSIONS

In this study, we demonstrate the feasibility and usability of PositiveTrends, which can map and track multi-domain recovery trajectories after major arthroplasty or spine surgery.

Implementing Smartphone-Based Telemedicine for Cervical Cancer Screening in Uganda: Qualitative Study of Stakeholders' Perceptions

BACKGROUND

In Uganda, cervical cancer (CaCx) is the commonest cancer, accounting for 35.7% of all cancer cases in women. The rates of human papillomavirus vaccination and CaCx screening remain low. Digital health tools and interventions have the potential to improve different aspects of CaCx screening and control in Uganda.

OBJECTIVE

This study aimed to describe stakeholders' perceptions of the telemedicine system we developed to improve CaCx screening in Uganda.

METHODS

We developed and implemented a smartphone-based telemedicine system for capturing and sharing cervical images and other clinical data, as well as an artificial intelligence model for automatic analysis of images. We conducted focus group discussions with health workers at the screening clinics (n=27) and women undergoing screening (n=15) to explore their perceptions of the system. The focus group discussions were supplemented with field observations and an evaluation survey of the health workers on system usability and the overall project.

RESULTS

In general, both patients and health workers had positive opinions about the system. Highlighted benefits included better cervical visualization, the ability to obtain a second opinion, improved communication between nurses and patients (to explain screening findings), improved clinical data management, performance monitoring and feedback, and modernization of screening service. However, there were also some negative perceptions. For example, some health workers felt the system is time-consuming, especially when it had just been introduced, while some patients were apprehensive about cervical image capture and sharing. Finally, commonplace challenges in digital health (eg, lack of interoperability and problems with sustainability) and challenges in cancer screening in general (eg, arduous referrals, inadequate monitoring and quality control) also resurfaced.

CONCLUSIONS

This study demonstrates the feasibility and value of digital health tools in CaCx screening in Uganda, particularly with regard to improving patient experience and the quality of screening services. It also provides examples of potential limitations that must be addressed for successful implementation.

Optimizing usability of a mobile health intervention for Spanish-speaking Latinx people with HIV through user-centered design: a post-implementation study

Objective

Latinx people comprise 30% of all new human immunodeficiency virus (HIV) infections in the United States and face many challenges to accessing and engaging with HIV care. To bridge these gaps in care, a Spanish-language mobile health (mHealth) intervention known as ConexionesPositivas (CP) was adapted from an established English-language platform called PositiveLinks (PL) to help improve engagement in care and reduce viral nonsuppression among its users. We aimed to determine how CP can address the challenges that Latinx people with HIV (PWH) in the United States face.

Materials and methods

We conducted a post-implementation study of the CP mHealth platform, guided by principles of user-centered design. We enrolled 20 Spanish-speaking CP users in the study, who completed the previously validated System Usability Scale (SUS) and semistructured interviews. Interviews were transcribed and translated for analysis. We performed thematic coding of interview transcripts in Dedoose.

Results

The SUS composite score was 75, which is within the range of good usability. Four categories of themes were identified in the interviews: client context, strengths of CP, barriers to use and dislikes, and suggestions to improve CP. Positive impacts included encouraging self-monitoring of medication adherence, mood and stress, connection to professional care, and development of a support system for PWH.

Discussion

While CP is an effective and easy-to-use application, participants expressed a desire for improved personalization and interactivity, which will guide further iteration.

Conclusion

This study highlights the importance of tailoring mHealth interventions to improve equity of access, especially for populations with limited English proficiency.

MASI, a Smartphone App to Improve Treatment Adherence Among South African Adolescents and Young Adults With HIV: Protocol for a Pilot Randomized Controlled Trial

BACKGROUND

Adolescents and young adults with HIV repeatedly demonstrate low rates of antiretroviral therapy (ART) adherence as well as low rates of viral suppression. Digital health interventions are a promising way to engage adolescents and young adults with HIV to support ART adherence. However, few digital health interventions have been developed and tested with adolescents and young adults in countries like South Africa, where the HIV burden among adolescents and young adults is greatest. Masakhane Siphucule Impilo Yethu (MASI; Xhosa for "Let's empower each other and improve our health") is a comprehensive ART adherence-supporting app for South African adolescents and young adults with HIV. It was culturally adapted using the HealthMpowerment platform.

OBJECTIVE

The aim of this paper is to describe the protocol for a pilot randomized controlled trial examining the feasibility, acceptability, and preliminary efficacy of MASI on self-reported ART adherence and social support.

METHODS

We will enroll 50 adolescents and young adults with HIV ages 15-21 years. Participants will be recruited from public ART clinics linked to a large government-funded teaching hospital in Cape Town, South Africa. Participants will be randomized 1:1 into either the intervention arm receiving a full version of MASI or the control arm receiving an information-only version of the app (n=25 per arm). Participants will be asked to engage with MASI daily for 6 months. All participants will complete baseline and follow-up assessments at 3 and 6 months.

RESULTS

Study screening began in May 2022 and the first participant was enrolled on June 21, 2022. As of June 12, 2023, 81 participants have completed screeners, and 36 eligible participants have been enrolled in the pilot randomized controlled trial. Recruitment is anticipated to last through August 31, 2023, with study activities anticipated through February 29, 2024.

CONCLUSIONS

There is an urgent need for innovative interventions to improve ART adherence among adolescents and young adults in settings like South Africa. If found to be feasible and acceptable, MASI could be implemented with adolescents and young adults with HIV in other parts of the country.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04661878; https://clinicaltrials.gov/ct2/show/study/NCT04661878.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/47137.

The feasibility of using a digital tool to enhance mobilisation following abdominal cancer surgery-a non-randomised controlled trial

BACKGROUND

Early mobilisation is strongly recommended following abdominal cancer surgery, but evidence on how to structure early mobilisation to improve outcomes and support patient adherence is lacking. Pedatim® (Phystec) is a novel digital tool designed to support mobilisation in hospital settings using prescribed activities and goals on a tablet. The aim of this study was to evaluate the feasibility of the Pedatim tablet to enhance mobilisation following abdominal cancer surgery.

METHODS

In a non-randomised feasibility trial design, participants were recruited between January and May 2022 at Karolinska University Hospital, Sweden. Participants used a Pedatim tablet from postoperative day 1 (POD 1) until hospital discharge. The primary objective was to evaluate process feasibility, regarding recruitment, compliance, and acceptability. Recruitment was measured by percentage of available patients included, eligibility criteria sufficiency, and number of dropouts. Compliance was measured by number of patients using versus not using the board. Acceptability was measured using the System Usability Scale. The secondary objective was to evaluate scientific feasibility, defined as an indication of treatment effects where physical activity was assessed using an activPAL accelerometer. Unforeseen events relating to the tablet were also registered.

RESULTS

Based on predetermined feasibility criteria, the overall study design was determined to be feasible regarding recruitment as 69% accepted participation (n = 20), compliance was 95%, and the acceptability mean score was high (77/100). Eligibility criteria were not feasible as 79% (n = 108) of available patients were excluded. The intervention was determined to be scientifically feasible, mean steps per day increased from 623 (SD 766) to 1823 (SD 1446), and mean sit-to-stand transitions per day increased from 11 (SD 8) to 29 (SD 12) POD 1-4. Technical issues emerged, highlighting the need for available technical support and "user champions" among healthcare professionals on the ward.

CONCLUSIONS

Using the Pedatim tablet to enhance mobilisation following abdominal cancer surgery was deemed feasible, but a randomised controlled trial is needed to determine the tool's effectiveness. The study process was determined to be feasible with revisions of the eligibility criteria needed before a future trial. Involving healthcare professionals and providing available technical support are important for future implementation.

A Tablet-Based App to Support Nursing Home Staff in Delivering an Individualized Cognitive and Physical Exercise Program for Individuals With Dementia: Mixed Methods Usability Study

BACKGROUND

The promotion of physical activity in individuals with dementia living in nursing homes is crucial for preserving physical and cognitive functions and the associated quality of life. Nevertheless, the implementation of physical activity programs in this setting is challenging, as the time and expertise of nursing home staff are limited. This situation was further exacerbated by the COVID-19 pandemic. Mobile health apps may be a sustainable approach to overcome these challenges in the long term. Therefore, the Individualized Cognitive and Physical Exercise-App (the InCoPE-App) was developed to support nursing home staff in delivering and implementing tailored cognitive and physical exercise training for individuals with dementia.

OBJECTIVE

This study aims to assess the usability of the InCoPE-App in terms of user performance and user perception in a laboratory setting using a mixed methods approach.

METHODS

Nursing home staff were encouraged to perform 5 basic tasks within the InCoPE-App. Their thoughts while using the app were captured by implementing a think aloud protocol. Then, participants completed the System Usability Scale questionnaire. The think aloud transcripts were qualitatively evaluated to unveil usability issues. All identified issues were rated in terms of their necessity to be fixed. Task completion (ie, success rate and time) and perceived usability were evaluated descriptively.

RESULTS

A total of 14 nursing home employees (mean age 53.7, SD 10.6 years; n=13, 93% women) participated in the study. The perceived usability of the InCoPE-App, as assessed by the System Usability Scale questionnaire, can be rated as "good." The main usability issues concerned navigation logic and comprehensibility of app content.

CONCLUSIONS

The InCoPE-App is a user-friendly app that enables nursing home staff to deliver and implement cognitive and physical exercise training for individuals with dementia in nursing homes. The InCoPE-App can be used with little training, even by people aged ≥50 years, who may have low digital literacy. To achieve sustainable use and high user satisfaction of the InCoPE-App in the long term, it should be implemented and evaluated in a field study.

Evaluation of Endo 10 mobile application as diagnostic tool in endodontics

Background

Endodontic diagnosis can be compared to a puzzle, requiring the interpretation of a series of clinical and imaging data. Mobile health, especially mobile application (apps), can assist professionals in endodontic diagnosis. This work aims to evaluate an app - Endo 10 app, designed to assist pulpal and periapical diagnosis based on the patient's signs and symptoms and radiographic data.

Material and Methods

A total of 41 dental students and dentists with different levels of expertise (10 multi-specialty clinic professors, 17 residents in endodontics and 14 dental students) were included. The System Usability Scale (SUS) was used to evaluate usability and the Davis' technology acceptance model was used to evaluate usefulness of Endo 10 app. The Mann-Whitney test was performed to compare SUS scores between professors and undergraduate dental students and to compare questions 6 and 7 of the utility test and verify whether participants who understood that the technology was useful also better understood the concepts of endodontic diagnosis. The agreement between professor's diagnosis with the app and professor without the app, and between professor and residents in endodontics with the app were evaluated.

Results

The SUS score at the 50th percentile was 77.5, graded as acceptable. No significant difference was observed in the SUS scores when analyzing professors and dental students separately (p = 0.442). Usefulness test showed positive responses ranging between 72% - 100%. No statistically significant difference was observed between questions 6 and 7 of the utility test (p = 0.206), indicating that the group of participants who understood that the technology was useful in endodontic diagnosis was associated with the agreement that the application helped to better understand the concepts related. The diagnosis agreement between professor in the common diagnosis process and professor with app was 100% (31) of cases. The concordance between professor and residents in endodontics with the app was 71% (22) of cases. The differences were associated with resident's misinterpreting the patient's data.

Conclusions

The Endo 10 app reached the usability and usefulness requirements. It proved accurate in diagnosing pulpal and periapical pathologies. Key words:Dental education, endodontics, diagnosis, smartphone, dental informatics.

Case study of virtual reality sepsis management- instructional design and ITEM outcomes

This case study focuses on the instructional design and outcomes of a virtual reality (VR) application for sepsis management in healthcare education. The instructional design of the VR sepsis application follows five principles adapted from Merrill's instructional design theory and Bloom's taxonomy. The VR simulation is structured to provide a coherent and realistic experience, with instructional materials and feedback incorporated to guide and support the learners. A pilot study was conducted with medical students on clinical placement. Participants experienced the VR sepsis simulation and completed a questionnaire using the Immersive Technology Evaluation Measure (ITEM) to assess their immersion, intrinsic motivation, cognitive load, system usability, and debrief feedback. Descriptive analysis of the data showed median scores indicating high immersion and presence, intrinsic motivation, and perceived learning. However, participants reported a moderately high cognitive load. Comparison with a neutral response to ITEM suggested that users had a significantly higher user experience (p <  0.05) in all domains. This case study highlights the potential of VR in healthcare education and its application in sepsis management. The findings suggest that the instructional design principles used in the VR application can effectively engage learners and provide a realistic learning experience. Further research and evaluation are necessary to assess the impact of VR on learning outcomes and its integration into healthcare education settings.

Personalized remotely guided preventive exercise therapy for a healthy heart (PRIORITY): protocol for an assessor-blinded, multicenter randomized controlled trial

Aims

A key treatment for patients with varying stages of heart failure with preserved ejection fraction (HFpEF) is exercise. Yet, despite a Class 1A recommendation, only one-third of patients exercise sufficiently. A huge treatment gap exists between guidelines and clinical practice. PRIORITY aims to establish the feasibility, clinical effectiveness and cost-effectiveness of a hybrid centre and home-based personalized exercise and physical activity intervention for patients along the HFpEF continuum.

Methods

An assessor-blinded, multicenter randomized controlled trial will be conducted among 312 patients along the HFpEF continuum. Participants will be randomized (1:1) to the PRIORITY intervention or a comparator group receiving only a written exercise prescription. Participants in the PRIORITY group will receive 18 supervised centre-based exercise sessions during one year, supplemented with a remotely guided home-based physical activity program. Outcomes will be assessed at baseline, 4 months, one and two years. The primary outcome is the peak oxygen uptake (pVO2) at 1-year. Secondary outcomes include physical activity, other physical fitness parameters, cardiovascular health, echocardiographic parameters, health-related quality of life and costs at 1-year FU. Machine learning algorithms will analyse big data on physical activity collected during the 1-year intervention to develop models that can predict physical activity uptake and adherence as well as changes in fitness and health. A cost-utility analysis will be performed to evaluate the cost-effectiveness of the PRIORITY intervention compared to the control condition.

Discussion

We anticipate that participants in the supervised home-based exercise intervention group will have a greater increase in pVO2 compared to those receiving a written exercise prescription.

Trial registration number

This trial is registered at ClinicalTrials.gov (NCT04745013) and is currently in the recruitment stage.

Comprehensive assessment of postoperative mobility during the first days after mini-invasive lung surgery: A prospective observational study

STUDY OBJECTIVE

Postoperative physical therapy and early mobilization are major elements for enhanced recovery after surgery. In contrast with supervised physical therapy sessions that can be monitored, self-mobilization is not easily quantifiable and has so far been estimated mainly through patient auto-reports. This study aimed to perform a comprehensive and objective evaluation of postoperative mobility.

DESIGN

Prospective observational study.

SETTING

Postoperative setting.

PATIENTS

Patients undergoing mini-invasive lung surgery.

INTERVENTIONS

Measurement of postoperative mobility during the first five postoperative days using an accelerometer (ActiGraph GT3X).

MEASUREMENTS

The primary outcome was the number of daily steps. Secondary outcomes included physical activity duration and intensity, sedentary time, number of breaks in sedentary time, sedentary patterns, daily evaluation by physiotherapists, postoperative complications, and acceptability of wearing the accelerometer.

MAIN RESULTS

Sixty patients were included in the study, of whom 56 provided at least one day of valid accelerometry data. There was no significant change during the first four PODs concerning the number of daily steps nor the mean cadence. One-minute cadence peak, total activity counts, and duration of light-intensity physical activity increased over time (p = 0.032, p = 0.001 and p = 0.001, respectively). Sedentary patterns changed favorably over time, with a decrease in prolonged sedentary bouts (≥ 60 consecutive min) (p < 0.001), and an increase in shorter bouts (< 10 min) (p = 0.001). Similar results were observed when analysis was adjusted for the day of the week when the surgery took place. The median acceptability of wearing the accelerometer was excellent (median 10 [9-10] on a 10-point Likert scale). Three patients had major complications.

CONCLUSIONS

Our findings suggest that daily steps may not be the only relevant indicator of early mobility following thoracic surgery and that accelerometry is suitable to follow patients' early postoperative activity.

A usability evaluation of the perceived user friendliness, accessibility, and inclusiveness of a personalized digital care pathway tool

OBJECTIVE

This study aimed to acquire insight into the perceived user friendliness, accessibility and inclusiveness of a personalized digital care pathway.

MATERIALS & METHODS

Usability of the tool was tested in an experimental setting. Mixed methods data collection consisted of scenario-based eye tracking tests in a web- or mobile-based prototype of the tool, followed by a questionnaire assessing user friendliness (System Usability Scale; SUS) and a structured interview. Inclusiveness was assessed by subgroup comparisons based on language proficiency, age and education level. Via purposive sampling a heterogeneous population of users (N = 24) was recruited. Eye tracking was used to measure gaze behavior.

RESULTS

Overall, participants were satisfied with the tool (scale 0-10, 7.5; SD = 1.29). User friendliness of the mobile version (68.3; SD = 21.6) was higher than the web version (50.9; SD = 17.3) measured by SUS score (0-100). With regard to accessibility, eye tracking scenarios showed that the menu bar was hard to find (17% mobile, 55% web). In all scenario's, information was found faster in the mobile version than the web version. Attention was easily drawn to images. Regarding inclusiveness of the tool, we found significantly longer completing time of the scenario tasks for low language proficiency (p-value = 0.029) and higher age subgroups (p-value = 0.049). Lower language proficiency scored a significant lower SUS score (p-value = 0.012).

CONCLUSIONS

Overall, user friendliness and accessibility were positively evaluated. Assessment of inclusiveness emphasized the need for tailoring digital tools to those with low language proficiency and/or an older age. Co-creation of digital care tools with users is therefore important to match users' needs, make tools easily understandable and accessible to all users, and ultimately result in better uptake and impact.

A novel smartphone application for the informal caregivers of cancer patients: Usability study

Informal caregivers are a critical source of support for cancer patients. However, their perspectives are not routinely collected, despite health impacts related to the burden of caregiving. We created the TOGETHERCare smartphone application (app) to collect observer-reported outcomes regarding the cancer patient's health and caregiver's perceptions of their own mental and physical health, and to provide tips and resources for self-care and patient care. We enrolled 54 caregivers between October 2020 and March 2021 from Kaiser Permanente Northern California (KPNC), an integrated healthcare system. Fifty caregivers used the app for approximately 28 days. Usability and acceptability were assessed using questions from the Mobile App Rating Scale (MARS), the System Usability Scale (SUS), the Net Promoter Score (NPS), and semi-structured interviews. The caregivers' mean age was 54.4 years, 38% were female and 36% were non-White. The SUS total mean score was 83.4 (SD = 14.2), for a percentile rank of 90-95 ("excellent"). Median MARS responses to the functionality questions were also high. The NPS score of 30 at the end of the study indicated that most caregivers would recommend the app. Themes from semi-structured interviews were consistent across the study period and indicated that the app was easy to use and helpful. Caregivers indicated a need for feedback from the app, suggested some changes to the wording of questions, the app's visuals, and timing of notifications. This study demonstrated that caregivers are willing to complete frequent surveys about themselves and their patients. The app is unique because it provides a remote method to collect caregivers' observations about the patient that may be useful for clinical care. To our knowledge, TOGETHERCare is the first mobile app developed specifically to capture adult cancer patient symptoms from the informal caregiver's perspective. Future research will examine whether use of this app can help improve patient outcomes.

Evaluating the effect and user satisfaction of an adapted and translated mobile health application ASTHMAXcel© among adults with asthma in Pune, India

OBJECTIVE

ASTHMAXcel^© is a mobile application previously shown to improve asthma knowledge, control, and quality of life. In this study, we translated the application to Marathi for pilot testing in Pune, India in order to evaluate its impact on user satisfaction and asthma knowledge among adult asthma patients.

METHODS

ASTHMAXcel^© was adapted to Marathi with the help of asthma patients and clinicians from Bharati Hospital. 57 different asthma patients were then recruited and received the Asthma Knowledge Questionnaire (AKQ), Asthma Control Questionnaire (ACQ), and Mini Asthma Quality of Life Questionnaire (Mini-AQLQ) to complete at baseline. Study participants then completed the adapted ASTHMAXcel^© application. Post-intervention, participants filled out a post-AKQ and Questionnaire for User Interface Satisfaction (QUIS). A subset of participants was also interviewed for qualitative feedback. Paired t-tests and Pearson's correlation were used for statistical analysis.

RESULTS

Mean AKQ improved from 5.0+/-2.4 to 12.4+/-1.6 (p = 0.0001). QUIS results revealed that participants were highly satisfied with the application, scoring an average of 50 out of 54 maximum points. Better baseline asthma control was correlated with greater overall experience with the application (-0.110, p = 0.0417). Finally, the qualitative feedback revealed four themes for future refinement.

CONCLUSION

The adapted version of ASTHMAXcel^© was linked to significant improvement in patient asthma knowledge and a high level of user satisfaction. These results support the potential utility of mHealth applications in promoting guideline-based asthma care in India. However, further studies are needed to establish a causal relationship between ASTHMAXcel^© and improved clinical outcomes.

Design and Evaluation of Personalized Services to Foster Active Aging: The Experience of Technology Pre-Validation in Italian Pilots

Assistive devices could promote independent living and support the active and healthy aging of an older population; however, several factors can badly influence the long-term use of new technologies. In this context, this paper presents a two-step methodology called “pre-validation” that aims to identify the factors that can bias the use of new services, thus minimizing the risk of an unsuccessful longer trial. The proposed pre-validation methodology is composed of two main phases that aim to assess the usability and the reliability of the technology assessed in a laboratory environment and the usability, acceptability, user experience, and reliability of the technology in real environments. The tested services include the socialization scenario, in which older adults are better connected to the community via technological solutions (i.e., socialization applications), and the monitoring scenario, which allows for the introduction of timely interventions (technologies involved include environmental monitoring sensors, a telepresence robot, wearable sensors, and a personalized dashboard). The obtained results underline an acceptable usability level (average System Usability Scale score > 65) for the tested technologies (i.e., socialization applications and a telepresence robot). Phase Two also underlines the good acceptability, user experience, and usability of the tested services. The statistical analysis underlines a correlation between the stress related to the use of technology, digital skills, and intention of use, among other factors. Qualitative feedback also remarks on a correlation between older adults with low digital skills and an anxiety about using technology. Positive correlation indexes were highlighted between the trust and usability scores. Eventually, future long-term trials with assistive technology should rely on motivated caregivers, be founded on a strong recruitment process, and should reassure older adults—especially the ones with low digital literacy—about the use of technology by proposing personalized training and mentoring, if necessary, to increase the trust.

Mobile-based program improves healthy eating of ulcerative colitis patients: A pilot study

Background

Dietary management plays a crucial role in the treatment of patients with ulcerative colitis (UC). While various e-services provide dietary advice, the long-term dietary management requires continuous monitoring and dynamic adjustment to accommodate the evolving nature of the disease and meet the patients' nutritional needs. Consequently, the development of a novel dietary management tool that incorporates diet tracking, personalized nutritional feedback, and evidence-based advice becomes imperative. This study aims to address this need by developing a WeChat applet called "HealthyGut" specifically designed for the dietary management of UC patients, and evaluate its feasibility, acceptability, and preliminary efficacy.

Methods

A total of 134 UC patients were equally allocated into the intervention group (receiving a 12-week mobile-based dietary management via HealthyGut) and control group (receiving a paper-based food diary and routine advice). The feasibility outcomes were recruitment, retention, engagement, satisfaction, and acceptability in the intervention group. Dietary intakes were effective outcomes.

Results

Both groups had satisfactory retention rates (89.6% and 77.6%, respectively). The System Usability Scale in the intervention group yielded "good usability" with a mean score of 79.63 (SD 7.39), and all participants reported good user experiences and perceived benefits after using HealthyGut. At week 12, intervention responders reported significantly higher daily energy intake than control group (Z = -3.089, p = 0.002).

Conclusions and Implications

The results display that HealthyGut as a dietary management tool is feasible and accepted by UC patients, and it may help them make healthier food choices. Larger sample studies should be considered in the future.

Performance and usability evaluation of a mobile health data capture application in clinical cancer trials follow-up

•

A mobile health data capture application can improve quality of data collection in clinical cancer trials.

•

The use of a mobile health data capture application optimizes the efficiency and effectivity of a trial participant’s visit to a clinic.

•

A mobile health data capture application relieves participants from instant recall on possible health changes during visits.

•

The use of a mobile health data capture application helps a healthcare provider in preparing a participant’s visit.

Mobile health data capture applications (mHDA’s) may improve communication between healthcare providers and patients. However, there is limited literature about the use of mHDA’s facilitating clinical trials. In this study, the effectiveness of an application, supporting follow-up visits of cancer trial participants was investigated. Twenty participants were provided with an e-questionnaire via the mHDA. Participants rated the usability of the application as high performing (mean Systems Usability Scale 87 points). The research team rated the mHDA as highly applicable and efficient in preparing visits. Anamnesis, physical examination and agreement on further policy were performed within an average of 31 min.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments (Preprint)

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.

Usability evaluation and Compatibility test of digital self-management support system for children with cancer and their caregivers: using cloud automation testing platform (Preprint)

BACKGROUND

Despite the increasing development of different smartphone apps in the health care domain, most of these apps lack proper evaluation. In fact, with the rapid development of smartphones and wireless communication infrastructure, many health care systems around the world are using these apps to provide health services for people without sufficient scientific efforts to design, develop, and evaluate them.

OBJECTIVE

The objective of this study was to evaluate the usability of CanSelfMan, a self-management app that provides access to reliable information to improve communication between health care providers and children with cancer and their parents/caregivers, facilitating remote monitoring and promoting medication adherence.

METHODS

We performed debugging and compatibility tests in a simulated environment to identify possible errors. Then, at the end of the 3-week period of using the app, children with cancer and their parents/caregivers filled out the User Experience Questionnaire (UEQ) to evaluate the usability of the CanSelfMan app and their level of user satisfaction.

RESULTS

During the 3 weeks of CanSelfMan use, 270 cases of symptom evaluation and 194 questions were recorded in the system by children and their parents/caregivers and answered by oncologists. After the end of the 3 weeks, 44 users completed the standard UEQ user experience questionnaire. According to the children's evaluations, attractiveness (mean 1.956, SD 0.547) and efficiency (mean 1.934, SD 0.499) achieved the best mean results compared with novelty (mean 1.711, SD 0.481). Parents/caregivers rated efficiency at a mean of 1.880 (SD 0.316) and attractiveness at a mean of 1.853 (SD 0.331). The lowest mean score was reported for novelty (mean 1.670, SD 0.225).

CONCLUSIONS

In this study, we describe the evaluation process of a self-management system to support children with cancer and their families. Based on the feedback and scores obtained from the usability evaluation, it seems that the children and their parents find CanSelfMan to be an interesting and practical idea to provide reliable and updated information on cancer and help them manage the complications of this disease.

Virtual conversational agents versus online forms: Patient experience and preferences for health data collection

Objective

Virtual conversational agents, or chatbots, have emerged as a novel approach to health data collection. However, research on patient perceptions of chatbots in comparison to traditional online forms is sparse. This study aimed to compare and assess the experience of completing a health assessment using a chatbot vs. an online form.

Methods

A counterbalanced, within-subject experimental design was used with participants recruited via Amazon Mechanical Turk (mTurk). Participants completed a standardized health assessment using a chatbot (i.e., Dokbot) and an online form (i.e., REDCap), each followed by usability and experience questionnaires. To address poor data quality and preserve integrity of mTurk responses, we employed a thorough data cleaning process informed by previous literature. Quantitative (descriptive and inferential statistics) and qualitative (thematic analysis and complex coding query) approaches were used for analysis.

Results

A total of 391 participants were recruited, 185 of whom were excluded, resulting in a final sample size of 206 individuals. Most participants (69.9%) preferred the chatbot over the online form. Average Net Promoter Score was higher for the chatbot (NPS = 24) than the online form (NPS = 13) at a statistically significant level. System Usability Scale scores were also higher for the chatbot (i.e. 69.7 vs. 67.7), but this difference was not statistically significant. The chatbot took longer to complete but was perceived as conversational, interactive, and intuitive. The online form received favorable comments for its familiar survey-like interface.

Conclusion

Our findings demonstrate that a chatbot provided superior engagement, intuitiveness, and interactivity despite increased completion time compared to online forms. Knowledge of patient preferences and barriers will inform future design and development of recommendations and best practice for chatbots for healthcare data collection.