System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis

Curating an HIV mHealth App for People Living with HIV in Washington, DC Through Think-Aloud Usability Testing and Beta Testing

This study evaluated the usability of a mobile health application for persons with HIV (PWH) in the District of Columbia (DC) area in preparation for a pragmatic cluster randomized efficacy trial of the PositiveLinks (PL) app. Fourteen participants from the DC Cohort, a multi-site longitudinal prospective HIV cohort study, participated in Think-aloud usability testing. Participants viewed a demonstration version of the PL app and narrated their thoughts, impressions, navigation, and comments as they moved through the app functions. A second set of fourteen DC Cohort participants used the PL app for a one-month period of Beta testing to identify any bugs or glitches, then completed an in-depth interview and System Usability Survey (SUS). A majority of participants identified as male and non-Hispanic Black. Qualitative data from the Think-alouds and Beta testing interviews were audio recorded, transcribed and analyzed in Dedoose using an iterative approach. Think-aloud and Beta testing showed that users viewed the app positively and considered it to be user- friendly. Participants particularly liked the daily reminders and ability to communicate with peers with HIV and their HIV care team. Although most participants reported no dislikes, they requested more customization and more resources. The app achieved a high mean SUS score of 76 and high response rates for app features. No bugs or glitches were identified. The version of PL to be used by participants in the DC Cohort was well liked and highly usable, and participant input informed optimization of the app prior to launching the efficacy trial.

Rheumatic? A diagnostic decision support tool for individuals suspecting rheumatic diseases: Mixed-methods usability and acceptability study

BACKGROUND

The early diagnosis of inflammatory rheumatic diseases (IRDs) is of paramount importance in order to prevent irreversible damage to joints and to optimize treatment outcomes. Nevertheless, conventional care pathways frequently entail diagnostic delays spanning several months. Symptom checkers (SCs) have the potential to provide a solution by offering validated symptom assessments, improving triage systems and expediting diagnostic evaluations. The objective of this mixed-methods study is to assess the usability and acceptability of the SC Rheumatic? among individuals with suspected rheumatic diseases.

METHODS

A total of 105 individuals with suspected IRDs who were newly presenting at an outpatient rheumatology clinic completed the Rheumatic? symptom checker and an evaluation questionnaire. The questionnaire comprised the System Usability Scale (SUS) and Net Promoter Score (NPS). Additionally, 14 participants were interviewed by telephone in order to gain further insights through the qualitative method.

RESULTS

The Rheumatic? symptom checker received a "good" usability score, with an average SUS of 78 ± 16 (range 0-100). Younger participants reported significantly higher usability scores (p < 0.03). However, the NPS was - 15 (range - 100 to 100), indicating lower acceptability. Qualitative data supported the positive usability ratings but emphasized the need for enhancements to increase user engagement and perceived value, such as a current perceived lack of immediate benefit for many users. Their experience varied in terms of impact, with some patients suggesting an increased awareness of their symptoms while others did not notice any difference.

CONCLUSION

Rheumatic? demonstrates good usability, particularly among younger users. Interviews revealed valuable suggestions for improvements, which could enhance overall acceptability and user satisfaction. Implementing Rheumatic? could lead to more efficient triage, potentially reducing diagnostic delays and an optimized allocation of resources. Future iterations should prioritize implementation strategies to maximize user impact and benefit.

CLINICAL TRIAL NUMBER

Not applicable.

The Co-Creation of a Psychosocial Support Website for Advanced Cancer Patients Obtaining a Long-Term Response to Immunotherapy or Targeted Therapy

Due to new treatment options, the number of patients living longer with advanced cancer is rapidly growing. While this is promising, many long-term responders (LTRs) face difficulties adapting to life with cancer due to persistent uncertainty, feeling misunderstood, and insufficient tools to navigate their "new normal". Using the Person-Based Approach, this study developed and evaluated a website in co-creation with LTRs, healthcare professionals, and service providers, offering evidence-based information and tools for LTRs. We identified the key issues (i.e., living with uncertainty, relationships with close others, mourning losses, and adapting to life with cancer) and established the website's main goals: acknowledging and normalizing emotions, difficulties, and challenges LTRs face and providing tailored information and practical tools. The prototype was improved through repeated feedback from a user panel (n = 9). In the evaluation phase (n = 43), 68% of participants rated the website's usability as good or excellent. Interview data indicated that participants experienced recognition through portrait videos and quotes, valued the psycho-education via written text and (animated) videos, and made use of the practical tools (e.g. conversation aid), confirming that the main goals were achieved. Approximately 90% of participants indicated they would recommend the website to other LTRs. The Dutch website-Doorlevenmetkanker (i.e., continuing life with cancer) was officially launched in March 2025 in the Netherlands.

Development and evaluation of an intelligent individualized nutrition management (iNutrition) applet for postdischarge gastric cancer patients following gastrectomy

PURPOSE

This study aimed to develop an intelligent individualized nutrition management (iNutrition) applet for postdischarge gastric cancer patients following gastrectomy based on the Health Action Process Approach (HAPA) theory and using the design thinking method.

METHODS

We developed the iNutrition applet using the design thinking method, including the following phases: (1) "empathize and define," which focused on understanding patients' needs and establishing the goals of the iNutrition applet through qualitative interviews; (2) "ideate and prototype," during which literature reviews and multidisciplinary interactive workshops to conceptualize innovative solutions were conducted and a fully functional iNutrition applet was developed; and (3) "test," during which the iNutrition applet was evaluated through usability testing.

RESULTS

Based on qualitative interviews with 15 patients, we determined that the iNutrition applet would need to satisfy patients' needs in the following areas: adjustment to postoperative anatomical changes, evidence-based knowledge, communication with peers, consultation with medical staff, tailored nutritional plans, coping plans, and needs for technology. Consequently, the applet was designed to have eight functions: gastrointestinal symptom management, nutrition knowledge, medical consultation, peer communication, nutrition goals, nutrition diary with calculator, weekly meal plan, and nutritional status score. During short-term usability testing, the patients' task completion rate ranged from 88 to 100%. Long-term usability testing resulted in a System Usability Scale (SUS) score of 75.20 ± 10.63 and a net promoter score (NPS) of 32%. The participants were generally satisfied with the applet's design and functionality, providing feedback that informed further refinements to the final version of the applet.

CONCLUSION

The iNutrition applet, which has received high ratings for usability and user satisfaction, is an acceptable and feasible method of postdischarge nutritional management for gastric cancer patients following gastrectomy.

Translation, Cross-Cultural Adaptation, and Psychometric Validation of the Health Information Technology Usability Evaluation Scale in China: Instrument Validation Study

BACKGROUND

The dramatic growth of digital health apps highlights an urgent need for rigorous usability evaluation tools. While the Health Information Technology Usability Evaluation Scale (Health-ITUES) has gained validation, a Chinese version has not yet been developed and validated.

OBJECTIVE

This study aimed to translate and culturally adapt the Health-ITUES into Chinese, customize it for both service consumers and professional health care providers, and evaluate its reliability and validity in the Chinese context.

METHODS

Following the Guidelines for the Process of Cross-Cultural Adaptation of Self-report measures, the Health-ITUES was meticulously translated and culturally adapted into Chinese version following 2 rounds of expert consultation. Subsequently, based on the SMART system, an intelligent and integrated older adult care model, the Chinese version of the Health-ITUES was customized into the care receiver version (Health-ITUES-R) and professional health care provider version (Health-ITUES-P). Older individuals and nurses participated in the validation testing conducted between December 2020 and February 2021, facilitated by the improvement of the COVID-19 pandemic and the timing preceding the Spring Festival, which ensured feasible recruitment and a sufficient sample size. In addition, the pandemic-driven increase in digital health app usage allowed us to assess usability in a relevant real-world health care setting. Content validity, internal consistency reliability, construct validity, convergent validity, discriminant validity, and criterion validity were used to evaluate the psychometric attributes of the Health-ITUES-R and Health-ITUES-P.

RESULTS

A Chinese version of the Health-ITUES comprising 20 items across 4 dimensions was formulated, informing the customization of the Health-ITUES-R and Health-ITUES-P. In total, 110 and 124 eligible older adults and nurses validated the customized Health-ITUES-R and Health-ITUES-P, respectively. Both versions exhibited satisfactory content validity (content validity index of items=0.83-1.00; content validity index of scale=0.99) and adequate internal consistency reliability (Cronbach α and McDonald ω>0.80 for the overall scale; >0.75 for individual items). Confirmatory factor analysis confirmed a 4D structure with acceptable construct validity, as indicated by model fit indices. Both the Health-ITUES-R and Health-ITUES-P showed satisfactory convergent validity (average variance extracted [AVE] value>0.5, composite reliability value>0.7), except for a slightly lower AVE value (0.478) for the second dimension of the Health-ITUES-R. Discriminant validity was supported, with the square root of AVE values exceeding correlation coefficients and the Hetereotrait-Monotrait ratio below 0.85. Furthermore, Pearson correlation coefficients for the perceived usefulness dimension, perceived ease of use dimension, and overall scale of the Health-ITUES-R and patient acceptance questionnaire for mobile health application were 0.587, 0.647, and 0.743 (all P<.01), demonstrating a significant correlation.

CONCLUSIONS

The Chinese version of the Health-ITUES can be used as a valid and reliable tool to evaluate the usability of digital health apps for both care receivers and professional health care providers in the Chinese context.

AI4Food, a feasibility study for the implementation of automated devices in the nutritional advice and follow up within a weight loss intervention

BACKGROUND & AIMS

The widespread prevalence of NCDs calls for an improvement in their prevention and treatment. Wearable technologies can be an important asset in the development of precision nutrition strategies, for both health professionals and patients. However, their clinical use is hindered by a lack of validation against current methodologies or appropriate tools to deliver nutritional strategies based on their data. Our study includes manual and automatic data capture methods within a weight loss intervention with the aim to create an essential asset for the implementation, validation, and benchmarking of AI-based tools in nutritional clinical practice.

METHODS

This is a feasibility prospective and crossover controlled trial for weight loss in overweight and obese participants, randomized into two groups: Group 1 used manual data collection methods based on validated questionnaires for the first two weeks; while Group 2 started with automatic data collection methods consisting of wearable sensors. After two weeks, the two groups switched data collection methods. Lifestyle data, anthropometric measurements and biological samples were collected from all participants.

RESULTS

A total of 93 participants completed the nutritional intervention designed for weight loss, achieving a mean reduction of 2 kg (V1: 84.99 SD ± 13.69, V3: 82.72 SD ± 13.32, p < 0.001). Significant reductions were observed in body mass index, visceral fat, waist circumference, total cholesterol, and HbA1c levels. The use of electronic devices proved satisfactory among the participants (System Usability Scale score 78.27 ± 12.86). We also report the presence of distinct patient groups based on continuous glucose measurements.

CONCLUSION

This study has yielded a large amount of data and has showcased how automatic data collection devices can be employed to gather data in the context of a nutritional intervention. This will enable the implementation of AI-based tools in nutritional clinical practice.

CLINICAL TRIAL REGISTRATION

ClinicalTrials.gov, NCT05807243.

A practical guide to usability questionnaires that evaluate clinicians' perceptions of health information technology

OBJECTIVE

Numerous usability questionnaires are available to evaluate the usability of health information technology (IT). It can be difficult for practitioners to determine which questionnaire most closely aligns with their health IT evaluation goals. Our objective was to develop a practical guide to enable practitioners to select an appropriate usability questionnaire for their health IT evaluation.

METHODS

Questionnaires were identified from the literature and input from usability experts. Inclusion criteria included: 1) post-test or post-task usability questionnaire; 2) demonstrated validity, with good internal reliability (Cronbach α ≥ 0.70); 3) freely available for use; 4) applicable across a wide range of health IT products; and 5) demonstrated use with health IT in peer-reviewed literature, even if not originally designed for healthcare.

RESULTS

Criteria were met by seven usability questionnaires. Results include a synopsis of each usability questionnaire along with a matrix to visually compare methodological characteristics across questionnaires. Additionally, results include an analysis of distinguishing methodological strengths and limitations that set each usability questionnaire apart. For each questionnaire, we also outline considerations for use when evaluating health IT.

CONCLUSION

This novel, practical guide provides an important methodological analysis of currently available usability questionnaires for health IT evaluation. This article can help practitioners make a more efficient, but also well-informed, choice when selecting a usability questionnaire for health IT evaluation. This practical, methodological guide applies to a wide range of health IT products, including electronic health records (EHRs).

Development and preliminary testing of a secure large language model-based chatbot for brief alcohol counseling in young adults

OBJECTIVE

Young adults face elevated risks from alcohol use yet encounter significant barriers to accessing evidence-based interventions. Large language models (LLMs) represent a promising advancement for delivering personalized behavioral interventions, but their application to alcohol counseling remains unexplored. This study evaluated the development and preliminary outcomes of a Secure GPT-4-powered text-based Motivational Interviewing Conversational Agent (MICA).

METHOD

Using a prospective single-arm pilot design, we evaluated MICA across two phases (Phase I: n = 8; Phase II: n = 37), editing the LLM prompts between Phases. Participants aged 18-25 who reported consuming ≥ 10 standard alcohol units weekly completed a counseling session with MICA. We evaluated safety and compared MI fidelity (relational and technical sub-scales of the Client Evaluation of MI [CEMI]) and usability (System Usability Scale) between Phases. We also explored surrogate measures of effectiveness (i.e. proportion of change talk to sustain talk from session logs) and qualitative feedback themes.

RESULTS

No unsafe responses were observed. MI fidelity improved significantly in the CEMI relational sub-scale from Phase I to II (67.2 % to 82.6 %, p = 0.03). Usability remained consistently high across phases (Phase I: 85.4; Phase II: 80.9; p = 0.45). The proportion of within-session change talk was also consistently high (Phase I: 65.2 %; Phase II: 75.8 %; p = 0.10).

CONCLUSIONS

This study provides preliminary evidence that LLM-based chatbots can deliver MI-adherent alcohol interventions that are both acceptable to young adults and maintain high MI fidelity. Future research should employ randomized controlled designs with longer follow-up periods to evaluate impact on drinking outcomes.

Tailoring Rheumatoid Arthritis Visit Timing Based on mHealth App Data: Mixed Methods Assessment of Implementation and Usability

Background

Visits to medical subspecialists are common, with follow-up timing often based on heuristics rather than evidence. Unnecessary visits contribute to long wait times for new patients. Specialists could enhance visit timing and reduce frequency by systematically monitoring patients' symptoms between visits, especially for symptom-driven conditions like rheumatoid arthritis (RA). We previously designed an intervention using a mobile health (mHealth) app to collect patient-reported outcomes (PRO). One of several aims of the app was to assist rheumatologists in determining visit timing for patients with RA. The intervention did not reduce visit frequency.

Objective

To explore possible reasons for the lack of association between the intervention and visit frequency, we describe app usage, assess usability, and identify barriers and facilitators for using between-visit PRO data to reduce visits when patients' symptoms are stable.

Methods

We analyzed patients' use of the app by reporting adherence (percent of PRO questionnaires completed during the 12-month study) and retention (use in the last month of the study). To examine rheumatologists' experiences, we summarized views of the electronic health record (EHR)-embedded PRO dashboard and EHR inbox messages suggesting early or deferred visits. We assessed app usability using the interactive mHealth App Usability Questionnaire for Ease of Use and Usefulness for patients and the System Usability Scale for rheumatologists. We assessed rheumatologist-level effects of intervention usage using Kruskal-Wallis rank sum and equality of proportion tests. We identified barriers and facilitators through interviews and surveys.

Results

The analysis included 150 patients with RA and their 11 rheumatologists. Patients answered a median of 53.3% (IQR 34.1%-69.2%) of PRO questionnaires; this proportion varied by rheumatologist (range 40.7%-67%). Over half of the patients used the app during the final month of the study (56%, range 51%-65%, by rheumatologists); the median number of months of use was 12 (IQR 9-12). Rheumatologists viewed the dashboard 78 times (17.6% of 443 visits) with significant differences in viewing rates by rheumatologist (range 10%-66%; P<.01). There were 108 generated messages sent to rheumatologists suggesting a deferred visit (24.4% of 443 visits) with significant differences in message counts received per visit by rheumatologist (range 10.8%-22.6%; P=.03). Rheumatologists' reported barriers to offering visit deferrals included already scheduling as far out as they were comfortable and rescheduling complexities for staff. Based on 39 patient interviews and 44 surveys, patients reported 2 main barriers to app usage: questionnaire frequency not being tailored to them and reduced motivation after not discussing PRO data with their rheumatologist. A total of 5 interviewed patients received the option to defer their visits, of which 3 elected to defer the appointment and 2 chose to keep it.

Conclusions

While an mHealth app for reporting RA PROs was used frequently by patients, using these data to reduce the frequency of unneeded visits was not straightforward. Better engagement of clinicians may improve the use of PRO data.

Development and Systematic Evaluation of a Progressive Web Application for Women With Cardiac Pain: Usability Study

BACKGROUND

Cardiac pain has been widely considered to be the primary indicator of coronary artery disease. The presentation of cardiac pain and associated symptoms vary in women, making it challenging to interpret as cardiac, possibly cardiac, or noncardiac. Women prefer to consult with family and friends instead of seeking immediate medical care.

OBJECTIVE

This study aimed to assess the user performance (ie, ease of use, efficiency, and errors) and user satisfaction (System Usability Scale; SUS) of a progressive web application for women with cardiac pain.

METHODS

Following ethics approval, a purposive sample of women aged >18 years with cardiac pain or associated symptoms lasting >3 months and able to speak and read English was recruited to participate in 2 iterative usability testing cycles. The first cycle assessed the performance of and satisfaction with at heart using a web application, and the second cycle assessed the performance of and satisfaction with at heart across various Android and iOS devices. In total, 2 investigators recorded user comments and documented problems. At the end of the testing session, the participants completed the SUS and 4 semistructured interview questions.

RESULTS

In total, 10 eligible women participated in usability testing from March 31, 2020, to April 17, 2020 (cycle 1), and from November 17, 2020, to November 30, 2020 (cycle 2). Women across usability testing cycles had a mean age of 55.6 (SD 7.3) years, and most (9/10, 90%) were well educated. In total, 50% (5/10) were employed full or part time, and 60% (6/10) earned >CAD $70,000 (US $48,881.80) annually. Participants across 2 testing cycles reported the overall usability of the at heart progressive web application as highly acceptable (mean SUS score 81.75, SD 10.41). In total, 90% (9/10) of participants rated the user-friendliness of at heart as good or excellent. All participants (10/10, 100%) thought at heart was easy to use and efficient. Only 2 testing errors were noted as high priority; these were low contrast or small font and clarification that the chatbot was not a real person. User satisfaction was assessed using themes that emerged from the debrief and 4 semistructured interview questions; at heart was engaging, comprehensive, understandable, credible, relevant, affirming, personalized, and innovative.

CONCLUSIONS

This study provides initial support for the at heart progressive web application for women living with cardiac pain and symptoms. Ongoing evaluations in phases 3 and 4 should aim to examine the feasibility and acceptability of and the extent of engagement with the at heart core feature set: Heart Check, Wellness Check, and the library. In addition to assessing effectiveness in the phase-4 effectiveness-implementation hybrid trial (type I), describing and better understanding the context for implementation (eg, race and ethnicity and geography) will be necessary.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2019-033092.

Acoustic and Natural Language Markers for Bipolar Disorder: A Pilot, mHealth Cross-Sectional Study

Background

Monitoring symptoms of bipolar disorder (BD) is a challenge faced by mental health services. Speech patterns are crucial in assessing the current experiences, emotions, and thought patterns of people with BD. Natural language processing (NLP) and acoustic signal processing may support ongoing BD assessment within a mobile health (mHealth) framework.

Objective

Using both acoustic and NLP-based features from the speech of people with BD, we built an app-based tool and tested its feasibility and performance to remotely assess the individual clinical status.

Methods

We carried out a pilot, observational study, sampling adults diagnosed with BD from the caseload of the Nord Milano Mental Health Trust (Italy) to explore the relationship between selected speech features and symptom severity and to test their potential to remotely assess mental health status. Symptom severity assessment was based on clinician ratings, using the Young Mania Rating Scale (YMRS) and Montgomery-Åsberg Depression Rating Scale (MADRS) for manic and depressive symptoms, respectively. Leveraging a digital health tool embedded in a mobile app, which records and processes speech, participants self-administered verbal performance tasks. Both NLP-based and acoustic features were extracted, testing associations with mood states and exploiting machine learning approaches based on random forest models.

Results

We included 32 subjects (mean [SD] age 49.6 [14.3] years; 50% [16/32] females) with a MADRS median (IQR) score of 13 (21) and a YMRS median (IQR) score of 5 (16). Participants freely managed the digital environment of the app, without perceiving it as intrusive and reporting an acceptable system usability level (average score 73.5, SD 19.7). Small-to-moderate correlations between speech features and symptom severity were uncovered, with sex-based differences in predictive capability. Higher latency time (ρ=0.152), increased silences (ρ=0.416), and vocal perturbations correlated with depressive symptomatology. Pressure of speech based on the mean intraword time (ρ=-0.343) and lower voice instability based on jitter-related parameters (ρ ranging from -0.19 to -0.27) were detected for manic symptoms. However, a higher contribution of NLP-based and conversational features, rather than acoustic features, was uncovered, especially for predictive models for depressive symptom severity (NLP-based: R2=0.25, mean squared error [MSE]=110.07, mean absolute error [MAE]=8.17; acoustics: R2=0.11, MSE=133.75, MAE=8.86; combined: R2=0.16; MSE=118.53, MAE=8.68).

Conclusions

Remotely collected speech patterns, including both linguistic and acoustic features, are associated with symptom severity levels and may help differentiate clinical conditions in individuals with BD during their mood state assessments. In the future, multimodal, smartphone-integrated digital ecological momentary assessments could serve as a powerful tool for clinical purposes, remotely complementing standard, in-person mental health evaluations.

Feasibility and usability of microinteraction ecological momentary assessment using a smartwatch in military personnel with a history of traumatic brain injury

Introduction

Microinteraction Ecological Momentary Assessment (miEMA) addresses the challenges of traditional self-report questionnaires by collecting data in real time. The purpose of this study was to examine the feasibility and usability of employing miEMA using a smartwatch in military service members undergoing traumatic brain injury (TBI) rehabilitation.

Materials and methods

Twenty-eight United States active duty service members with a TBI history were recruited as patients from a military outpatient TBI rehabilitation center, enrolled in either a 2-week or 3-week study arm, and administered miEMA surveys via a custom smartwatch app. The 3-week arm participants were also concurrently receiving cognitive rehabilitation. Select constructs evaluated with miEMA included mood, fatigue, pain, headache, self-efficacy, and cognitive strategy use. Outcome measures of adherence were completion (percentage of questions answered out of questions delivered) and compliance (percentage of questions answered out of questions scheduled). The Mobile Health Application Usability Questionnaire (MAUQ) and System Usability Scale (SUS) assessed participants' perceptions of smartwatch and app usability.

Results

Completion and compliance rates were 80.1% and 77.4%, respectively. Mean participant completion and compliance were 81.1% ± 12.0% and 78.1% ± 13.0%, respectively. Mean participant completion increased to 87.7% ± 8.8% when using an embedded question retry mechanism. Mean participant survey set completion was 69.8% ± 18.3% during the early morning but remained steady during the late morning/early afternoon (85.7% ± 12.8%), afternoon (86.2% ± 12.6%), and late afternoon/evening (85.0% ± 14.7%). The mean overall item score for the MAUQ was 6.3 ± 1.1 out of 7. The mean SUS score was 89.0 ± 7.2 out of 100 and mean SUS percentile ranking was 96.4% ± 8.4%.

Conclusion

Overall adherence was similar to previous studies in civilian populations. Participants rated the miEMA app and smartwatch as having high usability. These findings suggest that miEMA using a smartwatch for tracking symptoms and treatment strategy use is feasible in military service members with a TBI history, including those undergoing rehabilitation for cognitive difficulties.

Development and testing of mobile phone text messages for improving maternal and newborn care practice in Jimma Zone, Ethiopia: a user-centered design approach

BACKGROUND

Mobile phone-based messaging for maternal education has indicated promising outcomes in promoting maternal and newborn care practices. However, most mobile phone-based messaging interventions are not tested before their implementation, and the evidence on the message content development process and methods is limited and underreported. The aim of this study was to develop and test text messages for interventions designed to improve maternal services utilization and newborn care practices.

METHODS

The development process has 4 phases: engaging pregnant mothers and experts in selected districts of the Jimma Zone, Ethiopia. During phase 1, the draft messages were developed based on the message framing concept, and the number of messages was reduced after review. In phase 2, 12 mothers participated in card sorting and the messages were refined. In phase 3, 8 experts evaluated the messages using 10 items with a 5-point Likert scale. In phase 4, a pilot study (n = 30) was conducted to evaluate the messages and their delivery when they were sent to individuals' mobile telephones. In this phase, a system usability scale was also used to assess the usability of the planned intervention.

RESULTS

An initial bank of 120 messages was developed, 42 of which were discarded during the development process. Of these, 32 were discarded by the research team at phase 1, and 10 were discarded during the card sorting at phase 2. Based on the experts' evaluations, 14 messages were reworded and refined in phase 3. During the pilot testing phase, the majority (80%) of the participants mentioned that the time they had been receiving the message was early, and 66.7% of the participants preferred the time to receive text messages about health. The mean SUS score was 72.26, which is acceptable for the usability of the mobile phone-based messaging intervention.

CONCLUSION

A final bank of 78 (39 for each framing type) messages was obtained after refinement according to feedback from participants. The SUS showed an acceptable score for the planned intervention. This message development process confirms that the messages are acceptable, and important for improving maternal and newborn health care practices.

TRIAL REGISTRATION

PACTR202201753436676.

Healthcare practitioners and robotic-assisted rehabilitation: understanding needs and barriers

BACKGROUNDS

In recent years, numerous robotic devices, together with allied technologies, have been developed to support rehabilitation, both in research settings and industry. Although robotic-assisted rehabilitation and related technologies hold significant promise for supporting healthcare practitioners and enhancing patient care, their use in clinical practice remains limited. One of the motivations could be that final users' needs have not been given due consideration so far. As a matter of fact, understanding user needs and perceptions is crucial for designing these technological devices and for creating new organizational models within hospitals aiming to establish and maintain robotics-assisted rehabilitation gyms.

METHODS

We developed and distributed an online survey to the Italian community of healthcare practitioners working in rehabilitation, to depict the current landscape of robotic-assisted rehabilitation and to understand their opinions and demands across various domains and diseases. The questionnaire is divided into two main parts. The first section pertains to the respondents' demographics and professional experience. The second part includes questions about eight different categories of rehabilitative devices. For each category, practitioners can indicate whether they use a device in their practice, their perceptions, and any perceived barriers. Additionally, they can fill out a System Usability Scale for a specific device in that category.

RESULTS

We collected answers from 423 Italian rehabilitation professionals, including various clinical roles, that revealed significant insights into the use of robotics in rehabilitation. Gender distribution shows a high prevalence of female professionals. 40% of respondents reported being unfamiliar with any robotics devices. Advanced treadmills are the most known and used robots. Generally, usage and experience with devices are associated with positive attitudes towards robotics-assisted rehabilitation. Lack of financial resources and scientific evidence, as well as lack of opportunities and training, are the most reported barriers.

CONCLUSIONS

Despite a general positivity towards technology, there is a substantial lack of awareness about rehabilitation devices among professionals. The survey highlights the need for enhanced training and education on robotics in rehabilitation programs. Additionally, the limited focus on home rehabilitation is noted. The study emphasizes the importance of verifying both the effectiveness and economic sustainability of robotic devices in clinical practice.

Development, Validation, and Usability Evaluation of an Application for Health Education in Individuals With Cardiometabolic Diseases

Mobile applications are effective tools for health education in cardiometabolic diseases, enhancing patient's self-care, self-management, self-efficacy, and treatment adherence. This study describes the development, validation, and evaluation of the usability of a mobile application aimed at health education of individuals with cardiometabolic diseases. This is a methodological study developed using the theoretical-methodological framework of Systematic Instructional Design. The validity of the educational technology was assessed using the content validity index, with a threshold of 80% or higher. The application's usability was evaluated through the System Usability Scale. The Viva+ application was developed using Flutter, with Dart programming language and Firebase database. It comprises 49 screens and features functionalities such as registration, login, usage instructions, quizzes, note-taking, and health guidance pages covering topics such as contextualization of cardiometabolic diseases, risk factors, healthy eating, physical activity, weight control, medication management, treatment adherence, smoking and alcohol consumption, sleep improvement, stress management, and professional follow-up. The application was deemed valid by experts, achieving a global validity index of 0.99. Regarding usability, it obtained a total score of 78.07, being considered excellent. The application was shown to be a valid, suitable, comprehensive, and relevant technology for the education and health promotion of individuals with cardiometabolic diseases.

Clinical Validation and Outcome Measures From Bend Ease: A Novel, Sensor-Based Digital Measurement Tool for Assessing At-Home Morning Stiffness and Spinal Range of Motion in Axial Spondyloarthritis

INTRODUCTION

To evaluate the accuracy, reliability, and usability of Bend Ease, a novel smartphone-based digital health technology (DHT), which objectively self-measures spinal range of motion (SRoM) and remotely assesses morning stiffness.

METHODS

This phase 1 study involved healthy volunteers (HV) and patients with axial spondyloarthritis (axSpA). Participants used Bend Ease by placing a phone against their chest during a forward-flexion bend, and the application collected and processed accelerometry data to measure bend angle in both clinical and at-home settings. Bend Ease measurements were compared to the video-based method (gold standard) and functional ability questionnaires.

RESULTS

The study included 30 HV and 30 patients with axSpA. Bend Ease accurately measured forward-flexion bend angles, demonstrating strong correlation (r = 0.74) and concordance (ρc = 0.71) with measurement by video. Impaired bending for patients with axSpA relative to HV was most pronounced upon waking (65.3° versus 88.3°, P < 0.001), with increasing bend angle improvements observed for patients with axSpA at later time points (71.0° and 75.8° at 30 min and 1 h after waking, respectively). Waking bend angle correlated with self-reported morning stiffness and functional ability scores. A minimum clinically important difference in bend angle of 14 degrees was established for patients with axSpA, providing a benchmark for improvement. Bend Ease demonstrated robust test-retest reliability, and participants reported high usability.

CONCLUSIONS

Bend Ease is an accurate, reliable, and user-friendly tool for assessing SRoM. As the first DHT to objectively evaluate morning stiffness upon waking, Bend Ease provides valuable assessments of spinal mobility when it is most impaired.

Acceptability of a Web-Based Health App (PortfolioDiet.app) to Translate a Nutrition Therapy for Cardiovascular Disease in High-Risk Adults: Mixed Methods Randomized Ancillary Pilot Study

BACKGROUND

The Portfolio Diet is a dietary pattern for cardiovascular disease (CVD) risk reduction with 5 key categories including nuts and seeds; plant protein from specific food sources; viscous fiber sources; plant sterols; and plant-derived monounsaturated fatty acid sources. To enhance implementation of the Portfolio Diet, we developed the PortfolioDiet.app, an automated, web-based, multicomponent, patient-facing health app that was developed with psychological theory.

OBJECTIVE

We aimed to evaluate the effect of the PortfolioDiet.app on dietary adherence and its acceptability among adults with a high risk of CVD over 12 weeks.

METHODS

Potential participants with evidence of atherosclerosis and a minimum of one additional CVD risk factor in an ongoing trial were invited to participate in a remote web-based ancillary study by email. Eligible participants were randomized in a 1:1 ratio using a concealed computer-generated allocation sequence to the PortfolioDiet.app group or a control group for 12 weeks. Adherence to the Portfolio Diet was assessed by weighed 7-day diet records at baseline and 12 weeks using the clinical Portfolio Diet Score, ranging from 0 to 25. Acceptability of the app was evaluated using a multifaceted approach, including usability through the System Usability Scale ranging from 0 to 100, with a score >70 being considered acceptable, and a qualitative analysis of open-ended questions using NVivo 12.

RESULTS

In total, 41 participants were invited from the main trial to join the ancillary study by email, of which 15 agreed, and 14 were randomized (8 in the intervention group and 6 in the control group) and completed the ancillary study. At baseline, adherence to the Portfolio Diet was high in both groups with a mean clinical Portfolio Diet Score of 13.2 (SD 3.7; 13.2/25, 53%) and 13.7 (SD 5.8; 13.7/25, 55%) in the app and control groups, respectively. After the 12 weeks, there was a tendency for a mean increase in adherence to the Portfolio Diet by 1.25 (SD 2.8; 1.25/25, 5%) and 0.19 (SD 4.4; 0.19/25, 0.8%) points in the app and control group, respectively, with no difference between groups (P=.62). Participants used the app on average for 18 (SD 14) days per month and rated the app as usable (System Usability Scale of mean 80.9, SD 17.3). Qualitative analyses identified 4 main themes (user engagement, usability, external factors, and added components), which complemented the quantitative data obtained.

CONCLUSIONS

Although adherence was higher for the PortfolioDiet.app group, no difference in adherence was found between the groups in this small ancillary study. However, this study demonstrates that the PortfolioDiet.app is considered usable by high-risk adults and may reinforce dietitian advice to follow the Portfolio Diet when it is a part of a trial for CVD management.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02481466; https://clinicaltrials.gov/study/NCT02481466.

Initial assessment of a novel smoking cessation program integrating app-based behavioral therapy and an electronic cigarette: results of a pilot study

BACKGROUND

Detrimental smoking-related health outcomes warrant the investigation of novel smoking cessation interventions; the cessation program nuumi integrates digital behavioral therapy and an electronic cigarette (EC).

OBJECTIVE

The relationship between program participation and smoking cessation among adults who smoke and are motivated to quit was investigated, as well as program acceptability, changes in smoking-related outcomes, including cigarettes per day (CPD), urges to smoke and psychophysiological health variables (perceived stress, mindfulness, cessation-related self-efficacy, life satisfaction, subjective psychophysiological health) and their associations with smoking cessation.

METHODS

A prospective 6-month single-arm pilot study was conducted; 71 adults who smoked and were motivated to quit received a cognitive behavioral therapy (CBT) app, a closed-system EC, and pods containing decreasing nicotine concentrations. Online surveys were issued at baseline, and at 4, 8, 12, and 24 weeks post-baseline. Intention-to-treat (ITT) and complete-case analyses were conducted to assess self-reported 7-day point prevalence of smoking abstinence (PPA; primary outcome), 30-day PPA, and repeated PPA. T-tests and logistic regressions were used to assess changes in secondary outcomes CPD, urges to smoke, and psychophysiological health variables by smoking status at 12 and 24 weeks, and their relationship with cessation.

RESULTS

Per ITT, self-reported abstinence rates were high at 12 weeks (39.4%), and 24 weeks (32.4%), as was 30-day PPA of 32.4% at both 12 and 24 weeks. Repeated PPA per ITT was 22.5% at both 12 and 24 weeks. Non-abstinent participants significantly reduced their CPD at 12 weeks (t(34) = 6.12, p < 0.001), and at 24 weeks (t(30) = 6.38, p < 0.001). Urges to smoke and perceived stress decreased, and mindfulness, cessation-related self-efficacy, life satisfaction and subjective psychophysiological health increased significantly (all ps < 0.05), predominantly in individuals who reported abstinence. Lower urges to smoke, lower perceived stress, and higher self-efficacy and subjective mental health were related to greater odds of cessation at 24 weeks (all ps < 0.05). Most participants rated the program as highly (43%) or moderately (54%) acceptable.

DISCUSSION

Program participation seems to support cessation and improvements in smoking-related outcomes, but adjustments to the program may be needed to improve engagement and acceptability. Findings may inform the development of future trials and cessation programs.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00032652, registered prospectively 09/15/2023, https://drks.de/search/de/trial/DRKS00032652.

Usability Testing of Five Fentanyl Test Strip Brands in Real-World Settings

BACKGROUND

Fentanyl test strips (FTS) are a forensic tool designed for laboratory testing of urine samples. They have been adapted into a point-of-consumption drug-checking tool to detect illegally manufactured fentanyl in local drug supply. This creates unknown usability challenges for people who use drugs (PWUD). To assess ease of use in real-world settings, we conducted usability testing of five FTS brands.

METHODS

Six researchers independently completed a REDCap survey to evaluate usability of five FTS brands before and after testing a baking soda sample prepared in their homes. The survey first assessed ease of use of specific brand features, then usability of each brand as an interactive system with the System Usability Scale (SUS).

RESULTS

DanceSafe scored highest on ease of use ratings and highest on the SUS (above the usability threshold of ≤68). Dosetest scored lowest on ease of use ratings and below the SUS usability threshold. Respondents identified usability challenges with all brands, including difficulty opening single-strip envelopes, complex instruction design, and instruction discrepancies. Design of all brands assumes users have required testing resources (e.g. water, measuring tools, an Internet connection, and a timer), yet no brand provides these.

CONCLUSIONS

FTS are complex interactive systems not fully adapted for testing in real-world settings. The complexity of FTS may lead to testing error or limit their adoption by PWUD. We encourage manufacturers to design FTS directly for PWUD by considering resource limitations of real-world testing settings, human factors, and local supply complexity.

Acceptability and Preliminary Efficacy of a Novel Web-Based Physical Activity for the Heart (PATH) Intervention Designed to Promote Physical Activity in Adults With Obesity: Protocol for a Pilot Randomized Controlled Trial

BACKGROUND

Even in the absence of weight loss, any level of physical activity (PA) can reduce the risk of cardiovascular disease among individuals with obesity. However, these individuals face multifaceted barriers that reduce their motivation and engagement in PA. They prefer programs that are convenient, fun to engage in, and feature people who they can relate to. Yet, there is a paucity of PA interventions that are designed to incorporate these preferences. We designed the web-based PA for The Heart (PATH) intervention to address this gap.

OBJECTIVE

This study aimed to describe the protocol of a study that aims to examine the acceptability and preliminary efficacy of PATH intervention among insufficiently active adults with obesity aged at least 18 years.

METHODS

This is a 6-month pilot randomized controlled trial (RCT), using a parallel design with 1:1 allocation to intervention or control group. The PATH intervention group is given access to the PATH platform, but the resources each participant can access are tailored according to their baseline fitness level. Control group receives a self-help PA handout. Both groups self-monitor their PA using Fitbit (Google) and have Zoom (Zoom Video Communications) meetings twice a month with either the health coach (intervention) or study coordinator (control). The outcomes at 6-months include acceptability, changes in PA, and cardiometabolic risk from baseline to 6-months.

RESULTS

We screened 763 individuals for eligibility and 89 participants were enrolled and randomized to the intervention (45/504, 50.6%) and control arms (44/504, 49.4%). The average age was 48.7 (SD 12.17) years, and most participants were female (81/504, 90.1%), Black (45/504, 50.6%), and non-Hispanic (83/504, 93.3%). No systematic differences in baseline characteristics were observed between the study arms. The 6-month intervention is currently underway, and the completion of follow-up data collection is expected in February 2025, with results to be published soon after.

CONCLUSIONS

The PATH intervention offers a promising, evidence-based approach to overcoming the barriers that have hindered previous PA programs for adults with obesity. It can support new and existing programs to foster long-term maintenance of health-enhancing PA.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05803304; https://clinicaltrials.gov/study/NCT05803304.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/67972.

Transdiagnostic internet cognitive behavioural therapy for anxiety and depressive symptoms in postnatal women: protocol of a randomized controlled trial

BACKGROUND

Nearly 20% of women will be confronted with anxiety or depressive disorders during the perinatal period and this may lead to adverse outcomes for both mother and child. Cognitive behavioural therapy (CBT) is the psychological intervention with the most empirical support for the clinical management of anxiety and depressive disorders. Anxiety and depression frequently occur in women during the perinatal period, and there is growing evidence that internet-delivered CBT (iCBT) could be an acceptable and effective intervention. THIS WAY UP, an Australian digital mental health service, has developed a program for postnatal anxiety and depression. This study protocol aims to examine the acceptability and efficacy of a French-Canadian adaptation of the program.

METHODS/DESIGN

The research team propose to conduct a mixed hybrid type 1 pragmatic randomized clinical trial and implementation study to replicate the findings of the trial conducted in Australia by Loughnan et al. (2019), as well as explore barriers and facilitators to potential large-scale implementation. TREATMENT AND CONTROL CONDITIONS: a) postnatal anxiety and depression iCBT program with three lessons to complete in a six-week period, added to treatment-as-usual (TAU); b) TAU. Participants will include French-speaking women with probable postnatal depression or anxiety as per the Generalized Anxiety Disorder-7 (GAD-7) or the Edinburgh Postnatal Depression Scale (EPDS). The primary outcome measures will be the GAD-7 and the EPDS. Secondary outcome measures will comprise self-reported instruments to evaluate psychological distress, quality of life, mother-child experience, and treatment experience. Qualitative interviews with participants and health professionals will provide insights on acceptability and delivery of the iCBT program.

STATISTICAL ANALYSIS

Statistical analysis will follow intent-to-treat principles. A mixed model regression approach will be used to account for between- and within-subject variations in the analysis of the effects of iCBT compared to TAU only intervention.

DISCUSSION

The study will generate important data of efficacy and acceptability to patients, clinicians, and decision-makers to inform the scaling-up of the postnatal iCBT intervention in Canada.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT06778096, prospectively registered on 2025/01/16.

Preadolescent Children Using Real-Time Heart Rate During Moderate to Vigorous Physical Activity: A Feasibility Study

BACKGROUND

Given the global burden of insufficient physical activity (PA) in children, effective behavioral interventions are needed to increase PA levels. Novel technologies can help expand the reach and accessibility of these programs. Despite the potential to use heart rate (HR) to target moderate- to vigorous-intensity PA (MVPA), most HR research to date has focused on the accuracy of HR devices or used HR for PA surveillance rather than as an intervention tool. Furthermore, most commercial HR sensors are designed for adults, and their suitability for children is unknown. Further research about the feasibility and usability of commercial HR devices is required to understand how children may use HR during PA.

OBJECTIVE

This study aimed to explore the use of a chest-worn HR sensor paired with a real-time HR display as an intervention tool among preadolescent children and the usability of a custom-designed app (Connexx) for viewing real-time HR.

METHODS

We developed Connexx, an HR information display app with an HR analytics portal to view HR tracking. Children were recruited via flyers distributed at local public schools, word of mouth, and social media posts. Eligible participants were children aged 9 to 12 years who did not have any medical contraindications to MVPA. Participants took part in a single in-person study session where they monitored their own HR using a commercial HR sensor, learned about HR, and engaged in a series of PAs while using the Connexx app to view their real-time HR. We took field note observations about participant interactions with the HR devices. Participants engaged in a semistructured interview about their experience using Connexx and HR during PA and completed the System Usability Scale (SUS) about the Connexx app. Study sessions were audio and video recorded and transcribed verbatim.

RESULTS

A total of 11 participants (n=6, 55% male; n=9, 82%, non-Hispanic White) with an average age of 10.4 (SD 1.0) years were recruited for the study. Data from observations, interviews, and SUS indicated that preadolescent children can use real-time HR information during MVPA. Observational and interview data indicated that the participants were able to understand their HR after a basic lesson and demonstrated the ability to make use of their HR information during PA. Interview and SUS responses demonstrated that the Connexx app was highly usable, despite some accessibility challenges (eg, small display font). Feedback about usability issues has been incorporated into a redesign of the Connexx app, including larger, color-coded fonts for HR information.

CONCLUSIONS

The results of this study indicate that preadolescent children understood their HR data and were able to use it in real time during PA. The findings suggest that future interventions targeting MVPA in this population should test strategies to use HR and HR monitoring as direct program targets.

The Extent to Which Artificial Intelligence Can Help Fulfill Metastatic Breast Cancer Patient Healthcare Needs: A Mixed-Methods Study

The Artificial Intelligence Patient Librarian (AIPL) was designed to meet the psychosocial and supportive care needs of Metastatic Breast Cancer (MBC) patients with HR+/HER2- subtypes. AIPL provides conversational patient education, answers user questions, and offers tailored online resource recommendations. This study, conducted in three phases, assessed AIPL's impact on patients' ability to manage their advanced disease. In Phase 1, educational content was adapted for chatbot delivery, and over 100 credible online resources were annotated using a Convolutional Neural Network (CNN) to drive recommendations. Phase 2 involved 42 participants who completed pre- and post-surveys after using AIPL for two weeks. The surveys measured patient activation using the Patient Activation Measure (PAM) tool and evaluated user experience with the System Usability Scale (SUS). Phase 3 included focus groups to explore user experiences in depth. Of the 42 participants, 36 completed the study, with 10 participating in focus groups. Most participants were aged 40-64. PAM scores showed no significant differences between pre-survey (mean = 59.33, SD = 5.19) and post-survey (mean = 59.22, SD = 6.16), while SUS scores indicated good usability. Thematic analysis revealed four key themes: AIPL offers basic wellness and health guidance, provides limited support for managing relationships, offers limited condition-specific medical information, and is unable to offer hope to patients. Despite showing no impact on the PAM, possibly due to high baseline activation, AIPL demonstrated good usability and met basic information needs, particularly for newly diagnosed MBC patients. Future iterations will incorporate a large language model (LLM) to provide more comprehensive and personalized assistance.

Preliminary investigation of an artificial intelligence-based cognitive behavioral therapy training tool

We developed an asynchronous online cognitive behavioral therapy (CBT) training tool that provides artificial intelligence- (AI-) enabled feedback to learners across eight CBT skills. We sought to evaluate the technical reliability and to ascertain how practitioners would use the tool to inform product iteration and future deployment. We conducted a single-arm 2-week field trial among behavioral health practitioners who treat outpatients with psychosis. Practitioners (N = 21) were invited to use the AI-enabled CBT training tool over a 2-week (15 days, inclusive) period. To enable naturalistic observation, no adjustments were made to their workloads nor were prescriptions on use provided. We conducted daily assessments and collected backend analytics for all users. At end point, we assessed acceptability, appropriateness, feasibility of implementation, perceived usability, satisfaction, and perceived impact of training. We observed four types of technical issues: broken links, intermittent issues receiving AI-enabled feedback, video replay errors, and an HTML error. Participants averaged 6.57 logins over the 2 weeks, with more than half engaging daily. Most participants (44.7%) engaged for < 30-min increments. Usability scores exceeded industry standard and satisfaction scores indicated good promotion of the tool. All participants endorsed high feasibility, acceptability, and appropriateness. Twelve participants (57%) used the AI-enabled feedback feature; those who did tended to report improved satisfaction, feasibility, and perceived impact of the training. The training tool was used by practitioners in a routine care setting, met or exceeded conventional implementation benchmarks, and may support skill improvement; however, data suggest that practitioners may need support or accountability to fully leverage the training tool. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Development and Implementation of Strong Foundations, a Digitally Delivered Fall Prevention Program: Usability and Feasibility Pilot Exercise Cohort Study

Background

Falls remain a major public health problem and a significant cause of preventable injury. Maintaining strength and balance by staying active can prevent falls in older adults, and public health advocates support referral to community exercise programs. Given the growth in use and acceptance of technological interfaces, there remains an interest in understanding the role of a synchronous exercise program designed to improve strength, postural alignment, and balance specifically designed to be delivered in a digital environment with respect to usability and feasibility.

Objective

This study aims to design and implement a synchronously delivered digital fall prevention program to adults aged 60 years and older, to understand the usability, feasibility, and attendance.

Methods

The "Strong Foundations" program, a 12-week, live, digitally delivered fall-prevention exercise program was informed from different existing in-person exercises and piloted to older adults who were considered a low fall risk by scores of 4 or less from the Centers for Disease Control and Prevention's (CDC's) Stopping Elderly Accidents and Deaths Initiative (STEADI) Staying Independent questionnaire. The System Usability Scale (SUS) measured usability and feasibility at the completion of this program, and digital measures of age-related function (timed up and go [TUG] and 30-second chair stand [30 CS]) were collected pre- and postintervention. Data were collected in 2021.

Results

A total of 39 older adults were recruited and 38 completed the 12-week program with an average age of 72 years. The average SUS was 80.6, with an 85% attendance rate and an 8.5 (out of 10) self-reported satisfaction score. Digitally collected TUG and 30 CS statistically improved pre- and postintervention by 9% and 24%, respectively; by week 12, 64% (23/36) of participants improved in the timed up and go and 91% (32/35) improved the chair stands.

Conclusions

There was excellent usability and acceptability for Strong Foundations, a novel fall-prevention program designed to be delivered digitally and promising improvement of objective measures of fall risk.

Personalizing Biologic Therapy in Psoriasis: Development, Validation, and User Testing of a Precision-Dosing Dashboard

An increasing number of individuals receiving psoriasis biologics achieve clear/nearly clear skin (disease control). Clinical trial data indicate that some maintain disease control with lower doses, especially those with higher serum drug concentrations. This indicates the potential of model-informed precision dosing, an advanced therapeutic drug-monitoring technique, to guide dose minimization. We developed, validated, and user-tested a precision dosing dashboard. We applied a model-informed precision-dosing approach that leveraged Bayesian estimation to predict individual pharmacokinetic parameters for personalized dosing recommendations. A pharmacokinetic model of the exemplar biologic risankizumab derived from phases I-III psoriasis trial data (13,123 observations from 1899 patients) was externally validated using real-world data from the United Kingdom. The Bayesian model (posterior prediction: mean absolute error = 0.89 mg/l, mean percentage error = 19.55%, root mean square error = 1.24 mg/l, R2 = 0.86) had superior predictive power to the basic pharmacokinetic model (prior prediction). The model was incorporated into an interactive dashboard that enabled input of individual patient data (serum drug concentrations and model covariates). Healthcare professionals in the United Kingdom rated the dashboard as user friendly and acceptable. The mean time required to generate a dosing interval was 2 minutes. Our dashboard has the potential to incorporate other biologics and extend across disease contexts (nonresponse and other inflammatory diseases) for optimal real-world impact of precision dosing on health and cost outcomes.

How good is your synthetic data? SynthRO, a dashboard to evaluate and benchmark synthetic tabular data

BACKGROUND

The exponential growth in patient data collection by healthcare providers, governments, and private industries is yielding large and diverse datasets that offer new insights into critical medical questions. Leveraging extensive computational resources, Machine Learning and Artificial Intelligence are increasingly utilized to address health-related issues, such as predicting outcomes from Electronic Health Records and detecting patterns in multi-omics data. Despite the proliferation of medical devices based on Artificial Intelligence, data accessibility for research is limited due to privacy concerns. Efforts to de-identify data have met challenges in maintaining effectiveness, particularly with large datasets. As an alternative, synthetic data, that replicate main statistical properties of real patient data, are proposed. However, the lack of standardized evaluation metrics complicates the selection of appropriate synthetic data generation methods. Effective evaluation of synthetic data must consider resemblance, utility and privacy, tailored to specific applications. Despite available metrics, benchmarking efforts remain limited, necessitating further research in this area.

RESULTS

We present SynthRO (Synthetic data Rank and Order), a user-friendly tool for benchmarking health synthetic tabular data across various contexts. SynthRO offers accessible quality evaluation metrics and automated benchmarking, helping users determine the most suitable synthetic data models for specific use cases by prioritizing metrics and providing consistent quantitative scores. Our dashboard is divided into three main sections: (1) Loading Data section, where users can locally upload real and synthetic datasets; (2) Evaluation section, in which several quality assessments are performed by computing different metrics and measures; (3) Benchmarking section, where users can globally compare synthetic datasets based on quality evaluation.

CONCLUSIONS

Synthetic data mitigate concerns about privacy and data accessibility, yet lacks standardized evaluation metrics. SynthRO provides an accessible dashboard helping users select suitable synthetic data models, and it also supports various use cases in healthcare, enhancing prognostic scores and enabling federated learning. SynthRO's accessible GUI and modular structure facilitate effective data evaluation, promoting reliability and fairness. Future developments will include temporal data evaluation, further broadening its applicability.

Development and Implementation of a Personal Virtual Assistant for Patient Engagement and Communication in Postsurgical Cancer Care: Feasibility Cohort Study

Background

Cancer-care complexity heightens communication challenges between health care providers and patients, impacting their treatment adherence. This is especially evident upon hospital discharge in patients undergoing surgical procedures. Digital health tools offer potential solutions to address communication challenges seen in current discharge protocols. We aim to explore the usability and acceptability of an interactive health platform among discharged patients who underwent oncology-related procedures.

Methods

A 4-week exploratory cohort study was conducted. Following hospital discharge, a tablet equipped with an integrated Personal Virtual Assistant (PVA) system was provided to patients who underwent oncology-related procedures. The PVA encompasses automated features that provide personalized care plans, developed through collaboration among clinicians, researchers, and engineers from various disciplines. These plans include guidance on daily specific assignments that were divided into 4 categories: medication intake, exercise, symptom surveys, and postprocedural specific tasks. The aim was to explore the acceptability of the PVA by quantification of dropout rate and assessing adherence to each care plan category throughout the study duration. The secondary aim assessed acceptability of the PVA through a technology acceptance model (TAM) questionnaire that examined ease of use, usefulness, attitude toward use, and privacy concerns.

Results

In total, 17 patients were enrolled. However, 1 (5.8%) patient dropped out from the study after 3 days due to health deterioration, leaving 16/17 (94.2%) completing the study (mean age 54.5, SD 12.7, years; n=9, 52% Caucasian; n=14, 82% with a gynecological disease; n=3, 18% with a hepatobiliary disease). At the study end point, adherence to care plan categories were 78% (SD 25%) for medications, 81% (SD 24%) for exercises, 61% (SD 30%) for surveys, and 58% (SD 44%) for specific tasks such as following step-by step wound care instructions, managing drains, administering injectable medications independently, and performing pelvic baths as instructed. There was an 80% patient endorsement (strongly agree or agree) across all TAM categories.

Conclusions

This study suggests the potential acceptability of the PVA among patients discharged after oncology-related procedures, with a dropout rate of less than 6% and fair-to-good adherence to tasks such as medication intake and exercise. However, these findings are preliminary due to the small sample size and highlight the need for further research with larger cohorts to validate and refine the system.

Protocol for evaluation of iTEST, a novel blended intervention to enhance introspective accuracy in psychotic disorders

Poor introspective accuracy (IA), defined as inaccurate judgments of one's abilities and performance, is a strong and independent predictor of functional impairment in people with psychotic disorders. However, there are currently no treatments that directly target IA in this population as a primary outcome. We describe a protocol for a clinical trial to test a newly developed blended digital intervention, Improving Thinking through Everyday SelfAssessment Training (iTEST), aimed at improving IA in people with psychotic disorders to improve functional outcomes. iTEST involves daily training consisting of feedback on IA in mobile cognitive tests, coupled with individual coaching that applies improved IA to participant-identified recovery goals. Following the NIMH experimental therapeutics paradigm, the first step in the evaluation of iTEST is an open trial in 60 individuals with psychotic disorders to assess 1) feasibility and acceptability, and 2) whether the intervention leads to clinically significant improvement in an objective target: IA on trained tasks along with transfer to an untrained task-based measure of IA. After programming of the mobile intervention and the creation of treatment manuals, enrollment for an open trial started in November 2023 and will be completed by April 2025. If effective, iTEST could be integrated with cognitive training and other rehabilitative interventions to boost the impact on functional outcomes. Trial registration: ClinicalTrials.gov NCT05899348.

Effectiveness of Robot-Assisted Board Games on Cognitive Function and Mental Health for Older Adults with Mild Cognitive Impairment: A Cluster Randomized Trial

The effectiveness of robot-assisted board games targeting older adults with mild cognitive impairment was investigated to improve their cognitive function, general self-efficacy, and life satisfaction and reduce depression. A quasiexperimental research design was adopted with 109 older adults from 8 long-term care facilities and day-care centers assigned to the experimental (n = 52) and comparison groups (n = 57). The experimental group underwent a 12-week cognitive training program. Both groups completed before-and-after and 3-month follow-up measurements for outcome variables, including cognitive function (scores of mini-mental state examination [MMSE] and Alzheimer's Disease Assessment Scale Cognitive Subscale [ADAS-Cog]), depression, general self-efficacy, and life satisfaction. The data were analyzed using the generalized estimating equation (GEE). The program's usability was assessed using the system usability scale (SUS). The GEE analyses revealed significant postintervention improvements in the experimental group's MMSE, ADAS-Cog, depression, general self-efficacy, and satisfaction with life scores. These effects persisted for the 3-month follow-up. The mean SUS score was 87.50, indicating the feasibility of robot-assisted interventions among older adults. These findings confirmed that interactive robot-assisted board games can improve cognitive function, general self-efficacy, and life satisfaction and reduce depression among older adults. The administration of long-term care facilities or day-care centers can adopt robot-assisted board games as a training tool to supplement health promotion activities to prevent cognitive deterioration and enhance mental health among older adults.

Involving End Users in the Development and Usability Testing of a Smartphone App Designed for Individuals With Prediabetes: Mixed-Methods Focus Group Study

Background

Technology is more likely to be used when it is designed to meet the needs of end users. To supplement the Small Steps for Big Changes diabetes prevention program, a smartphone app was developed in partnership with past Small Steps for Big Changes clientele. Usability testing is critical for the ongoing use and adoption of mobile health apps by providing insight on where appropriate adjustments and improvements need to be made to ensure user satisfaction.

Objective

A focus group with 7 participants was conducted to examine the app's usability and collect feedback for future iterations.

Methods

Past Small Steps for Big Changes clientele participated in a cognitive walkthrough of 8 novel tasks and completed the System Usability Scale survey. Participants were then given the option to use the app for 3 weeks before completing the User-Mobile Application Rating Scale.

Results

Analysis of the cognitive walkthrough identified 26 usability problems; each was coded using a heuristic evaluation to describe usability errors. The most frequently coded errors included inappropriate progress feedback, information appearing in an illogical order, counterintuitive design, and issues with app aesthetics. A mean summary score of 66.8% (SD 18.91) was reported for the System Usability Scale, representing a marginal acceptability score and indicating that design issues needed to be resolved. A User-Mobile Application Rating Scale mean score of 3.59 (SD 0.33) was reported, implying an average acceptability rating.

Conclusions

These findings identified necessary improvements in the app, ranging from minor aesthetic problems to major functionality problems. Involving end users allows the app to be tailored to the client's preferences and increases the likelihood of usage. This app aligns with Small Steps for Big Changes' program components and behavior change techniques that can improve health outcomes for future clients and allow them to self-monitor their exercise, diet, and goals.

A Web-Based Resource Informed by Cognitive Behavioral Therapy and Positive Psychology to Address Stress, Negative Affect, and Problematic Alcohol Use: A Usability and Descriptive Study

Background

Research documents that drinking to cope behavior can be disrupted by enhancing emotion regulation and coping skills related to the experience of stress and negative affect. The Alpha Element Self-Coaching Plan incorporates principles of positive psychology and cognitive behavioral therapy to redirect negative thinking and emotions and, therefore, has the potential to benefit individuals who use alcohol to cope with stress.

Objective

This study aimed to evaluate satisfaction and usability of the web-based Alpha Element Self-Coaching Plan in order to inform the development of an expanded digital platform based on the Alpha Element framework.

Methods

Participants enrolled in the web-based program as part of their clinical care were eligible to participate. A total of 20 individuals (14 women and 6 men) between ages 30 and 79 (mean 54.5, SD 14.14) years completed web-based questionnaires to assess product performance in areas such as ease of technology use, quality of videos and handouts, and the value of the activities. Participants also completed the System Usability Scale (SUS) and provided background and demographic information, including alcohol use.

Results

Only 1 participant reported no alcohol use in the past year; 55% (11/20) of participants drank alcohol 2-4 times per month or less and 45% (9/20) reported drinking alcohol 2-3 times per week or more. The average SUS score of 76.38 (SD 17.85) was well above the commonly accepted threshold of 68, indicating high system usability. A majority of the sample (16/19, 84%) agreed or strongly agreed that the activities in the program inspired behavioral changes; and most agreed or strongly agreed that the program was engaging (16/20, 80%), well-organized (18/20, 90%), and easy to follow (17/20, 85%). Only 2 participants endorsed experiencing difficulty using the program on a smartphone. Suggestions for program improvements included expanding the platform, updating the web format, adding user interactivity, and enhancing navigation.

Conclusions

These data suggest that participants were generally satisfied with the web-based Alpha Element Self-Coaching Plan, and rated usability of the program as favorable. Importantly, a significant portion of participants reported that the program inspired behavioral changes. More research is needed with a larger sample to obtain specific data about alcohol consumption and investigate associations between alcohol use and program components, as well as examine gender differences. Data collected from this study will be used to expand the platform and improve user experience.

Multimodal Digital Phenotyping of Behavior in a Neurology Clinic: Development of the Neurobooth Platform and the First Two Years of Data Collection

Quantitative analysis of human behavior is critical for objective characterization of neurological phenotypes, early detection of neurodegenerative diseases, and development of more sensitive measures of disease progression to support clinical trials and translation of new therapies into clinical practice. Sophisticated computational modeling can support these objectives, but requires large, information-rich data sets. This work introduces Neurobooth, a customizable platform for time-synchronized multimodal capture of human behavior. Over a two year period, a Neurobooth implementation integrated into a clinical setting facilitated data collection across multiple behavioral domains from a cohort of 470 individuals (82 controls and 388 with neurologic diseases) who participated in a collective 782 sessions. Visualization of the multimodal time series data demonstrates the presence of rich phenotypic signs across a range of diseases. These data and the open-source platform offer potential for advancing our understanding of neurological diseases and facilitating therapy development, and may be a valuable resource for related fields that study human behavior.

Clinician Experiences With Ambient Scribe Technology to Assist With Documentation Burden and Efficiency

Importance

Timely evaluation of ambient scribing technology is warranted to assess whether this technology can lessen the burden of clinical documentation on clinicians.

Objective

To investigate the association of ambient scribing technology with efficiency, quality, and perceived burden of clinical documentation in the outpatient setting.

Design, Setting, and Participants

This prospective, single-group pre-post quality improvement study was conducted between April and June 2024 in the outpatient setting of an academic health system in Philadelphia, Pennsylvania. Participants included physicians, nurse practitioners, and physician assistants. Data were analyzed from July to August 2024.

Exposure

Access to an artificial intelligence-driven ambient scribing tool for outpatient notes.

Main Outcomes and Measures

The primary outcomes were time in notes per appointment, same-day appointment closure, after-hours work time, perceived burden of clinical documentation, and comments on clinicians' experiences using ambient scribing. A mixed-effects model was used. Both objective metrics and survey feedback were obtained. Targeted perspective questions, designed to assess clinician-perceived patient engagement and perceived documentation burden, were assessed on a 7-point Likert scale. Standard System Usability Scale (SUS) and net promoter score (NPS) formulas were used to analyze usability and recommendability data.

Results

This study included 46 clinicians from 17 different medical specialties, with a mean (SD) of 11.1 (8.7) years in practice. From baseline to post intervention, use of the ambient scribing tool was associated with 20.4% less time in notes per appointment (from 10.3 to 8.2 minutes; P < .001), 9.3% greater same-day appointment closure (from 66.2% to 72.4%; P <.001), and 30.0% less after-hours work time per workday (from 50.6 to 35.4 minutes per workday; P = .02). Targeted perspective questions showed more favorable scores. Open-ended qualitative feedback from clinicians showed a range of positive, negative, and mixed feedback regarding their use of ambient scribing technology. SUS scores showed that the ambient scribing tool was easy to use. NPSs reflected the mixed qualitative feedback.

Conclusions and Relevance

In this quality improvement study, the use of ambient scribe technology was associated with greater efficiency of outpatient clinical documentation, lower mental burden of documentation for clinicians, and greater sense of engagement with patients during outpatient appointments. Additional studies exploring urgent care settings, examining patient experience, and comparing multiple tools will be important to better understand the effect of ambient scribing on ambulatory care.

Parent-Reported Usability of a Patient Portal-Based Asthma Care Tool for Parents of Children With Asthma

INTRODUCTION

This study evaluates our new EHR-integrated patient portal for asthma care (PAC) management module for parents of children with asthma. The module includes a previsit asthma intake questionnaire via the portal. The parent answers are integrated into the provider's clinic progress note to support clinical decision-making. Our goals were to measure the functionality and usability of the PAC module and to understand facilitators and barriers to its use for parents.

METHODS

Parents of children ages 0-11 years old (n = 45) completed the PAC module's asthma intake questionnaires prior to their upcoming pediatric pulmonology clinic visit. To assess functionality, provider progress notes were manually reviewed to measure the amount of key asthma-related data captured. Differences in percent data captured with and without the PAC module were compared. Electronic surveys capture demographics, usability data (the System Usability Scale [SUS]), and open-ended experiential feedback about the module. Analysis included descriptive statistics for demographics and usability, as well as the constant comparative method for open-ended feedback.

RESULTS

The PAC module at this early stage of design significantly improved the capture of key asthma data in physician notes, increasing from 77% to 92% (p < 0.001). The average SUS score (83.8) indicated high usability. Favorable aspects of the module that were identified included time savings and ease of use.

CONCLUSION

Our PAC module enhanced data capture of key asthma management elements and demonstrated high parental usability. We will continue to refine the module through an iterative approach based on end-user feedback, with future expansion planned for broader patient populations.

Examining the potential of an internet-based emotion regulation intervention added to acute psychiatric inpatient care: results from a randomized controlled pilot trial

The feasibility and the preliminary effectiveness of an internet-based emotion regulation intervention added to acute psychiatric inpatient care were assessed with a randomized controlled pilot trial. Sixty patients were allocated in a 1:1 ratio to the intervention group or treatment as usual (TAU). Feasibility was evaluated via patient satisfaction, system usability, and program usage. The primary outcome was symptom severity (Brief-Syptom-Inventory-18-GSI), secondary outcomes included two emotion regulation measures. Assessments occurred at baseline, after four weeks (T1), eight weeks (T2), and patient discharge. Satisfaction scores (CSQ-8: M = 2.97, SD = 0.64) and usability ratings (SUS: M = 69.89, SD = 10.34) were positive. Program usage was low, with only 12 of 30 patients completing at least 50 % of the program. In the ITT-analysis, no significant group-by-time interaction effects were found for symptom severity or emotion regulation. Descriptively, effect sizes favored the intervention for symptom severity at T1 (d = 0.16) and T2 (d = 0.12) and favored TAU for the emotion regulation parameters at both time points. Thus, while the internet-based program showed good usability and satisfaction, it did not significantly impact symptom severity or emotion regulation. Future research should examine how to enhance program use in this treatment context.

Digital Home-Based Self-Monitoring System for People with Heart Failure: Protocol for Development of SmartHeart and Evaluation of Feasibility and Acceptability

BACKGROUND

Heart failure (HF) is a chronic, progressive condition where the heart cannot pump enough blood to meet the body's needs. In addition to the daily challenges that HF poses, acute exacerbations can lead to costly hospitalizations and increased mortality. High health care costs and the burden of HF have led to the emerging application of new technologies to support people living with HF to stay well while living in the community. However, many digital solutions have not involved consumers and health care professionals in their design, leading to poor adoption. The SmartHeart project aimed to codevelop a smart health ecosystem to support the early detection of HF deterioration and encourage self-care, potentially preventing hospitalizations.

OBJECTIVE

This study aims to provide an overview of the SmartHeart project by describing our approach to designing the SmartHeart system, outlining its features, and describing the planned pilot study to determine the feasibility of the system.

METHODS

We used the Integrate, Design, Assess, and Share (IDEAS) framework to guide the development of the SmartHeart system, involving users (people with HF and their caregivers) and stakeholders (health care providers involved in the management of HF) in its design. SmartHeart is a complete remote heart health monitoring and automated feedback delivery system. It includes 2 user interfaces for patients: an Amazon Alexa conversational agent and a smartphone app. The system collects physiological, symptom, and behavioral data through wireless sensors and self-reports from users. These data are processed and analyzed to provide personalized health insights, self-care support, and alerts in case of health deterioration. The system also includes a web-based user interface for health care professionals, allowing them to access data, send messages to users, and receive notifications about potential health deterioration. A single-arm, multicenter pilot trial (N=20) is planned to determine the feasibility and acceptability of SmartHeart before evaluation through a randomized controlled trial. The primary outcome will be a description of the study's feasibility (recruitment, attrition, engagement, and changes in self-care).

RESULTS

The SmartHeart study started in January 2021 on procurement of funding. Recruitment for the pilot trial started in August 2024 and will be completed by March 2025. We have currently enrolled 12 participants. Follow-up of all participants will be completed by the end of May 2025.

CONCLUSIONS

We have co-designed and developed a complete remote heart health monitoring and automated feedback delivery system for the early detection of HF deterioration and prevention of HF-related hospitalizations. The next step is a pilot study, which will provide valuable information on feasibility and preliminary effects to inform a larger evaluation trial. SmartHeart has the potential to augment existing health services and help people with HF stay well while living in the community.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/62964.

Mobile Phone App to Promote Lifestyle Change in People at Risk of Type 2 Diabetes: Feasibility 3-Arm Randomized Controlled Trial

Background

The use of mobile health interventions, such as apps, are proposed to meet the challenges faced by preventive health care services due to the increasing prevalence of type 2 diabetes (T2D). Thus, we developed and conducted initial feasibility testing of the Plunde app for promoting and monitoring individual goals related to lifestyle change for people at risk of T2D.

Objective

The primary aim of this study was to assess the feasibility of an app for promoting lifestyle change in people at risk of T2D. The secondary aim was to assess recruitment rate, resource requirements, and change in potential outcomes for a full scale randomized controlled trial (RCT) study .

Methods

A 3-arm feasibility RCT lasting 12 weeks was designed. Participants were recruited from 9 general practitioners in Norway. Eligible participants were randomized to either (1) app follow-up; (2) app follow-up and referral to care as usual in Healthy Life Centers; or (3) referral to care as usual in a Healthy Life Center, only. The primary outcome was feasibility and was measured by app adherence (actual usage of the app), the System Usability Scale, and app motivation score gained from a questionnaire designed for this study. Criteria for success were preset based on these measures. Secondary outcomes included recruitment rate, resource requirements, and potential primary outcomes of a full-scale RCT. This included change in body weight, waist circumference, and self-evaluated functional health status, assessed with the Dartmouth Primary Care Cooperative Research Network/World Organization of Family Doctors (COOP/WONCA) functional health assessment chart.

Results

Within 8 months, 9 general practitioners recruited a total of 54 participants, of which 45 were eligble for participation in the study. Mean age was 61 (SD 13) years and 53% (n=24) were female. App adherence was 86%, the mean System Usability Scale score was 87.3 (SD 11.9), and the mean app motivation score was 74.8 (SD 30.3). Throughout the intervention period, health care professionals spent on average 3.0 (SD 1.0) minutes per participant per week providing follow-up. Statistically significant reduction in body weight and waist circumference was shown in group 1 and 3.

Conclusions

Based on the preset criteria for success, the Plunde app is feasible in providing support for lifestyle change. The Plunde app had excellent user satisfaction. The amount of time spent on monitoring and promoting lifestyle change through the app was low; however, the recruitment was slow. Results from this study will guide the development of further research within this field.

Developing and evaluating a gamified self-management application for inflammatory bowel disease using the ADDIE model and Sukr framework

BACKGROUND

The prevalence and chronic nature of Inflammatory Bowel Diseases (IBD) is a significant global concern. As the essential part of treatments approach, patient adherence to treatment protocols and self-management practices are crucial to = IBD management. Healthcare initiatives focused on chronic conditions are strongly needed to consider various aspects of gamification and how it can positively affect self-management.

AIM

The current cognitive study aims to develop a mobile application to integrate the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) instructional design model and elaborate on a gamification design based on the reputable Sukr Wheel framework.

METHODS

The current study uses the ADDIE approach to integrate behavior change strategies derived from the self-management theory using the Sukr Wheel gamification (My IBD Buddy) framework on the Android platform.

RESULTS

The final evaluation was conducted over 14 days. User satisfaction comprised 22 participants aged 20 to 64, all diagnosed with inflammatory bowel diseases. System usability was measured on a scale ranging from 50 to 100. The average usability score for the entire user group was 80.68, indicating a "good" level of satisfaction among the program users based on the ranking scale.

CONCLUSION

"My IBD Buddy" mobile application, equipped with gamification for IBD patients, enhances self-efficacy and self-management.

Immersive virtual reality to assess unilateral spatial neglect in stroke patients: a preliminary study

OBJECTIVES

The conventional test to detect unilateral spatial neglect (USN) is the Bells Test performed in a paper-and-pencil format. While several studies showed immersive virtual reality (VR) tests may provide greater sensitivity in revealing the presence of USN using visual scanning tasks, none has investigated the Bells Test in VR. This study compares the Bells Test performed in paper-and-pencil format (PP) and in VR in conventional (CVR) and ecological (EVR) format, which differ by the size of the display, in stroke patients.

DESIGN

Cross-sectional study.

SETTING

Stroke patients.

PARTICIPANTS

A convenience sample of 32 stroke patients.

INTERVENTIONS

VR assessments were performed using an immersive system with a head-mounted display. In CVR, the Bells Test is reproduced in the same format as PP (A4 sheet), while in EVR, the targets are displayed in a wider space corresponding to a hemisphere of 1-m radius.

RESULTS

The number of cancelled targets out of 35 was 32.5 (3.5) for PP, 33 (4) for CVR, and 34 (2) for EVR (mean [SD]), with a significant difference between PP and EVR (p < 0.05). The time to complete the Bells Test was 186 (69) s for PP, 184 (65) s for CVR, and 170 (58) s for EVR, without differences between modalities (p > 0.05). Bells Tests in the 3 modalities revealed the presence of USN, except for 1 patient in EVR.

CONCLUSION

VR assessment of USN could be used in the same way as conventional cancellations tests. Moreover, VR could provide additional information on the type of USN through the different testing modalities available.

A pilot study of trauma-sensitive yoga and Breathe2Relax among service members in an intensive outpatient program

Emerging research indicates that yoga is a promising adjunct to psychological trauma treatment. The current pilot study examined the associations between psychophysiological stress, diaphragmatic breathing (DB), and a trauma-sensitive yoga (TSY) regimen developed specifically for trauma-exposed service members in alignment with recent calls for precision in reporting therapeutic yoga protocols. Participants were 31 service members enrolled in a trauma-focused intensive outpatient program (IOP). Service members participated in a brief diaphragmatic breathing (DB) session using the Breathe2Relax (B2R) app followed by the TSY session. Heart rate (HR) and perceived stress were measured at baseline and after both the DB practice and the TSY session. We assessed Yoga and DB expectancies at baseline and post TSY. Participants also rated the acceptability and usability of the B2R app. Results of linear mixed effects regression models showed decreases in HR and perceived stress, compared to baseline, following DB (HR, b = -8.68, CI 95% = -13.34, -4.02; perceived stress, b = -1.77, CI 95% = -2.35, -1.18) and TSY (HR, b = -12.44, CI 95% = -17.15, -7.73; perceived stress b = -3.69, CI 95% = -4.29, -3.08). Higher levels of expectancies, compared to lower levels, related to stronger decreases in HR and perceived stress, particularly after TSY. Overall, participants rated the B2R usability as high; virtually all participants reported that "most would learn to use the app quickly," and 76.6% reported that they would use it frequently.

Feasibility of a computerized clinical decision support system delivered via a socially assistive robot during grand rounds: A pilot study

Aims and Objective

The aim of this study was to explore the feasibility, usability and acceptance of integrating Clinical Decision Support Systems with Socially Assistive Robots into hospital grand rounds.

Background

Adopting Clinical Decision Support Systems in healthcare faces challenges such as complexity, poor integration with workflows, and concerns about data privacy and quality. Issues such as too many alerts, confusing errors, and difficulty using the technology in front of patients make adoption challenging and prevent it from fitting into daily workflows. Making Clinical Decision Support System simple, intuitive and user-friendly is essential to enable its use in daily practice to improve patient care and decision-making.

Methods

This six-month pilot study had two participant groups, with total of 40 participants: a longitudinal intervention group (n = 8) and a single-session evaluation group (n = 32). Participants were medical doctors at the University Clinical Center Maribor. The intervention involved implementing a Clinical Decision Support System delivered via a Socially Assistive Robot during hospital grand rounds. We developed a system that employed the HL7 FHIR standard for integrating data from hospital monitors, electronic health records, and patient-reported outcomes into a single dashboard. A Pepper-based SAR provided patient specific recommendations through a voice and SAR tablet enabled interface. Key evaluation metrics were assessed using the System Usability Scale (SUS) and the Unified Theory of Acceptance, Use of Technology (UTAUT2) questionnaire, including Effort Expectancy, Performance Expectancy and open ended questions. The longitudinal group used the system for 6 months and completed the assessments twice, after one week and at the end of the study. The single-session group completed the assessment once, immediately after the experiment. Qualitative data were gathered through open-ended questions. Data analysis included descriptive statistics, paired t-tests, and thematic analysis.

Results

System usability was rated highly across both groups, with the longitudinal group reporting consistently excellent scores (M = 82.08 at final evaluation) compared to the acceptable scores of the single-session group (M = 68.96). Extended exposure improved user engagement, reflected in significant increases in Effort Expectancy and Habit over time. Participants found the system enjoyable to use, and while no significant changes were seen in Performance Expectancy, feedback emphasized its efficiency in saving time and improving access to clinical data, supporting its feasibility and acceptability.

Conclusions

This research supports the potential of robotic technologies to transform CDSS into more interactive, efficient, and user-friendly tools for healthcare professionals. The paper also suggests further research directions and technical improvements to maximize the impact of innovative technologies in healthcare.

Co-design and usability of an interactive web-based fertility decision aid for transgender youth and young adults

OBJECTIVE

To develop a patient- and family-centered Aid For Fertility-Related Medical Decisions (AFFRMED) interactive website targeted for transgender and nonbinary (TNB) youth/young adults and their parents to facilitate shared decision-making about fertility preservation interventions through user-centered participatory design.

METHOD

TNB youth/young adults interested in or currently receiving pubertal suppression or gender-affirming hormone treatment and parents of eligible TNB youth/young adults were recruited to participate in a series of iterative human-centered co-design sessions to develop an initial AFFRMED prototype. Subsequently, TNB youth/young adults and parents of TNB youth/young adults were recruited for usability testing interviews, involving measures of usability (i.e., After Scenario Questionnaire, Net Promotor Score, System Usability Scale).

RESULTS

Twenty-seven participants completed 18 iterative co-design sessions and provided feedback on 10 versions of AFFRMED (16 TNB youth/young adults and 11 parents). Nine TNB youth/young adults and six parents completed individual usability testing interviews. Overall, participants rated AFFRMED highly on measures of acceptability, appropriateness, usability, and satisfaction. However, scores varied by treatment cohort, with TNB youth interested in or currently receiving pubertal suppression treatment reporting the lowest usability scores.

CONCLUSIONS

We co-created a youth- and family-centered fertility decision aid prototype that provides education and decision support in an online, interactive format. Future directions include testing the efficacy of the decision aid in improving fertility and fertility preservation knowledge, decisional self-efficacy, and decision satisfaction.

Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study

Introduction

There is a lack of comprehensive clinical research to assess potential benefits of mHealth solutions in post discharge follow-up care after spinal surgery.

Purpose

This quasi-experimental study evaluated the effectiveness, usability, and patient satisfaction of an mHealth pathway with an electronic Patient-Reported Outcome (ePRO)-based post-discharge nurse-led intervention for patients undergoing surgery for lumbar spine degenerative disorders, compared to standard care.

Methods

Conducted at a Danish tertiary spine center, this study represents the final stage of a three-phase participatory design. The primary outcome was patient quality of recovery, measured by the Quality of Recovery-15 (QoR-15) questionnaire. Secondary outcomes included patient-perceived usability, assessed with the Danish System Usability Scale (SUS). To capture additional patient insights, an open-ended feedback question was included at the end of the survey.

Results

Data from 150 patients (77 women and 73 men) were analyzed, with 104 in the intervention group and 46 in the comparison group. Both groups showed significant improvement over time, but no significant difference between groups. Of 154 potential SUS respondents, 110 participated. Analysis revealed 48 statements, categorized into five themes: (1) Usability and functionality of the mHealth solution, (2) Feedback on the QoR-15 questionnaire, (3) Safety and support, (4) Missing functions and suggestions for improvements, and (5) Patient satisfaction.

Conclusion

No significant differences in effectiveness between the mHealth pathway and standard care were found for post-surgery recovery in lumbar spine patients. Usability and patient satisfaction were generally high, though suggestions for improvements were noted. The study highlighted the importance of post-discharge support, suggesting that the intervention provided crucial security and support, potentially demonstrating compassion through nurse-led care.

Enhancing accessibility and scalability of school-based programs to improve youth attention and behavior: Open feasibility trial of the remote CLS-R-FUERTE program in Mexico

Neurodevelopmental disorders of inattention and disruptive behavior, such as attention-deficit/hyperactivity disorder and oppositional defiant disorder, are among the most common youth mental health conditions across cultures. There is a need to develop more accessible school-based intervention and training programs, as well as create a system with clinical research capacity for scalable school clinician training and evaluation, to support students with attention and behavior concerns worldwide. We adapted the collaborative life skills program for Mexico (i.e., CLS-FUERTE) for remote delivery (i.e., CLS-R-FUERTE) and conducted a three-school open trial with N = 67 participants (n = 7-8 students per school [ages 6-12] and their parents, teachers, and school clinicians). We examined fidelity to program content, attendance and adherence records, in vivo observations of program delivery, and postmeeting feedback informing iterative program changes between each school cohort. We also examined improvements in youth attention and behavior rated by parents and teachers to evaluate the remote program effectiveness. CLS-R-FUERTE feasibility, acceptability, and usability findings were promising. Iterative program changes between each school cohort were minor and included adapted curriculum order, enhanced engagement strategies, and technology adjustments. Many students demonstrated reliable change, and the pre-post program improvements were comparable to outcomes from the in-person CLS-FUERTE trial, indicating preliminary effectiveness. Our pilot CLS-R-FUERTE effort supports the process of iteratively adapting, implementing, and evaluating remote school-based intervention and training programs to enhance potential flexibility, accessibility, and scalability. Challenges emerging from technological problems and in context of the COVID-19 pandemic, as well as solutions, are discussed. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

ADL+: A Digital Toolkit for Multidomain Cognitive, Physical, and Nutritional Interventions to Prevent Cognitive Decline in Community-Dwelling Older Adults

BACKGROUND

Current research highlights the importance of addressing multiple risk factors concurrently to tackle the complex etiology of dementia. However, limited evidence exists on the efficacy of technology-driven, multidomain community-based interventions for preventing cognitive decline.

OBJECTIVES

To evaluate the efficacy of ADL+, an artificial intelligence (AI)-enabled digital toolkit integrating cognitive assessments and multidomain interventions, on outcomes of cognitive function, activity levels, and quality of life in older adults at risk of cognitive decline. Adherence and usability were also evaluated.

METHODS

We conducted a quasi-experimental study including community-dwelling older adults aged 60 years and above without dementia, but with subjective memory complaints (AD8 score ≥ 2). Participants received a six-month intervention (app-based cognitive training, personalized nutritional, physical, and social activities recommendations) or a control group treatment (cognitive health educational package). The primary outcome was a change in neuropsychological test battery (NTB) Z-scores (NTB composite and its individual domains: attention, processing speed, memory, and executive function). Secondary outcomes were activity levels (Frenchay Activities Index, FAI), and quality of life (EQ-5D). Outcomes were assessed at the end of the intervention and three months post-intervention using linear mixed-effects models.

RESULTS

96% of participants in the intervention and 89% in the control group completed the study. At six months, the intervention group showed a significant NTB composite score improvement (mean change: 0.086 (95% CI 0.020 to 0.15)), with a between-group difference of 0.17 (95% CI 0.071 to 0.27). Significant differences in attention, processing speed, and memory domains were observed, with benefits sustained in the processing speed domain at nine months. The control group's FAI scores declined at six months (mean change: -1.04 (95% CI -1.83 to -0.26)), while the intervention group's scores remained stable. The intervention group's EQ-5D visual analogue scale (VAS) scores improved at both six and nine months, with between-group differences of 4.06 (95% CI 0.23 to 7.90) at six months and 5.12 (95% CI 0.81 to 9.43) at nine months. Adherence was high, while average usability scores were obtained.

CONCLUSIONS

The ADL+ toolkit shows potential beneficial effects on cognitive function, activity levels, and quality of life for older adults at risk of cognitive decline. Findings will guide future randomized controlled trials and implementation efforts.

Optimization of a Web-Based Self-Assessment Tool for Preconception Health in People of Reproductive Age in Australia: User Feedback and User-Experience Testing Study

Background

Good preconception health reduces the incidence of preventable morbidity and mortality for women, their babies, and future generations. In Australia, there is a need to increase health literacy and awareness about the importance of good preconception health. Digital health tools are a possible enabler to increase this awareness at a population level. The Healthy Conception Tool (HCT) is an existing web-based, preconception health self-assessment tool, that has been developed by academics and clinicians.

Objective

This study aims to optimize the HCT and to seek user feedback to increase the engagement and impact of the tool.

Methods

In-depth interviews were held with women and men aged 18-41 years, who spoke and read English and were residing in Australia. Interview transcripts were analyzed, and findings were used to inform an enhanced HCT prototype. This prototype underwent user-experience testing and feedback from users to inform a final round of design changes to the tool.

Results

A total of 20 women and 5 men were interviewed; all wanted a tool that was quick and easy to use with personalized results. Almost all participants were unfamiliar with the term "preconception care" and stated they would not have found this tool on the internet with its current title. User-experience testing with 6 women and 5 men identified 11 usability issues. These informed further changes to the tool's title, the information on how to use the tool, and the presentation of results.

Conclusions

Web-based self-assessment tools need to be easy to find and should communicate health messages effectively. End users' feedback informed changes to improve the tool's acceptability, engagement, and impact. We expect that the revised tool will have greater reach and prompt more people to prepare well for pregnancy.

User-Centred Design and Development of a Smartphone Application (OverSight) for Digital Phenotyping in Ophthalmology

BACKGROUND

Visual impairment can significantly impact an individual's daily activities. Patients require regular monitoring, typically occurring within hospital eye services. Capacity constraints have necessitated innovative solutions to improve patient care. Existing digital solutions rely on task-based digital home monitoring such as visual acuity testing. These require active involvement from patients and do not typically offer an indication of quality of life. Digital phenotyping refers to the use of personal digital devices to quantify passive behaviour for detecting clinically significant changes in vision and act as biomarkers for disease. Its uniqueness lies in the ability to detect changes passively. The objective was to co-design an accessible smartphone app (OverSight) for the purposes of digital phenotyping in people with sight impairment.

METHODS

Development of OverSight included stakeholder consultations following principles of user-centred design. Apple iOS software frameworks (HealthKit, ResearchKit, and SensorKit) and a SwiftUI developer toolkit were used to enable the collection of active and passive data streams. Accessibility and usability were assessed using the System Usability Scale (SUS) and feedback following a 3-month pilot study. Consultations with patients informed the design of OverSight, including preferred survey scheduling and the relevancy of patient support resources.

RESULTS

Twenty visually impaired participants (mean age 42 ± 19 years) were recruited to the pilot study. The average score on the SUS was 76.8 (±8.9), indicating good usability. There was a statistically significant moderate negative correlation between SUS scores and visual acuity in both the better (r = -0.494; p ≤ 0.001) and worse eye (r = -0.421; p ≤ 0.001).

CONCLUSIONS

OverSight offers promising potential for collecting patient-generated health data for the purposes of digital phenotyping in patients with eye disease. Through further testing and validation, this novel approach to patient care may ultimately provide opportunities for remote monitoring in ophthalmology.

Proposal for the implementation and monitoring of a health care nursing protocol for patients with external ventricular drains

OBJECTIVE

To structure a proposal for implementing and monitoring the nursing care protocol for the safety of adult patients with external ventricular drains.

METHOD

Descriptive exploratory research based on Implementation Science, with intentional sampling. Carried out in a hospital complex in southern Brazil, between May and December 2022, in four stages: online questionnaire on knowledge of care for patients with external ventricular shunt for nurses; matrix of proposed indicators with key actors; self-instructional online course for nurses; synthesis of the existing protocol. Contextualized instructional design was used to develop the course.

RESULTS

Six nurses from the operating room and intensive care unit participated, identifying knowledge gaps, pointing out the need for specific training; four key actors in the neurosurgery service defined six indicators. An online course was created and made available on the platform of the institution with two modules, including eight videos. Protocol synthesis was adapted to the standards of the institution's quality sector.

CONCLUSION

A report regarding the implementation of the nursing care protocol for patients with external ventricular drains was delivered for the nurse who was the technical responsible. Future studies should evaluate the implementation and impacts it will generate. This model can be adapted by other institutions.

A Gender-Informed Smoking Cessation App for Women: Protocol for an Acceptability and Feasibility Study

BACKGROUND

Tobacco smoking remains the leading preventable cause of death and disease among women. Quitting smoking offers numerous health benefits; however, women tend to have less success than men when attempting to quit. This discrepancy is partly due to sex- and gender-related factors, including the lower effectiveness of smoking cessation medication and the presence of unique motives for smoking and barriers to quitting among women. Despite the gendered nature of smoking, most smoking cessation apps are gender-neutral and fail to address women's specific needs.

OBJECTIVE

This study aims to test the acceptability and feasibility of a smartphone app that delivers gender-informed content to support women in quitting smoking.

METHODS

We co-developed a smoking cessation app specifically tailored for women, named My Change Plan-Women (MCP-W). This app builds upon our previous gender-neutral app, MCP, by retaining its content grounded in behavioral change techniques aimed at supporting tobacco reduction and cessation. This includes goal setting for quitting, identifying triggers to smoking, creating coping strategies, tracking cigarettes and cravings, and assessing financial savings from quitting smoking. The MCP-W app contains additional gender-informed content that acknowledges barriers to quitting, such as coping with stress, having smokers in one's social circle, and managing unpleasant emotions. This content is delivered through testimonials and animated videos. This study is a prospective, single-group, mixed methods investigation in which 30 women smokers will trial the app for a period of 28 days. Once participants provide informed consent, they will complete a baseline survey and download the app on their smartphones. After 28 days, participants will complete follow-up surveys. Acceptability will be assessed using the Theoretical Framework of Acceptability, which evaluates whether participants perceive the app as helpful in changing their smoking. The app will be deemed acceptable if the majority of participants rate it as such, and feasible if the majority of the participants use it for at least 7 days. Furthermore, after the 28-day trial period, participants will complete a semistructured interview regarding their experience with the app and suggestions for improvement.

RESULTS

Development of the MCP-W app was completed in September 2023. Participant recruitment for testing of the app commenced in February 2024 and was completed in July 2024. We will analyze the data upon completion of data collection from all 30 participants. We expect to share the results of this acceptability trial in the middle of 2025.

CONCLUSIONS

Offering smoking cessation support tailored specifically to address the unique needs of women through a smartphone app represents a novel approach. This study will test whether women who smoke perceive this approach to be acceptable and feasible in their journey toward smoking cessation.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/60677.