Personalized Management of Fatigue in Individuals With Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and Long COVID Using a Smart Digital mHealth Solution: Protocol for a Participatory Design Approach

BACKGROUND

Fatigue is the most common symptom in myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) and long COVID, impacting patients' quality of life; however, there is currently a lack of evidence-based context-aware tools for fatigue self-management in these populations.

OBJECTIVE

This study aimed to (1) address fatigue in ME/CFS and long COVID through the development of digital mobile health solutions for self-management, (2) predict perceived fatigue severity using real-time data, and (3) assess the feasibility and potential benefits of personalized digital mobile health solutions.

METHODS

The MyFatigue project adopts a patient-centered approach within the participatory health informatics domain. Patient representatives will be actively involved in decision-making processes. This study combines inductive and deductive research approaches, using qualitative studies to generate new knowledge and quantitative methods to test hypotheses regarding the relationship between factors like physical activity, sleep behaviors, and perceived fatigue in ME/CFS and long COVID. Co-design methods will be used to develop a personalized digital solution for fatigue self-management based on the generated knowledge. Finally, a pilot study will evaluate the feasibility, acceptance, and potential benefits of the digital health solution.

RESULTS

The MyFatigue project opened to enrollment in November 2023. Initial results are expected to be published by the end of 2024.

CONCLUSIONS

This study protocol holds the potential to expand understanding, create personalized self-management approaches, engage stakeholders, and ultimately improve the well-being of individuals with ME/CFS and long COVID.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/50157.

Clustering user preferences for personalized teleoperation control schemes via trajectory similarity analysis

Successful operation of a teleoperated robot depends on a well-designed control scheme to translate human motion into robot motion; however, a single control scheme may not be suitable for all users. On the other hand, individual personalization of control schemes may be infeasible for designers to produce. In this paper, we present a method by which users may be classified into groups with mutually compatible control scheme preferences. Users are asked to demonstrate freehand motions to control a simulated robot in a virtual reality environment. Hand pose data is captured and compared with other users using SLAM trajectory similarity analysis techniques. The resulting pairwise trajectory error metrics are used to cluster participants based on their control motions, without foreknowledge of the number or types of control scheme preferences that may exist. The clusters identified for two different robots shows that a small number of clusters form stably for each case, each with its own control scheme paradigm. Survey data from participants validates that the clusters identified through this method correspond to the participants' control scheme rationales, and also identify nuances in participant control scheme descriptions that may not be obvious to designers relying only on participant explanations of their preferences.

A Technological Tool Aimed at Self-Care in Patients With Multimorbidity: Cross-Sectional Usability Study

BACKGROUND

Information and communication technologies (ICTs) have been positioned as useful tools to facilitate self-care. The interaction between a patient and technology, known as usability, is particularly important for achieving positive health outcomes. Specific characteristics of patients with chronic diseases, including multimorbidity, can affect their interaction with different technologies. Thus, studying the usability of ICTs in the field of multimorbidity has become a key element to ensure their relevant role in promoting self-care.

OBJECTIVE

The aim of this study was to analyze the usability of a technological tool dedicated to health and self-care in patients with multimorbidity in primary care.

METHODS

A descriptive observational cross-sectional usability study was performed framed in the clinical trial in the primary care health centers of Madrid Health Service of the TeNDER (Affective Based Integrated Care for Better Quality of Life) project. The TeNDER technological tool integrates sensors for monitoring physical and sleep activity along with a mobile app for consulting the data collected and working with self-management tools. This project included patients over 60 years of age who had one or more chronic diseases, at least one of which was mild-moderate cognitive impairment, Parkinson disease, or cardiovascular disease. From the 250 patients included in the project, 38 agreed to participate in the usability study. The usability variables investigated were effectiveness, which was determined by the degree of completion and the total number of errors per task; efficiency, evaluated as the average time to perform each task; and satisfaction, quantified by the System Usability Scale. Five tasks were evaluated based on real case scenarios. Usability variables were analyzed according to the sociodemographic and clinical characteristics of patients. A logistic regression model was constructed to estimate the factors associated with the type of support provided for task completion.

RESULTS

The median age of the 38 participants was 75 (IQR 72.0-79.0) years. There was a slight majority of women (20/38, 52.6%) and the participants had a median of 8 (IQR 7.0-11.0) chronic diseases. Thirty patients completed the usability study, with a usability effectiveness result of 89.3% (134/150 tasks completed). Among the 30 patients, 66.7% (n=20) completed all tasks and 56.7% (17/30) required personalized help on at least one task. In the multivariate analysis, educational level emerged as a facilitating factor for independent task completion (odds ratio 1.79, 95% CI 0.47-6.83). The median time to complete the total tasks was 296 seconds (IQR 210.0-397.0) and the median satisfaction score was 55 (IQR 45.0-62.5) out of 100.

CONCLUSIONS

Although usability effectiveness was high, the poor efficiency and usability satisfaction scores suggest that there are other factors that may interfere with the results. Multimorbidity was not confirmed to be a key factor affecting the usability of the technological tool.

TRIAL REGISTRATION

Clinicaltrials.gov NCT05681065; https://clinicaltrials.gov/study/NCT05681065.

Multidisciplinary collaboration on exoskeleton development adopting user-centered design: a systematic integrative review

Purpose: The world population is ageing, along with an increasing possibility of functional limitations that affect independent living. Assistive technologies such as exoskeletons for rehabilitative purposes and daily activities assistance maintaining the independence of people with disabilities, especially older adults who wish to ageing-in-place. The purpose of this systematic integrative review was threefold: to explore the development team compositions and involvement, to understand the recruitment and engagement of stakeholders, and to synthesise reported or anticipated consequences of multidisciplinary collaboration.Methods: Databases searched included PubMed, CINAHL Plus, PsycINFO, Web of Science, Scopus, and IEEE Xplore. A total of 34 studies that reported the development of exoskeleton adopting user-centered design (UCD) method in healthcare or community settings that were published in English from 2000 to July 2022 were included.Results: Three major trends emerged from the analysis of included studies. First, there is a need to redefine multidisciplinary collaboration, from within-discipline collaboration to cross-discipline collaboration. Second, the level of engagement of stakeholders during the exoskeleton development remained low. Third, there was no standardised measurement to quantify knowledge production currently.Conclusion: As suggested by the synthesised results in this review, exoskeleton development has been increasing to improve the functioning of people with disabilities. Exoskeleton development often required expertise from different disciplines and the involvement of stakeholders to increase acceptance, thus we propose the Multidisciplinary Collaboration Appraisal Tool to assess multidisciplinary collaboration using the UCD approach. Future research is required to understand the effectiveness of multidisciplinary collaboration on exoskeleton development using the UCD approach.IMPLICATIONS FOR REHABILITATIONGlobal trend of population ageing causes a higher risk of disability in older adults who require rehabilitation and assistance in daily living.Assistive technologies such as exoskeletons have the potential to contribute to rehabilitation training and daily activity assistance demand closer multidisciplinary collaboration.A Multidisciplinary Collaboration Appraisal Tool using user-centered design approach (MCAT) is proposed to understand the effectiveness as well as limitations and barriers associated with multidisciplinary collaboration in developing exoskeletons.

[User-oriented needs assessment of the potential use of assistive robots in direct nursing care: A mixed methods study]

User-oriented needs assessment of the potential use of assistive robots in direct nursing care: A mixed methods study Abstract. Background: So far, hardly any robots have been used in nursing that take over patient-related activities and thereby reduce the physical strain on the caregivers. Using user-centered design approaches, the interdisciplinary project "PfleKoRo" was therefore developing a robotic assistance system that can be used in the direct care of bedridden patients requiring intensive or very intensive care. Aim: The aim of this study was to identify nursing activities with the greatest support potential for an assistant robot for the direct care of bedridden patients. Method: Focus groups (n = 3) with nursing professionals (n = 14) from acute and long-term care were conducted first in an explorative mixed method design and then evaluated by means of content analysis. A selection of nursing activities was then prioritized by the participants of the focus groups (n = 10) with regard to their potential for support from an assistant robot in a standardized survey. Results: The highest priority was given to turning and holding patients in a lateral position as well as holding their legs in order to perform nursing tasks. Further support was needed, among other things, for repositioning the patient to the head of the bed and for tasks such as the transfer of patients. Conclusion: Turning patients and holding them in a lateral position as well as holding the leg are seen as target activities with the greatest support potential for "PfleKoRo", presenting the starting point for further development.

Developing accessible speech technology with users with dysarthric speech

The Voiceitt app is designed for people with dysarthric speech, to support vocal communication and access to voice-driven technologies. Sixty-six participants were recruited to test the Voiceitt app and share feedback with developers. Most had physical, sensory, or cognitive impairments in addition to atypical speech. The project team liaised with individuals, their families and local support teams to provide access to the app and associated equipment. Testing was user-led, with participants asked to identify and test use cases most relevant to their daily lives over three months or more. Ongoing technical support and training were provided remotely and in-person throughout their testing. Semi-structured interviews were used to collect feedback on users' experiences, with delivery adapted to individuals' needs and preferences. Informal feedback was collected through ongoing contact between participants, their families and support teams and the project team. User feedback has led to improvements to the user interface and functionality, including faster voice training, simplified navigation, the introduction of game-style features and of switch access as an alternative to touchscreen access. This work offers a case-study in meaningful engagement with diverse disabled users of assistive technology in commercial software development.

The REHAB-LAB model for individualized assistive device co-creation and production

Assistive devices are designed to enhance individuals with disabilities' functional abilities. The rise of 3D printing technology enabled the production of individualized assistive devices (IADs). A REHAB-LAB is intended for IAD provision involving technical referents and occupational therapists. This study aimed to develop the REHAB-LAB logic model; to explore its fidelity and desirability; and to explore the characteristics of arising initiatives of IAD production. The REHAB-LAB logic model development involved stakeholders throughout the research process. A pragmatic multimethod approach followed two phases 1) logic model development and 2) exploration of its fidelity and desirability. The REHAB-LAB logic model presented the resources (equipment, space, human) required to implement IAD provision in a rehabilitation center, and the expected deliverables (activities and outputs). The REHAB-LAB logic model highlights the interdisciplinarity of IAD provision including occupational therapists, doctors, engineers, managers, and technical referents and places the users at the center of the IAD production. Results confirmed the fidelity and desirability of the REHAB-LAB logic model. The REHAB-LAB logic model can be used as a reference for future healthcare organizations wishing to implement an IAD provision. This research highlighted the interest of IAD provision based on the REHAB-LAB model involving users and transdisciplinary practices.

Iterative Patient Testing of a Stimuli-Responsive Swallowing Activity Sensor to Promote Extended User Engagement During the First Year After Radiation: Multiphase Remote and In-Person Observational Cohort Study

BACKGROUND

Frequent sensor-assisted monitoring of changes in swallowing function may help improve detection of radiation-associated dysphagia before it becomes permanent. While our group has prototyped an epidermal strain/surface electromyography sensor that can detect minute changes in swallowing muscle movement, it is unknown whether patients with head and neck cancer would be willing to wear such a device at home after radiation for several months.

OBJECTIVE

We iteratively assessed patients' design preferences and perceived barriers to long-term use of the prototype sensor.

METHODS

In study 1 (questionnaire only), survivors of pharyngeal cancer who were 3-5 years post treatment and part of a larger prospective study were asked their design preferences for a hypothetical throat sensor and rated their willingness to use the sensor at home during the first year after radiation. In studies 2 and 3 (iterative user testing), patients with and survivors of head and neck cancer attending visits at MD Anderson's Head and Neck Cancer Center were recruited for two rounds of on-throat testing with prototype sensors while completing a series of swallowing tasks. Afterward, participants were asked about their willingness to use the sensor during the first year post radiation. In study 2, patients also rated the sensor's ease of use and comfort, whereas in study 3, preferences were elicited regarding haptic feedback.

RESULTS

The majority of respondents in study 1 (116/138, 84%) were willing to wear the sensor 9 months after radiation, and participant willingness rates were similar in studies 2 (10/14, 71.4%) and 3 (12/14, 85.7%). The most prevalent reasons for participants' unwillingness to wear the sensor were 9 months being excessive, unwanted increase in responsibility, and feeling self-conscious. Across all three studies, the sensor's ability to detect developing dysphagia increased willingness the most compared to its appearance and ability to increase adherence to preventive speech pathology exercises. Direct haptic signaling was also rated highly, especially to indicate correct sensor placement and swallowing exercise performance.

CONCLUSIONS

Patients and survivors were receptive to the idea of wearing a personalized risk sensor for an extended period during the first year after radiation, although this may have been limited to well-educated non-Hispanic participants. A significant minority of patients expressed concern with various aspects of the sensor's burden and its appearance.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03010150; https://clinicaltrials.gov/study/NCT03010150.

Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review

BACKGROUND

The increasing prevalence of DH applications has outpaced research and practice in digital health (DH) evaluations. Patient experience (PEx) was reported as one of the challenges facing the health system by the World Health Organization. To generate evidence on DH and promote the appropriate integration and use of technologies, a standard evaluation of PEx in DH is required.

OBJECTIVE

This study aims to systematically identify evaluation timing considerations (ie, when to measure), evaluation indicators (ie, what to measure), and evaluation approaches (ie, how to measure) with regard to digital PEx. The overall aim of this study is to generate an evaluation guide for further improving digital PEx evaluation.

METHODS

This is a 2-phase study parallel to our previous study. In phase 1, literature reviews related to PEx in DH were systematically searched from Scopus, PubMed, and Web of Science databases. Two independent raters conducted 2 rounds of paper screening, including title and abstract screening and full-text screening, and assessed the interrater reliability for 20% (round 1: 23/115 and round 2: 12/58) random samples using the Fleiss-Cohen coefficient (round 1: k1=0.88 and round 2: k2=0.80). When reaching interrater reliability (k>0.60), TW conducted the rest of the screening process, leaving any uncertainties for group discussions. Overall, 38% (45/119) of the articles were considered eligible for further thematic analysis. In phase 2, to check if there were any meaningful novel insights that would change our conclusions, we performed an updated literature search in which we collected 294 newly published reviews, of which 102 (34.7%) were identified as eligible articles. We considered them to have no important changes to our original results on the research objectives. Therefore, they were not integrated into the synthesis of this review and were used as supplementary materials.

RESULTS

Our review highlights 5 typical evaluation objectives that serve 5 stakeholder groups separately. We identified a set of key evaluation timing considerations and classified them into 3 categories: intervention maturity stages, timing of the evaluation, and timing of data collection. Information on evaluation indicators of digital PEx was identified and summarized into 3 categories (intervention outputs, patient outcomes, and health care system impact), 9 themes, and 22 subthemes. A set of evaluation theories, common study designs, data collection methods and instruments, and data analysis approaches was captured, which can be used or adapted to evaluate digital PEx.

CONCLUSIONS

Our findings enabled us to generate an evaluation guide to help DH intervention researchers, designers, developers, and program evaluators evaluate digital PEx. Finally, we propose 6 directions for encouraging further digital PEx evaluation research and practice to address the challenge of poor PEx.

Improving the design of patient-generated health data visualizations: design considerations from a Fitbit sleep study

Interactive data visualization can be a viable way to discover patterns in patient-generated health data and enable health behavior changes. However, very few studies have investigated the design and usability of such data visualization. The present study aimed to (1) explore user experiences with sleep data visualizations in the Fitbit app, and (2) focus on end users' perspectives to identify areas of improvement and potential solutions. The study recruited eighteen pre-medicine college students, who wore Fitbit watches for a two-week sleep data collection period and participated in an exit semi-structured interview to share their experience. A focus group was conducted subsequently to ideate potential solutions. The qualitative analysis identified six pain points (PPs) from the interview data using affinity mapping. Four design solutions were proposed by the focus group to address these PPs and illustrated by a set of mock-ups. The study findings informed four design considerations: (1) usability, (2) transparency and explainability, (3) understandability and actionability, and (4) individualized benchmarking. Further research is needed to examine the design guidelines and best practices of sleep data visualization, to create well-designed visualizations for the general population that enables health behavior changes.

Clinical Needs Assessment of a Machine Learning-Based Asthma Management Tool: User-Centered Design Approach

BACKGROUND

Personalized asthma management depends on a clinician's ability to efficiently review patient's data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages.

OBJECTIVE

We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians' experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use.

METHODS

At the "discovery" phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the "define" phase, a synthesis analysis was conducted to converge key results from interviewees' insights into themes, eventually forming critical "how might we" research questions to guide model development and implementation.

RESULTS

We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients' high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants.

CONCLUSIONS

As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design.

User Perceptions of Visual Clot in a High-Fidelity Simulation Study: Mixed Qualitative-Quantitative Study

BACKGROUND

Viscoelastic hemostatic assays, such as rotational thromboelastometry (ROTEM) or thromboelastography, enable prompt diagnosis and accelerate targeted treatment. However, the complex interpretation of the results remains challenging. Visual Clot-a situation awareness-based visualization technology-was developed to assist clinicians in interpreting viscoelastic tests.

OBJECTIVE

Following a previous high-fidelity simulation study, we analyzed users' perceptions of the technology, to identify its strengths and limitations from clinicians' perspectives.

METHODS

This is a mixed qualitative-quantitative study consisting of interviews and a survey. After solving coagulation scenarios using Visual Clot in high-fidelity simulations, we interviewed anesthesia personnel about the perceived advantages and disadvantages of the new tool. We used a template approach to identify dominant themes in interview responses. From these themes, we defined 5 statements, which were then rated on Likert scales in a questionnaire.

RESULTS

We interviewed 77 participants and 23 completed the survey. We identified 9 frequently mentioned topics by analyzing the interview responses. The most common themes were "positive design features," "intuitive and easy to learn," and "lack of a quantitative component." In the survey, 21 respondents agreed that Visual Clot is easy to learn and 16 respondents stated that a combination of Visual Clot and ROTEM would help them manage complex hemostatic situations.

CONCLUSIONS

A group of anesthesia care providers found Visual Clot well-designed, intuitive, and easy to learn. Participants highlighted its usefulness in emergencies, especially for clinicians inexperienced in coagulation management. However, the lack of quantitative information is an area for improvement.

Design of Virtual Reality Exergames for Upper Limb Stroke Rehabilitation Following Iterative Design Methods: Usability Study

BACKGROUND

Since the early 2000s, there has been a growing interest in using exercise video games (exergames) and virtual reality (VR)-based interventions as innovative methods to enhance physical rehabilitation for individuals with multiple disabilities. Over the past decade, researchers and exercise professionals have focused on developing specialized immersive exercise video games for various populations, including those who have experienced a stroke, revealing tangible benefits for upper limb rehabilitation. However, it is necessary to develop highly engaging, personalized games that can facilitate the creation of experiences aligned with the preferences, motivations, and challenges communicated by people who have had an episode of stroke.

OBJECTIVE

This study seeks to explore the customization potential of an exergame for individuals who have undergone a stroke, concurrently evaluating its usability as a technological tool in the realm of physical therapy and rehabilitation.

METHODS

We introduce a playtest methodology to enhance the design of a VR exergame developed using a user-centered approach for upper limb rehabilitation in stroke survivors. Over 4 playtesting sessions, stroke survivors interacted with initial game versions using VR headsets, providing essential feedback for refining game content and mechanics. Additionally, a pilot study involving 10 stroke survivors collected data through VR-related questionnaires to assess game design aspects such as mechanics, assistance, experience, motion sickness, and immersion.

RESULTS

The playtest methodology was beneficial for improving the exergame to align with user needs, consistently incorporating their perspectives and achieving noteworthy results. The pilot study revealed that users had a positive response. In the first scenario, a carpenter presents a game based on the flexion-extension movement of the elbow; the second scenario includes a tejo game (a traditional Colombian throwing game) designed around game mechanics related to the flexion-extension movement of the shoulder; and in the third scenario, a farmer challenges the player to perform a movement combining elbow flexion and extension with internal and external rotation of the shoulder. These findings suggest the potential of the studied exergame as a tool for the upper limb rehabilitation of individuals who have experienced a stroke.

CONCLUSIONS

The inclusion of exergames in rehabilitation for stroke-induced hemiparesis has significantly benefited the recovery process by focusing on essential shoulder and elbow movements. These interactive games play a crucial role in helping users regain mobility and restore practical use of affected limbs. They also serve as valuable data sources for researchers, improving the system's responsiveness. This iterative approach enhances game design and markedly boosts user satisfaction, suggesting exergames have promising potential as adjunctive elements in traditional therapeutic approaches.

Acceptance and User Experiences of a Wearable Device for the Management of Hospitalized Patients in COVID-19-Designated Wards in Ho Chi Minh City, Vietnam: Action Learning Project

BACKGROUND

Wearable devices have been used extensively both inside and outside of the hospital setting. During the COVID-19 pandemic, in some contexts, there was an increased need to remotely monitor pulse and saturated oxygen for patients due to the lack of staff and bedside monitors.

OBJECTIVE

A prototype of a remote monitoring system using wearable pulse oximeter devices was implemented at the Hospital for Tropical Diseases in Ho Chi Minh City, Vietnam, from August to December 2021. The aim of this work was to support the ongoing implementation of the remote monitoring system.

METHODS

We used an action learning approach with rapid pragmatic methods, including informal discussions and observations as well as a feedback survey form designed based on the technology acceptance model to assess the use and acceptability of the system. Based on these results, we facilitated a meeting using user-centered design principles to explore user needs and ideas about its development in more detail.

RESULTS

In total, 21 users filled in the feedback form. The mean technology acceptance model scores ranged from 3.5 (for perceived ease of use) to 4.4 (for attitude) with behavioral intention (3.8) and perceived usefulness (4.2) scoring in between. Those working as nurses scored higher on perceived usefulness, attitude, and behavioral intention than did physicians. Based on informal discussions, we realized there was a mismatch between how we (ie, the research team) and the ward teams perceived the use and wider purpose of the technology.

CONCLUSIONS

Designing and implementing the devices to be more nurse-centric from their introduction could have helped to increase their efficiency and use during the complex pandemic period.

Insights into implementation planning for point-of-care testing to guide treatment of chronic obstructive pulmonary disease exacerbation: a mixed methods feasibility study

The purpose of this mixed methods feasibility study was to gain insights into unmet clinical needs, stakeholder preferences and potential barriers and enablers to adoption for planning the implementation of point-of-care testing for earlier detection and guided treatment of chronic obstructive pulmonary disease (COPD) acute exacerbation in the NHS in England. Exacerbations of COPD cause considerable mortality and morbidity. Earlier identification of exacerbations and guided treatment would lead to reduced exacerbation duration, reduced hospitalizations and mortality, improve health-related quality of life, reduce unnecessary treatments (including inappropriate antibiotic prescribing) which could save the NHS over £400 per patient. During the early stages of product design, we took a multi-disciplinary approach to evidence generation, gaining insights from key stakeholders to test the product concept and inform evidence-based implementation planning. Primary data was collected from 11 health care and service professionals involved in the management of acute COPD exacerbations. Overall, participants agreed that by earlier differentiation of acute exacerbation from stable COPD, patients could be started on appropriate treatment. To implement point-of-care testing into clinical practice, evidence is required to demonstrate the accuracy of differentiating between exacerbation etiologies and to provide information on the beneficial impact to the system in terms of optimized management, reduced long-term side effects, admission avoidance, and cost-effectiveness. This research provides an evidence base for future implementation planning of point-of-care testing for earlier detection and guided treatment of COPD acute exacerbation. Moreover, the technology developers can decide whether to refine the product design and value proposition thereby de-risking product development.

An mHealth Design to Promote Medication Safety in Children with Medical Complexity

BACKGROUND

Children with medical complexity (CMC) are uniquely vulnerable to medication errors and preventable adverse drug events because of their extreme polypharmacy, medical fragility, and reliance on complicated medication schedules and routes managed by undersupported family caregivers. There is an opportunity to improve CMC outcomes by designing health information technologies that support medication administration accuracy, timeliness, and communication within CMC caregiving networks.

OBJECTIVES

The present study engaged family caregivers, secondary caregivers, and clinicians who work with CMC in a codesign process to identify: (1) medication safety challenges experienced by CMC caregivers and (2) design requirements for a mobile health application to improve medication safety for CMC in the home.

METHODS

Study staff recruited family caregivers, secondary caregivers, and clinicians from a children's hospital-based pediatric complex care program to participate in virtual codesign sessions. During sessions, the facilitator-guided codesigners in generating and converging upon medication safety challenges and design requirements. Between sessions, the research team reviewed notes from the session to identify design specifications and modify the prototype. After design sessions concluded, each session recording was reviewed to confirm that all designer comments had been captured.

RESULTS

A total of N = 16 codesigners participated. Analyses yielded 11 challenges to medication safety and 11 corresponding design requirements that fit into three broader challenges: giving the right medication at the right time; communicating with others about medications; and accommodating complex medical routines. Supporting quotations from codesigners and prototype features associated with each design requirement are presented.

CONCLUSION

This study generated design requirements for a tool that may improve medication safety by creating distributed situation awareness within the caregiving network. The next steps are to pilot test tools that integrate these design requirements for usability and feasibility, and to conduct a randomized control trial to determine if use of these tools reduces medication errors.

User-Centered Design of a Diabetes Self-Management Tool for Underserved Populations

BACKGROUND

User-centered design (UCD) or user experience design (UXD) methods have gained recent popularity for the design of mobile health (mHealth) interventions. However, there is a gap in application of these methods for diabetes self-management. This study aims to document the UCD process for a self-management mobile application aimed for patients with diabetes in underserved communities.

METHODS

A UCD mixed-methods approach including interviews with patients and providers, a review of literature, and a technology landscape analysis were used to define the app functional information requirements that informed the user experience/user interface design process. Usability studies with the app designers and developers, intended users, and a focus group of nurse educators and dieticians were used to test and improve the design.

RESULTS

An mHealth app was developed with health-tracking features for stress, blood sugar, food, exercise, medications, weight, and blood pressure. We tackled a range of usability and user experience challenges, which encompassed addressing issues like low health literacy by employing a combination of user interface design principles, intuitive visualizations, customizable icons, seamless database integration, and automated data input features. Special attention was given to the design of educational content accounting for the intended users' cultural background and literacy levels.

CONCLUSIONS

User-centered design approach contributed to a better understanding of the intended users' needs, limitations, mental models, and expectations, facilitating the design of a comprehensive mobile app for patients with diabetes in underserved communities that includes essential features for self-management while providing a strong educational component, addressing an important gap in the literature.

A case study of pragmatic measure development of the Treatment Integrity for Elementary Settings self-report measure for teachers

Introduction

Due to usability, feasibility, and acceptability concerns, observational treatment fidelity measures are often challenging to deploy in schools. Teacher self-report fidelity measures with specific design features might address some of these barriers. This case study outlines a community-engaged, iterative process to adapt the observational Treatment Integrity for Elementary Settings (TIES-O) to a teacher self-report version designed to assess the use of practices to support children's social-emotional competencies in elementary classrooms.

Method

Cognitive walkthrough interviews were conducted with teachers to improve the usability of the teacher self-report measure, called the Treatment Integrity for Elementary Schools-Teacher Report (TIES-T). Qualitative content analysis was used to extract themes from the interviews and inform changes to the measure.

Results

Increasing clarity and interactive elements in the measure training were the dominant themes, but suggestions for the measure format and jargon were also suggested.

Conclusion

The suggested changes resulted in a brief measure, training, and feedback system designed to support the teacher's use of practices to support children's social-emotional competencies in elementary classrooms. Future research with the TIES-T will examine the score reliability and validity of the measure.

Design and Usability Assessment of a User-Centered, Modular Platform for Real-World Data Acquisition in Clinical Trials involving Post-bariatric Surgery Patients

BACKGROUND

Clinical trials often face challenges in efficient data collection and participant monitoring. To address these issues, we developed the IMPACT platform, comprising a real-time mobile application for data collection and a web-based dashboard for remote monitoring and management.

METHODS

This article presents the design, development, and usability assessment of the IMPACT platform customized for patients with post-bariatric surgery hypoglycemia (PBH). We focus on adapting key IMPACT components, including continuous glucose monitoring (CGM), symptom tracking, and meal logging, as crucial elements for user-friendly and efficient PBH monitoring.

RESULTS

The adapted IMPACT platform demonstrated effectiveness in data collection and remote participant monitoring. The mobile application allowed patients to easily track their data, while the clinician dashboard provided a comprehensive overview of enrolled patients, featuring filtering options and alert mechanisms for identifying data collection issues. The platform incorporated various visual representations, including time plots and category-based visualizations, which greatly facilitated data interpretation and analysis. The System Usability Scale questionnaire results indicated a high level of usability for the web dashboard, with an average score of 86.3 out of 100. The active involvement of clinicians throughout the development process ensured that the platform allowed for the collection and visualization of clinically meaningful data.

CONCLUSIONS

By leveraging IMPACT's existing features and infrastructure, the adapted version streamlined data collection, analysis, and trial customization for PBH research. The platform's high usability underscores its alignment with the requirements for conducting research using continuous real-world data in PBH patients and other populations of interest.

Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach

OBJECTIVES

To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design.

MATERIALS AND METHODS

Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements.

RESULTS

Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist.

CONCLUSION

User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.

A Brain Computer Interface Neuromodulatory Device for Stroke Rehabilitation: Iterative User-Centered Design Approach

BACKGROUND

Rehabilitation technologies for people with stroke are rapidly evolving. These technologies have the potential to support higher volumes of rehabilitation to improve outcomes for people with stroke. Despite growing evidence of their efficacy, there is a lack of uptake and sustained use in stroke rehabilitation and a call for user-centered design approaches during technology design and development. This study focuses on a novel rehabilitation technology called exciteBCI, a complex neuromodulatory wearable technology in the prototype stage that augments locomotor rehabilitation for people with stroke. The exciteBCI consists of a brain computer interface, a muscle electrical stimulator, and a mobile app.

OBJECTIVE

This study presents the evaluation phase of an iterative user-centered design approach supported by a qualitative descriptive methodology that sought to (1) explore users' perspectives and experiences of exciteBCI and how well it fits with rehabilitation, and (2) facilitate modifications to exciteBCI design features.

METHODS

The iterative usability evaluation of exciteBCI was conducted in 2 phases. Phase 1 consisted of 3 sprint cycles consisting of single usability sessions with people with stroke (n=4) and physiotherapists (n=4). During their interactions with exciteBCI, participants used a "think-aloud" approach, followed by a semistructured interview. At the end of each sprint cycle, device requirements were gathered and the device was modified in preparation for the next cycle. Phase 2 focused on a "near-live" approach in which 2 people with stroke and 1 physiotherapist participated in a 3-week program of rehabilitation augmented by exciteBCI (n=3). Participants completed a semistructured interview at the end of the program. Data were analyzed from both phases using conventional content analysis.

RESULTS

Overall, participants perceived and experienced exciteBCI positively, while providing guidance for iterative changes. Five interrelated themes were identified from the data: (1) "This is rehab" illustrated that participants viewed exciteBCI as having a good fit with rehabilitation practice; (2) "Getting the most out of rehab" highlighted that exciteBCI was perceived as a means to enhance rehabilitation through increased engagement and challenge; (3) "It is a tool not a therapist," revealed views that the technology could either enhance or disrupt the therapeutic relationship; and (4) "Weighing up the benefits versus the burden" and (5) "Don't make me look different" emphasized important design considerations related to device set-up, use, and social acceptability.

CONCLUSIONS

This study offers several important findings that can inform the design and implementation of rehabilitation technologies. These include (1) the design of rehabilitation technology should support the therapeutic relationship between the patient and therapist, (2) social acceptability is a design priority in rehabilitation technology but its importance varies depending on the use context, and (3) there is value in using design research methods that support understanding usability in the context of sustained use.

A Newly Developed Exergame-Based Telerehabilitation System for Older Adults: Usability and Technology Acceptance Study

BACKGROUND

Telerehabilitation has gained significance as a tool to deliver and supervise therapy and training as effective as traditional rehabilitation methods yet more accessible and affordable. An exergame-based telerehabilitation system has recently been developed within the scope of the international Continuum-of-Care (COCARE) project. The system comprises training devices for use in clinics (Dividat Senso) and at home (Dividat Senso Flex), an assessment system, and a rehabilitation cockpit, and its focus lies on home-based motor-cognitive training, which is remotely managed by health care professionals (HPs).

OBJECTIVE

This study aims to analyze the usability, acceptance, and enjoyment of the COCARE system from the perspective of primary (older adults [OAs]) and secondary (HPs) end users.

METHODS

At 3 trial sites (located in Switzerland, Italy, and Cyprus), participants engaged in a single-session trial of the COCARE system, including testing of exergames and assessments. Mixed methods encompassing qualitative approaches (eg, think aloud) and quantitative measures (eg, Exergame Enjoyment Questionnaire [EEQ], System Usability Scale [SUS], and Unified Theory of Acceptance and Use of Technology [UTAUT] questionnaire) were used to analyze participants' perceptions of the system and identify potential barriers to its implementation in a home setting. In addition, the associations of performance during gameplay and assessments, demographics, and training motivation (Behavioral Regulation in Exercise Questionnaire-3 [BREQ-3]) with usability, acceptance, and enjoyment were explored.

RESULTS

A total of 45 OAs and 15 HPs participated in this study. The COCARE system achieved good acceptance ratings (OAs: 83%, range 36%-100% and HPs: 81%, range 63.8%-93.3% of the maximum score), and OAs indicated high enjoyment (mean 73.3, SD 12.7 out of 100 points in the EEQ) during the exergame session. The system's usability, assessed with the SUS, received scores of 68.1 (SD 18.8; OAs) and 70.7 (SD 12.3; HPs) out of 100 points, with substantial differences observed between the trial sites. Several requirements for improvement were identified. Commonly mentioned barriers to adoption included the movement-recognition sensitivity of the Senso Flex, its limited markings, and difficulties in understanding certain instructions for assessments and games. Performance in games and assessments showed the highest significant correlations with the SUS (Spearman ρ=0.35, P=.02 to ρ=0.52, P<.001). The BREQ-3 had significant correlations with all usability measures, thereby even large significant correlations with enjoyment (Spearman ρ=0.58; P<.001). Age had moderately significant correlations with the SUS (Spearman ρ=-0.35; P=.02) and the UTAUT total score (ρ=-0.35; P=.02) but no significant correlation with the EEQ. Concerning sex and years of education, no significant correlations were found.

CONCLUSIONS

The study's findings will inform the further development of the COCARE system toward a user-friendly and widely accepted version, enhancing cognitive and physical functions in OAs. Future randomized controlled trials should evaluate the system's feasibility and effectiveness.

Multilayer framework for digital multicomponent platform design for colorectal survivors and carers: a qualitative study

Background

The advent of eHealth services offers the potential to support colorectal cancer (CRC) survivors and their informal caregivers (ICs), yet research into user needs and design requirements remains scant. This exploratory qualitative study addresses this knowledge gap by focusing on the development of a Digital Multicomponent Platform (DMP) designed to provide comprehensive support to these populations.

Aims

The objective of this research is to use qualitative methodologies to identify key user needs and design requirements for eHealth services. It seeks to propose and apply a multi-tiered framework for creating a DMP that encapsulates the needs of CRC survivors and their ICs.

Methods

Skype-based focus groups (FGs) were utilized to gather qualitative data from CRC survivors and ICs. This approach served to elicit crucial themes integral to the design of the DMP. A multi-tiered framework was subsequently developed to integrate user-centered design (UCD) principles and requirements with predetermined outcomes, eHealth services, and IT infrastructure.

Results

The first stage of the analysis identified five crucial themes: (1) the importance of healthcare system interaction via eHealth, (2) interaction between healthcare providers and peers, (3) lifestyle and wellness considerations, (4) platform content and user interface requirements, (5) caregiver support. The second stage analysis applied the multi-tiered framework, to determine the DMP that was conceptualized from these themes, underscores the significance of personalized content, caregiver involvement, and integration with electronic health records (EHRs).

Conclusion

The study offers novel insights into the design and development of digital supportive care interventions for CRC survivors and their caregivers. The results highlight the utility of user-centered design principles, the significance of personalized content and caregiver involvement, and the need for a unified health data platform that promotes communication among patients, healthcare providers, and peers. This multi-tiered framework could serve as a prototype for future eHealth service designs.

User-Centered Design of a Gamified Mental Health App for Adolescents in Sub-Saharan Africa: Multicycle Usability Testing Study

BACKGROUND

There is an urgent need for scalable psychological treatments to address adolescent depression in low-resource settings. Digital mental health interventions have many potential advantages, but few have been specifically designed for or rigorously evaluated with adolescents in sub-Saharan Africa.

OBJECTIVE

This study had 2 main objectives. The first was to describe the user-centered development of a smartphone app that delivers behavioral activation (BA) to treat depression among adolescents in rural South Africa and Uganda. The second was to summarize the findings from multicycle usability testing.

METHODS

An iterative user-centered agile design approach was used to co-design the app to ensure that it was engaging, culturally relevant, and usable for the target populations. An array of qualitative methods, including focus group discussions, in-depth individual interviews, participatory workshops, usability testing, and extensive expert consultation, was used to iteratively refine the app throughout each phase of development.

RESULTS

A total of 160 adolescents from rural South Africa and Uganda were involved in the development process. The app was built to be consistent with the principles of BA and supported by brief weekly phone calls from peer mentors who would help users overcome barriers to engagement. Drawing on the findings of the formative work, we applied a narrative game format to develop the Kuamsha app. This approach taught the principles of BA using storytelling techniques and game design elements. The stories were developed collaboratively with adolescents from the study sites and included decision points that allowed users to shape the narrative, character personalization, in-app points, and notifications. Each story consists of 6 modules ("episodes") played in sequential order, and each covers different BA skills. Between modules, users were encouraged to work on weekly activities and report on their progress and mood as they completed these activities. The results of the multicycle usability testing showed that the Kuamsha app was acceptable in terms of usability and engagement.

CONCLUSIONS

The Kuamsha app uniquely delivered BA for adolescent depression via an interactive narrative game format tailored to the South African and Ugandan contexts. Further studies are currently underway to examine the intervention's feasibility, acceptability, and efficacy in reducing depressive symptoms.

User-Centered Development of Bolster, an mHealth Intervention for Early Psychosis Caregivers: Needs Assessment, Prototyping, and Field Trial

BACKGROUND

Caregivers play a critical role in the treatment and recovery of youth and young adults at risk for psychosis. Caregivers often report feeling isolated, overwhelmed, and lacking in resources. Mobile health (mHealth) has the potential to provide scalable, accessible, and in-the-moment support to caregivers. To date, few if any mHealth resources have been developed specifically for this population.

OBJECTIVE

The aim of this study was to conduct user-centered design and testing of an mHealth intervention to support early psychosis caregivers.

METHODS

We conducted a multiphase user-centered development process to develop the Bolster mobile app. In phase 1, a total of 21 caregivers were recruited to participate in a qualitative needs assessment and respond to an initial prototype of the Bolster platform. Content analysis was used to identify key needs and design objectives, which guided the development of the Bolster mobile app. In phase 2, a total of 11 caregivers were recruited to participate in a 1-week field trial wherein they provided qualitative and quantitative feedback regarding the usability and acceptability of Bolster; in addition, they provided baseline and posttest assessments of the measures of distress, illness appraisals, and family communication.

RESULTS

In phase 1, participants identified psychoeducation, communication coaching, a guide to seeking services, and support for coping as areas to address. Live prototype interaction sessions led to multiple design objectives, including ensuring that messages from the platform were actionable and tailored to the caregiver experience, delivering messages in multiple modalities (eg, video and text), and eliminating a messaging-style interface. These conclusions were used to develop the final version of Bolster tested in the field trial. In phase 2, of the 11 caregivers, 10 (91%) reported that they would use Bolster if they had access to it and would recommend it to another caregiver. They also reported marked changes in their appraisals of illness (Cohen d=0.55-0.68), distress (Cohen d=1.77), and expressed emotion (Cohen d=0.52).

CONCLUSIONS

To our knowledge, this study is the first to design an mHealth intervention specifically for early psychosis caregivers. Preliminary data suggest that Bolster is usable, acceptable, and promising to improve key targets and outcomes. A future fully powered clinical trial will help determine whether mHealth can reduce caregiver burdens and increase engagement in services among individuals affected by psychosis.

Pediatric Anesthesia Providers' Perspective on the Real-Life Implementation of the Philips Visual Patient Avatar: A Qualitative Study

The Philips Visual Patient Avatar represents an alternative method of patient monitoring that, according to computer-based simulation studies, enhances diagnostic accuracy and confidence and reduces workload. After its clinical integration, we assessed pediatric anesthesia providers' perspectives on this technology. This is a single-center qualitative study, conducted at the University Hospital Zurich using in-depth individual interviews. We aimed to identify the advantages and limitations of the Visual Patient Avatar in pediatric anesthesia and to assess children's and parents' reactions from caregivers' perspectives. Thematic analysis was used to identify the dominant themes. Fourteen members of the institution's pediatric anesthesia team were interviewed. The most prevalent themes were children's positive reactions towards the Visual Patient Avatar (92.9%) and enhanced speed in problem identification (71.4%). Additionally, 50% of participants reported finding the Visual Patient Avatar useful for diverting children's attention during anesthesia induction, and 50% suggested that its vital sign thresholds should be adaptable for different age groups. The study revealed that the Visual Patient Avatar was recognized as a tool in pediatric anesthesia, enabling prompt identification of underlying issues and receiving positive feedback from both children and parents. The most commonly voiced wish for improvement in the study was the ability to customize the Visual Patient Avatar's thresholds for different age groups.

Fostering Patient-Clinician Communication to Promote Rapid HIV, Hepatitis B Virus, and Hepatitis C Virus Diagnostic Testing: Conceptual Development of a Multilingual App

BACKGROUND

Migrants are disproportionately affected by HIV, hepatitis B virus (HBV), and hepatitis C virus (HCV). Clinicians, at times, fail to offer rapid diagnostic testing (RDT) for these viruses when a language barrier exists in the patient-clinician relationship, therefore creating missed testing opportunities. Although their effectiveness has been demonstrated elsewhere, conventional, in-person interpreters are costly and underused in practice. Furthermore, clinicians often call upon ad hoc interpreters, which introduces complexities in the clinical relationship. Digital solutions exist to diminish the burden of language barriers; however, the challenges of developing a multilingual and multicultural app have yet to be documented with respect to RDT in the nonfrancophone migrant population in France.

OBJECTIVE

Our goal was to design a multilingual app to overcome language barriers, health literacy barriers, and fears related to being tested to promote RDT of HIV, HBV, and HCV in the nonfrancophone migrant population in France.

METHODS

A combination of qualitative methods, agile development, and user-centered design was used. We conducted 2 focus groups (FGs) with 12 participants, including physicians, nurses, and social workers conducting RDT, as well as 1 modified Delphi survey with 68 participants including physicians and nurses. FGs explored the content (risk factors and medical history), functions (cultural adaptation and instant translation), and interface ergonomics (graphics and font) needed in the app. The Delphi presented 95 content items that the researchers sought to include in the app.

RESULTS

Using FGs to inform the Delphi survey, we scientifically determined the app's content consisting of 95 items using expert consensus, developed a mock-up, and conducted initial user testing. We created an app that contains both migrant and clinician interfaces and includes a sociodemographic, risk assessment, health literacy, and testing barrier questionnaires available in 11 languages. Educational content is related to HIV, HBV, and HCV, along with the ability to understand whether the migrant agrees to be tested.

CONCLUSIONS

This study allowed us to conceptualize a multilingual app that aims to increase the acceptance of RDT for HIV, HBV, and HCV. The specific features of the Assistant intelligent au dépistage des allophones app were designed to overcome the testing barriers in the nonfrancophone migrant population. The next phase will be an implementation study, as we intend to validate our app.

Toward a strengths-based model for designing virtual reality learning experiences for autistic users

LAY ABSTRACT

Virtual reality has been studied for its potential in supporting individuals with autism, but existing research often focuses on deficits and lacks consideration of individual preferences and strengths. This article introduces a framework that emphasizes the strengths and abilities of autistic individuals when designing virtual reality interventions. It builds upon an existing taxonomy of educational technology affordances and extends it to align with the unique needs of autistic individuals. The framework provides guidance for incorporating virtual reality technology that supports and amplifies autistic strengths, such as visual perception and response to positive feedback. The framework has implications for practice, research, and policy. For practitioners, it offers a tool for designing virtual reality experiences that cater to the strengths of autistic individuals, enhancing engagement and educational outcomes. Researchers can utilize the framework to guide the development of user-centered virtual reality interventions and expand our understanding of the potential benefits of virtual reality for autistic populations. Policymakers and educators can consider this framework when incorporating virtual reality into educational settings, ensuring that virtual reality technology is used in a way that aligns with the strengths and needs of autistic learners. Overall, the framework promotes a strengths-based approach in utilizing virtual reality technology for individuals with autism, fostering inclusivity and maximizing the benefits of immersive experiences.

User Perceptions of Avatar-Based Patient Monitoring for Intensive Care Units: An International Exploratory Sequential Mixed-Methods Study

Visual Patient Avatar ICU is an innovative approach to patient monitoring, enhancing the user's situation awareness in intensive care settings. It dynamically displays the patient's current vital signs using changes in color, shape, and animation. The technology can also indicate patient-inserted devices, such as arterial lines, central lines, and urinary catheters, along with their insertion locations. We conducted an international, multi-center study using a sequential qualitative-quantitative design to evaluate users' perception of Visual Patient Avatar ICU among physicians and nurses. Twenty-five nurses and twenty-five physicians from the ICU participated in the structured interviews. Forty of them completed the online survey. Overall, ICU professionals expressed a positive outlook on Visual Patient Avatar ICU. They described Visual Patient Avatar ICU as a simple and intuitive tool that improved information retention and facilitated problem identification. However, a subset of participants expressed concerns about potential information overload and a sense of incompleteness due to missing exact numerical values. These findings provide valuable insights into user perceptions of Visual Patient Avatar ICU and encourage further technology development before clinical implementation.

Improving the Usability of Written Exposure Therapy for Therapists in the Department of Veterans Affairs Telemental Health: Formative Study Using Qualitative and User-Centered Design Methods

BACKGROUND

User modifications are common in evidence-based psychosocial interventions (EBPIs) for mental health disorders. Often, EBPIs fit poorly into clinical workflows, require extensive resources, or pose considerable burden to patients and therapists. Implementation science is increasingly researching ways to improve the usability of EBPIs before implementation. A user-centered design can be used to support implementation methods to prioritize user needs and solutions to improve EBPI usability.

OBJECTIVE

Trauma-focused EBPIs are a first-line treatment for patients with posttraumatic stress disorder (PTSD) in the Department of Veterans Affairs. Written exposure therapy (WET) is a brief, trauma-focused EBPI wherein patients handwrite about trauma associated with their PTSD. Initially developed for in-person delivery, WET is increasingly being delivered remotely, and outcomes appear to be equivalent to in-person delivery. However, there are logistical issues in delivering WET via video. In this evaluation, we explored usability issues related to WET telehealth delivery via videoconferencing software and designed a solution for therapist-facing challenges to systematize WET telehealth delivery.

METHODS

The Discover, Design and Build, and Test framework guided this formative evaluation and served to inform a larger Virtual Care Quality Enhancement Research Initiative. We used qualitative descriptive methods in the Discover phase to understand the experiences and needs of 2 groups of users providing care within the Department of Veterans Affairs: in-person therapists delivering WET via video because of the COVID-19 pandemic and telehealth therapists who regularly deliver PTSD therapies. We then used user-centered design methods in the Design and Build phase to brainstorm, develop, and iteratively refine potential workflows to address identified usability issues. All procedures were conducted remotely.

RESULTS

In the Discover phase, both groups had challenges delivering WET and other PTSD therapies via telehealth because of technology issues with videoconferencing software, environmental distractions, and workflow disruptions. Narrative transfer (ie, patients sending handwritten trauma accounts to therapists) was the first target for design solution development as it was deemed most critical to WET delivery. In the Design and Build phase, we identified design constraints and brainstormed solution ideas. This led to the development of 3 solution workflows that were presented to a subgroup of therapist users through cognitive walkthroughs. Meetings with this subgroup allowed workflow refinement to improve narrative transfers. Finally, to facilitate using these workflows, we developed PDF manuals that are being refined in subsequent phases of the implementation project (not mentioned in this paper).

CONCLUSIONS

The Discover, Design and Build, and Test framework can be a useful tool for understanding user needs in complex EBPI interventions and designing solutions to user-identified usability issues. Building on this work, an iterative evaluation of the 3 solution workflows and accompanying manuals with therapists and patients is underway as part of a nationwide WET implementation in telehealth settings.

Creating an Intercultural User-Centric Design for a Digital Sexual Health Education App for Young Women in Resource-Poor Regions of Kenya: Qualitative Self-Extended Double Diamond Model for Requirements Engineering Analysis

BACKGROUND

The stigmatization around sexual health due to culture, religious traditions, and norms leads to myths and a lack of available information and resources related to universal access to sexual and reproductive health services. Additional sexual health education not being part of the Kenya school curriculum leads to insufficient access to knowledge about safe contraception, menstruation, and female genital mutilation. A digital app could support and provide education and information for universal equal access, addressing United Nations Sustainable Development Goals 3, 4, and 5.

OBJECTIVE

The study targeted the requirements for establishing a reusable framework to develop a successful accessible web-based sexual health education app and the behavioral intention to use it to obtain sexual health information by mainly young women in Kenya.

METHODS

The double diamond model, with a problem room including the discover and define phases, enriched with cultural aspects and modeled to a self-expanded intercultural research model was used in a user-centered design thinking approach to develop a framework for requirements engineering analysis. For problem identification, semistructured pilot phase interviews based on Consolidated Criteria for Reporting Qualitative Research guidelines were conducted, followed by expert interviews for qualitative content analysis. A sample size of 12 pilot phase interviews and 5 expert interviews was determined using data saturation. The responses were coded and analyzed according to the affinity mapping method.

RESULTS

The requirements engineering analysis showed potential enablers of and barriers to the use of a digital sexual health education app. Through this qualitative study, a conservative cultural background, classic text communication, and the influence of social affiliation within society were identified as barriers, which should be enhanced through visual and auditory channels as well as a fictional character in the app.

CONCLUSIONS

The developed intercultural research model provides an impetus to providing digital sexual health education, integrating culture-specific aspects in the design process, while focusing on cultural and religious stigmata. The reusable framework enables identifying and overcoming hurdles in providing information about taboo and intimate topics. The overall use of online education tools focusing on intimate topics is correlated with accessibility and understanding specific cultural needs while delivering content on a basic and comprehensive level. It helps the target user from a social conservative background and in resource-poor circumstances to benefit from a digital educational solution.

Design, Development, and Evaluation of an mHealth App for Reporting of Side Effects During Cytostatic Treatment: Usability Test and Interview Study

BACKGROUND

Using mobile health (mHealth) interventions such as smartphone apps to deliver health services is an opportunity to engage patients more actively in their own treatment. Usability tests allow for the evaluation of a service by testing it out on the relevant users before implementation in clinical practice.

OBJECTIVE

The objective of this study was to design, develop, and evaluate the user interface of an app that would aid patients with cancer in reporting a more comprehensive summary of their side effects.

METHODS

The usability test was conducted by exposing patients with cancer to a prototype of an mHealth app that allowed for reporting of side effects from a chemotherapy regimen. After solving a set of 13 tasks, the test participants completed a system usability scale questionnaire and were interviewed using a semistructured interview guide. The interviews were later transcribed and analyzed.

RESULTS

The 10 test participants had a mean age of 56.5 (SD 7.11) years. The mean total task completion time for the task-solving session was 240.15 (SD 166.78) seconds. The calculated system usability scale score was 92.5. Most participants solved most of the tasks without any major issues. A minority reported having difficulties using apps on smartphones in general. One patient never achieved a meaningful interaction with our app prototype. Most of those who engaged with the app approved of features that calmed them down, made them more empowered, and put them in control. They preferred to report on side effects in a detailed and concise manner. App features that provided specific advice could provoke both fear and rational action.

CONCLUSIONS

The user tests uncovered design flaws that allowed for subsequent refining of an app that has the potential to enhance the safety of patients undergoing home-based chemotherapy. However, a refined version of the app is unlikely to be of value to all patients. Some might not be able to use apps on smartphones in general, or their ability to use apps is impaired because of their disease. This finding should have implications for health care providers' overall design of their follow-up service as the service must allow for all the patients to receive safe treatment whether they can use an mHealth app or not.

Integrating Clinical Decision Support Into Electronic Health Record Systems Using a Novel Platform (EvidencePoint): Developmental Study

BACKGROUND

Through our work, we have demonstrated how clinical decision support (CDS) tools integrated into the electronic health record (EHR) assist providers in adopting evidence-based practices. This requires confronting technical challenges that result from relying on the EHR as the foundation for tool development; for example, the individual CDS tools need to be built independently for each different EHR.

OBJECTIVE

The objective of our research was to build and implement an EHR-agnostic platform for integrating CDS tools, which would remove the technical constraints inherent in relying on the EHR as the foundation and enable a single set of CDS tools that can work with any EHR.

METHODS

We developed EvidencePoint, a novel, cloud-based, EHR-agnostic CDS platform, and we will describe the development of EvidencePoint and the deployment of its initial CDS tools, which include EHR-integrated applications for clinical use cases such as prediction of hospitalization survival for patients with COVID-19, venous thromboembolism prophylaxis, and pulmonary embolism diagnosis.

RESULTS

The results below highlight the adoption of the CDS tools, the International Medical Prevention Registry on Venous Thromboembolism-D-Dimer, the Wells' criteria, and the Northwell COVID-19 Survival (NOCOS), following development, usability testing, and implementation. The International Medical Prevention Registry on Venous Thromboembolism-D-Dimer CDS was used in 5249 patients at the 2 clinical intervention sites. The intervention group tool adoption was 77.8% (4083/5249 possible uses). For the NOCOS tool, which was designed to assist with triaging patients with COVID-19 for hospital admission in the event of constrained hospital resources, the worst-case resourcing scenario never materialized and triaging was never required. As a result, the NOCOS tool was not frequently used, though the EvidencePoint platform's flexibility and customizability enabled the tool to be developed and deployed rapidly under the emergency conditions of the pandemic. Adoption rates for the Wells' criteria tool will be reported in a future publication.

CONCLUSIONS

The EvidencePoint system successfully demonstrated that a flexible, user-friendly platform for hosting CDS tools outside of a specific EHR is feasible. The forthcoming results of our outcomes analyses will demonstrate the adoption rate of EvidencePoint tools as well as the impact of behavioral economics "nudges" on the adoption rate. Due to the EHR-agnostic nature of EvidencePoint, the development process for additional forms of CDS will be simpler than traditional and cumbersome IT integration approaches and will benefit from the capabilities provided by the core system of EvidencePoint.

Coagulation Management of Critically Bleeding Patients With Viscoelastic Testing Presented as a 3D-Animated Blood Clot (The Visual Clot): Randomized Controlled High-Fidelity Simulation Study

BACKGROUND

Guidelines recommend using viscoelastic coagulation tests to guide coagulation management, but interpreting the results remains challenging. Visual Clot, a 3D animated blood clot, facilitates interpretation through a user-centered and situation awareness-oriented design.

OBJECTIVE

This study aims to compare the effects of Visual Clot versus conventional viscoelastic test results (rotational thrombelastometry [ROTEM] temograms) on the coagulation management performance of anesthesia teams in critical bleeding situations.

METHODS

We conducted a prospective, randomized, high-fidelity simulation study in which anesthesia teams (consisting of a senior anesthesiologist, a resident anesthesiologist, and an anesthesia nurse) managed perioperative bleeding scenarios. Teams had either Visual Clot or ROTEM temograms available to perform targeted coagulation management. We analyzed the 15-minute simulations with post hoc video analysis. The primary outcome was correct targeted coagulation therapy. Secondary outcomes were time to targeted coagulation therapy, confidence, and workload. In addition, we have conducted a qualitative survey on user acceptance of Visual Clot. We used Poisson regression, Cox regression, and mixed logistic regression models, adjusted for various potential confounders, to analyze the data.

RESULTS

We analyzed 59 simulations. Teams using Visual Clot were more likely to deliver the overall targeted coagulation therapy correctly (rate ratio 1.56, 95% CI 1.00-2.47; P=.05) and administer the first targeted coagulation product faster (hazard ratio 2.58, 95% CI 1.37-4.85; P=.003). In addition, participants showed higher decision confidence with Visual Clot (odds ratio 3.60, 95% CI 1.49-8.71; P=.005). We found no difference in workload (coefficient -0.03, 95% CI -3.08 to 2.88; P=.99).

CONCLUSIONS

Using Visual Clot led to a more accurate and faster-targeted coagulation therapy than using ROTEM temograms. We suggest that relevant viscoelastic test manufacturers consider augmenting their complex result presentation with intuitive, easy-to-understand visualization to ease users' burden from unnecessary cognitive load and enhance patient care.

Online sleep diaries: considerations for system development and recommendations for data management

STUDY OBJECTIVES

To present development considerations for online sleep diary systems that result in robust, interpretable, and reliable data; furthermore, to describe data management procedures to address common data entry errors that occur despite those considerations.

METHODS

The online sleep diary capture component of the Sleep Healthy Using the Internet (SHUTi) intervention has been designed to promote data integrity. Features include diary entry restrictions to limit retrospective bias, reminder prompts and data visualizations to support user engagement, and data validation checks to reduce data entry errors. Despite these features, data entry errors still occur. Data management procedures relying largely on programming syntax to minimize researcher effort and maximize reliability and replicability. Presumed data entry errors are identified where users are believed to have incorrectly selected a date or AM versus PM on the 12-hour clock. Following these corrections, diaries are identified that have unresolvable errors, like negative total sleep time.

RESULTS

Using the example of one of our fully-powered, U.S. national SHUTi randomized controlled trials, we demonstrate the application of these procedures: of 45,598 total submitted diaries, 487 diaries (0.01%) required modification due to date and/or AM/PM errors and 27 diaries (<0.001%) were eliminated due to unresolvable errors.

CONCLUSION

To secure the most complete and valid data from online sleep diary systems, it is critical to consider the design of the data collection system and to develop replicable processes to manage data.

CLINICAL TRIAL REGISTRATION

Sleep Healthy Using The Internet for Older Adult Sufferers of Insomnia and Sleeplessness (SHUTiOASIS); https://clinicaltrials.gov/ct2/show/NCT03213132; ClinicalTrials.gov ID: NCT03213132.

Coadaptation Between Smart Technologies and Older Adults Over Time: Protocol for a Scoping Review

BACKGROUND

The Internet of Things (IoT) has gained significant attention due to advancements in technology and has potential applications in meeting the needs of an aging population. Smart technologies, a subset of IoT, can support older adults in aging in place, promoting independent living and improving their quality of life. However, there is a lack of research on how older adults and smart technologies coadapt over time to maximize their benefits and sustain adoption.

OBJECTIVE

We will aim to comprehensively review and analyze the existing scientific literature pertaining to the coadaptation between smart technologies and older adults. The primary focus will be to investigate the extent and nature of this coadaptation process and explore how older adults and technology coevolve over time to enhance older adults' experience with technology.

METHODS

This scoping review will follow the methodology outlined in the Joanna Briggs Institute Reviewer's Manual and adhere to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews) guidelines for reporting. Peer-reviewed articles will be searched in databases like Ovid MEDLINE, OVID Embase, PEDro, OVID PsycINFO, EBSCO CINAHL, the Cochrane Library, Scopus, IEEE Xplore, Web of Science, and Global Index Medicus. The research team will create a data extraction form covering study characteristics, participant characteristics, underlying models and frameworks, research findings, implications for technology coadaptation, and any identified study limitations. A directed content analysis approach will be used, incorporating the Selection, Optimization, and Compensation framework and Sex- and Gender-Based Analysis Plus theoretical framework.

RESULTS

The results of this study are expected in January 2024.

CONCLUSIONS

This scoping review endeavors to present a thorough overview of the available evidence concerning how smart technologies interact with older adults over an extended period. The insights gained from this review will lay the groundwork for a research program that explores how older adults adapt to and use smart technologies throughout their lives, ultimately leading to improved user satisfaction and experience and facilitating aging in place with tailored support and user-centered design principles.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/51129.

Evaluating the Acceptability and Feasibility of a Sexual Health-Focused Contraceptive Decision Aid for Diverse Young Adults: User-Centered Usability Study

BACKGROUND

Young adults with low sexual health literacy levels may find it difficult to make informed decisions about contraceptive methods. We developed and pilot-tested a web-based decision aid-Healthy Sex Choices-designed to support diverse young adults with their contraceptive decision-making.

OBJECTIVE

This pilot study aimed to evaluate whether the Healthy Sex Choices decision aid is acceptable and feasible to patients and clinicians.

METHODS

We used the Ottawa Decision Support Framework and the International Patient Decision Aid Standards to develop and pilot the decision tool. We first conducted a needs assessment with our advisory panel (5 clinicians and 2 patients) that informed decision aid development. All panelists participated in semistructured interviews about their experience with contraceptive counseling. Clinicians also completed a focus group session centered around the development of sex education content for the tool. Before commencing the pilot study, 5 participants from ResearchMatch (Vanderbilt University Medical Center) assessed the tool and suggested improvements.

RESULTS

Participants were satisfied with the tool, rating the acceptability as "good." Interviewees revealed that the tool made contraceptive decision-making easier and would recommend the tool to a family member or friend. Participants had a nonsignificant change in knowledge scores (53% before vs 45% after; P=.99). Overall, decisional conflict scores significantly decreased (16.1 before vs 2.8 after; P<.001) with the informed subscale (patients feeling more informed) having the greatest decline (23.1 vs 4.7; mean difference 19.0, SD 27.1). Subanalyses of contraceptive knowledge and decisional conflict illustrated that participants of color had lower knowledge scores (48% vs 55%) and higher decisional conflict (20.0 vs 14.5) at baseline than their white counterparts.

CONCLUSIONS

Participants found Healthy Sex Choices to be acceptable and reported reduced decisional conflict after using the tool. The development and pilot phases of this study provided a foundation for creating reproductive health decision aids that acknowledge and provide guidance for diverse patient populations.

Evaluating and refining a pain quality information visualization tool with patients and interpreters to facilitate pain assessment in primary care settings

Information visualization (InfoViz) tools offer a potential solution to pain communication challenges. Incongruencies in communication styles between patients with limited English proficiency (LEP), interpreters, and providers contribute to significant disparities in pain care and outcomes. This study's purpose is to evaluate and refine a culturally appropriate InfoViz pain quality assessment tool for LEP Hmong patients. We conducted a three-part iterative user-centered study with LEP Hmong, bilingual Hmong, and Hmong interpreters with (1) participatory design sessions to evaluate and refine pain infographics for inclusion on the tool, (2) card-sorting to organize the infographics to match the mental models of LEP patients, and (3) a tool assessment to identify which tool accurately represented LEP patients' mental models and was preferred in clinical settings. Fifty-five participants provided three common themes for pain infographics refinement: culturally-relevant colors, infographics resembling human anatomy, and action-specific squiggle lines. The card-sorting sessions revealed three organizational themes: sensation (n = 15; 71.4%), localization (n = 6; 28.6%), and severity of pain quality (n = 5; 24.3%). Most participants selected the localization as the most accurate tool and preferred it in clinical settings. Using a multi-step, user-centered approach resulted in a culturally appropriate pain InfoViz tool for LEP Hmong patients.

Cerner Millennium's Care Pathways for Specialty Care Referrals: Provider and Nurse Experiences, Perceptions, and Recommendations for Improvements

BACKGROUND

Using structured templates to guide providers in communicating key information in electronic referrals is an evidence-based practice for improving care quality. To facilitate referrals in Veterans Health Administration's (VA) Cerner Millennium electronic health record, VA and Cerner have created "Care Pathways"-templated electronic forms, capturing needed information and prompting ordering of appropriate pre-referral tests.

OBJECTIVE

To inform their iterative improvement, we sought to elicit experiences, perceptions, and recommendations regarding Care Pathways from frontline clinicians and staff in the first VA site to deploy Cerner Millennium.

DESIGN

Qualitative interviews, conducted 12-20 months after Cerner Millennium deployment.

PARTICIPANTS

We conducted interviews with primary care providers, primary care registered nurses, and specialty providers requesting and/or receiving referrals.

APPROACH

We used rapid qualitative analysis. Two researchers independently summarized interview transcripts with bullet points; summaries were merged by consensus. Constant comparison was used to sort bullet points into themes. A matrix was used to view bullet points by theme and participant.

RESULTS

Some interviewees liked aspects of the Care Pathways, expressing appreciation of their premise and logic. However, interviewees commonly expressed frustration with their poor usability across multiple attributes. Care Pathways were reported as being inefficient; lacking simplicity, naturalness, consistency, and effective use of language; imposing an unacceptable cognitive load; and not employing forgiveness and feedback for errors. Specialists reported not receiving the information needed for referral triaging.

CONCLUSIONS

Cerner Millennium's Care Pathways, and their associated organizational policies and processes, need substantial revision across several usability attributes. Problems with design and technical limitations are compounding challenges in using standardized templates nationally, across VA sites having diverse organizational and contextual characteristics. VA is actively working to make improvements; however, significant additional investments are needed for Care Pathways to achieve their intended purpose of optimizing specialty care referrals for Veterans.

A novel mobile health application to support cancer surveillance needs of patients and families with cancer predisposition syndromes

BACKGROUND

At least 5%-10% of malignancies occur secondary to an underlying cancer predisposition syndrome (CPS). For these families, cancer surveillance is recommended with the goal of identifying malignancy earlier, in a presumably more curable form. Surveillance protocols, including imaging studies, bloodwork, and procedures, can be complex and differ based on age, gender, and syndrome, which adversely affect adherence. Mobile health (mHealth) applications (apps) have been utilized in oncology and could help to facilitate adherence to cancer surveillance protocols.

METHODS

Applying a user-centered mobile app design approach, patients with a CPS and/or primary caregivers were interviewed to identify current methods for care management and barriers to compliance with recommended surveillance protocols. Broad themes from these interviews informed the design of the mobile app, HomeTown, which was subsequently evaluated by usability experts. The design was then converted into software code in phases, evaluated by patients and caregivers in an iterative fashion. User population growth and app usage data were assessed.

RESULTS

Common themes identified included general distress surrounding surveillance protocol scheduling and results, difficulty remembering medical history, assembling a care team, and seeking resources for self-education. These themes were translated into specific functional app features, including push reminders, syndrome-specific surveillance recommendations, ability to annotate visits and results, storage of medical histories, and links to reliable educational resources.

CONCLUSIONS

Families with CPS demonstrate a desire for mHealth tools to facilitate adherence to cancer surveillance protocols, reduce related distress, relay medical information, and provide educational resources. HomeTown may be a useful tool for engaging this patient population.

User Perceptions of Visual Blood: An International Mixed Methods Study on Novel Blood Gas Analysis Visualization

Blood gas analysis plays a central role in modern medicine. Advances in technology have expanded the range of available parameters and increased the complexity of their interpretation. By applying user-centered design principles, it is possible to reduce the cognitive load associated with interpreting blood gas analysis. In this international, multicenter study, we explored anesthesiologists' perspectives on Visual Blood, a novel visualization technique for presenting blood gas analysis results. We conducted interviews with participants following two computer-based simulation studies, the first utilizing virtual reality (VR) (50 participants) and the second without VR (70 participants). Employing the template approach, we identified key themes in the interview responses and formulated six statements, which were rated using Likert scales from 1 (strongly disagree) to 5 (strongly agree) in an online questionnaire. The most frequently mentioned theme was the positive usability features of Visual Blood. The online survey revealed that participants found Visual Blood to be an intuitive method for interpreting blood gas analysis (median 4, interquartile range (IQR) 4-4, p < 0.001). Participants noted that minimal training was required to effectively learn how to interpret Visual Blood (median 4, IQR 4-4, p < 0.001). However, adjustments are necessary to reduce visual overload (median 4, IQR 2-4, p < 0.001). Overall, Visual Blood received a favorable response. The strengths and weaknesses derived from these data will help optimize future versions of Visual Blood to improve the presentation of blood gas analysis results.

User-Centered Development of a Mobile App to Assess the Quality of Life of Patients With Cancer: Iterative Investigation and Usability Testing

BACKGROUND

The treatment for cancer can have a negative impact not only on physical well-being but also on mental health and the quality of life (QoL). Health apps enable the monitoring of different parameters, but to date, there are only few that support patients with cancer and none that focuses on the assessment of QoL. Furthermore, patients as stakeholders are often only integrated at the late stage of the development process, if at all.

OBJECTIVE

The aim of this research was to develop and evaluate a smartphone app (Lion-App) to enable patients with cancer to autonomously measure the QoL with an iterative, user-centered approach.

METHODS

Patients with cancer were involved in a 3-stage process from conceptualization to the point when the app was available on the tester's private device. First, focus groups with members (N=21) of cancer support groups were conducted to understand their expectations and needs. Thereafter, individual tests were performed. After developing a prototype that incorporated findings from the focus groups, a second test cycle was conducted, followed by a beta test lasting 2 months. In our app, the QoL can be assessed via a patient diary and an integrated questionnaire. Through all stages, usability was evaluated using the modular extended version of the User Experience Questionnaire (UEQ+), including the calculation of a key performance indicator (KPI). If possible, the impact of sex on the results was evaluated. As part of the beta test, usage rates as well as age-dependent differences were also assessed.

RESULTS

A total of 21 participants took part in the initial 3 focus groups. In the subsequent usability testing (N=18), 17 (94%) participants rated their impression through the UEQ+, with a mean KPI of 2.12 (SD 0.64, range: -3 to 3). In the second usability test (N=14), the mean KPI increased to 2.28 (SD=0.49). In the beta test, the usage rate of 19 participants was evaluated, of whom 14 (74%) also answered the UEQ+ (mean KPI 1.78, SD 0.84). An influence of age on the number of questionnaire responses in Lion-App was observed, with a decrease in responses with increasing age (P=.02). Sex-dependent analyses were only possible for the first usability test and the beta test. The main adjustments based on user feedback were a restructuring of the diary as well as integration of a shorter questionnaire to assess the QoL.

CONCLUSIONS

The iterative, user-centered approach for development and usability testing resulted in positive evaluations of Lion-App. Our app was rated as suitable for everyday use to monitor the QoL of patients with cancer. Initial results indicated that the sex and age of participants seem to play only a minor role.

Usability and Feasibility Testing of an Atrial Fibrillation Educational Website with Patients Referred to an Atrial Fibrillation Specialty Clinic

BACKGROUND

The purpose of this study was to design, usability test, and explore the feasibility of a web-based educational platform/intervention for patients with atrial fibrillation (AF) as part of their virtual AF care.

METHODS

Participants were patients attending a specialized AF clinic. The multiple mixed-methods design included website design, think-aloud usability test, 1-month unstructured pre-testing analysis using Google Analytics, follow-up interviews, and a non-randomized one-group feasibility test using pre/post online surveys and Google Analytics.

RESULTS

Usability testing participants (n = 2) guided adjustments for improving navigation. Pre-testing participants' (n = 9) website activity averaged four sessions (SD = 2.6) at 10 (SD 8) minutes per session during a 1-month study period. In the feasibility test, 30 patients referred to AF specialty clinic care completed the baseline survey, and 20 of these completed the 6-month follow-up survey. A total of 19 patients accessed the website over the 6 months, and all 30 participants were sent email prompts containing information from the website. Health-related quality of life, treatment satisfaction, household activity, and AF knowledge scores were higher at follow-up than baseline. There was an overall downward trend in self-reported healthcare utilization at follow-up.

CONCLUSIONS

Access to a credible education website for patients with AF has great potential to complement virtual and hybrid models of care.

Dashboard of Short-Term Postoperative Patient Outcomes for Anesthesiologists: Development and Preliminary Evaluation

BACKGROUND

Anesthesiologists require an understanding of their patients' outcomes to evaluate their performance and improve their practice. Traditionally, anesthesiologists had limited information about their surgical outpatients' outcomes due to minimal contact post discharge. Leveraging digital health innovations for analyzing personal and population outcomes may improve perioperative care. BC Children's Hospital's postoperative follow-up registry for outpatient surgeries collects short-term outcomes such as pain, nausea, and vomiting. Yet, these data were previously not available to anesthesiologists.

OBJECTIVE

This quality improvement study aimed to visualize postoperative outcome data to allow anesthesiologists to reflect on their care and compare their performance with their peers.

METHODS

The postoperative follow-up registry contains nurse-reported postoperative outcomes, including opioid and antiemetic administration in the postanesthetic care unit (PACU), and family-reported outcomes, including pain, nausea, and vomiting, within 24 hours post discharge. Dashboards were iteratively co-designed with 5 anesthesiologists, and a department-wide usability survey gathered anesthesiologists' feedback on the dashboards, allowing further design improvements. A final dashboard version has been deployed, with data updated weekly.

RESULTS

The dashboard contains three sections: (1) 24-hour outcomes, (2) PACU outcomes, and (3) a practice profile containing individual anesthesiologist's case mix, grouped by age groups, sex, and surgical service. At the time of evaluation, the dashboard included 24-hour data from 7877 cases collected from September 2020 to February 2023 and PACU data from 8716 cases collected from April 2021 to February 2023. The co-design process and usability evaluation indicated that anesthesiologists preferred simpler designs for data summaries but also required the ability to explore details of specific outcomes and cases if needed. Anesthesiologists considered security and confidentiality to be key features of the design and most deemed the dashboard information useful and potentially beneficial for their practice.

CONCLUSIONS

We designed and deployed a dynamic, personalized dashboard for anesthesiologists to review their outpatients' short-term postoperative outcomes. This dashboard facilitates personal reflection on individual practice in the context of peer and departmental performance and, hence, the opportunity to evaluate iterative practice changes. Further work is required to establish their effect on improving individual and department performance and patient outcomes.

Usability evaluation of the Agente Escuta application: translational research

PURPOSE

To evaluate the usability and satisfaction of users with the interface of the 'Agente Escuta' application, in addition to identifying problems and possibilities for improvement.

METHODS

Descriptive exploratory translational study, characterized by a usability test with a quantitative and qualitative approach, subdivided into three stages: (I) prior evaluation of usability by 10 judges, including students, primary care professionals, professors and researchers in Information Technology and Speech Therapy; (II) evaluation of the application by the target audience, that is, community health agents from six municipalities in Rio Grande do Norte; (III) evaluation of the satisfaction of the agents who used the application in their work routine. The System Usability Scale and the Net Promoter Score were used, in addition to a qualitative evaluation of the opinions.

RESULTS

Usability was rated as excellent by judges, regardless of category. In the evaluation by community health agents, usability was considered good and there was no effect of the city of origin. It was found that the perception of the judges and the target audience were different, with a lower score for the participants in the second stage. However, most would give positive publicity to the product. The heuristic with the highest score was ease of memorization and participants in the third stage were interested in continuing to use the tool in practice, even after the end of the study.

CONCLUSION

The Agente Escuta prototype showed good usability and satisfaction and aspects that could be improved in future solutions were identified.

20 years of the Loughborough user centered assistive technology design process: has it made a difference?

An estimated 518 students were taught the Loughborough User Centered Assistive Technology design process, LUCAT, since 2000. Graduates were contacted via a professional networking website to take part in a survey with four being interviewed. The purpose of the survey was to find out if: 1) Did they still use any parts of the process; and 2) Where they had applied them. The respondents ranged from returning placement undergraduates to senior managers in research and development within major companies. From the 105 respondents 23% stated they used parts of the LUCAT process every working day, a further 25% once a week and 27% once a month. The elements of the process used were predominantly semi-structured interviews, concept generation, codesign, design presentation and feedback. Respondents highlighted the benefits of using this process including gaining insights from users, being time efficient, saving money, developing a relationship with end users and making a difference to the lives of people with disability. It was suggested some industries had still not incorporated a User Centered Design approach within New Product Development. The use of the process was shown to be useful beyond AT product design into most areas of design activity.

Design and usability evaluation of a mobile-based-self-management application for caregivers of children with severe burns

Paediatric burns are a major public health issue because of long-term physical, psychological and social consequences and the high cost of treatment. The aim of this study was to design and evaluate a mobile-based self-management application for caregivers of children with severe burns. A participatory design technique was employed to develop the Burn application, which included three main phases: the determination of application requirements, the design and evaluation of the low-fidelity prototype, and the design and evaluation of the high-fidelity prototypes. In the first phase, application requirements were determined via validated paper questionnaires using the Delphi technique. In the second step, a low-fidelity prototype was prepared using conceptual models and evaluated through a focus group with specialists. Seven specialists reviewed the application and evaluated how this prototype meets functional requirements and objectives. The third phase was performed in three stages. First, the high-fidelity prototype was designed and developed by the JAVA programming language. Second, a cognitive walk-through was carried out to show how users can interact with the mobile application and how it works. Third, this program was installed on the mobile phones of 28 caregivers of burned children, eight IT experts, and two general surgeries, and the prototype's usability was evaluated. In the present study, most caregivers of children with burns stated that after discharge, they face problems regarding infection control and wound care (4.07) and how to perform physical activity (4.12). User registration, educational materials, caregiver-clinician communication, chat box, and appointment booking, safe login were the most important characteristic of the Burn application. Mean usability evaluation scores were in the range of 7.92 ± 0.238 to 8.10 ± 0.103, which is considered at a "good" level. From the Burn program design experience, it can be concluded that co-design with health care specialists can significantly support and meet the specialists' and patients' needs and ensure the program's usefulness. In addition, application evaluation by users involved and not involved in the application design process can help enhance usability.

Food Access Support Technology (FAST): a Centralized City-Wide Platform for Rapid Response to Food Insecurity

BACKGROUND

Food access for patients remains a critical need for health systems to address given varying resource availability and inefficient coordination among health and food services.

AIM

Develop and evaluate the Food Access Support Technology (FAST), a centralized digital platform for food access that pairs health systems with food and delivery community-based organizations (CBOs).

SETTING AND PARTICIPANTS

Two health systems, 12 food partners, and 2 delivery partners in Philadelphia, PA.

PROGRAM DESCRIPTION

Using FAST, referrers can post requests for food delivery on recipients' behalf, which are reviewed and claimed by eligible CBOs that can prepare food boxes for delivery to people's homes.

PROGRAM EVALUATION

Between March 2021 and July 2022, FAST received 364 requests, representing 207 food insecure households in 51 postal codes. The platform facilitated the completion of 258 (70.9%) requests, with a median completion time of 5 (IQR 0-7) days and a median of only 1.5 days (IQR 0-5) for requests marked "urgent." Qualitative interviews with FAST end-users endorsed the usability of the FAST platform and its effectiveness in facilitating resource-sharing between partners.

DISCUSSION

Our findings suggest that centralized platforms can address household food insecurity by (1) streamlining partnerships between health systems and CBOs for food delivery and (2) facilitating the real-time coordination of resources among CBOs.

Graphical User Interface Development for a Hospital-Based Predictive Risk Tool: Protocol for a Co-Design Study

BACKGROUND

This co-design research method details the iterative process developed to identify health professional recommendations for the graphical user interface (GUI) of an artificial intelligence (AI)-enabled risk prediction tool. Driving the decision to include a co-design process is the belief that choices regarding the aesthetic and functionality of an intervention are best made by its intended users and that engaging these users in its design will promote the tool's adoption and use.

OBJECTIVE

The aim of this research is to identify health professional design and uptake recommendations for the GUI of an AI-enabled predictive risk tool.

METHODS

We will hold 3 research phases, each consisting of 2 workshops with health professionals, between mid-2023 and mid-2024. A total of 6 health professionals will be sought per workshop, resulting in a total enrollment of 36 health professionals at the conclusion of the research. A total of 7 workshop activities have been scheduled across the 3 workshops; these include context of use, notifiers, format, AI survey-Likert, prototype, AI survey-written, and testing. The first 6 of these activities will be repeated in each workshop to enable the iterative development and refinement of GUI. The last activity (testing) will be performed in the final workshop to examine health professionals' thoughts on the final GUI iteration. Qualitative and quantitative results data will be produced from tasks in each research activity. Qualitative data will be examined through inductive thematic analysis or deductive thematic analysis in accordance with the Nonadoption, Abandonment, and Challenges to the Scale-up, Spread, and Sustainability (NASSS) framework; visual data will be examined in accordance with "framework of interactivity;" and quantitative data will be examined using descriptive statistics.

RESULTS

Project registration with the Australia and New Zealand Clinical Trial Registry has been requested (#384098). Finalized design recommendations are expected in early to mid-2024, with a results manuscript to be submitted in mid-2024. This research method has human research ethics approval from the South Australian Department of Health and Wellbeing (#2022/HRE00131) as well as from the Human Research Ethics Committee of the University of South Australia (application ID#204143).

CONCLUSIONS

Understanding whether an intervention is needed in a particular situation is just the start; designing an intervention so that it is used within that situation is paramount. This co-design process engages end users to create a GUI that includes the aesthetic and functional details they need in a manner that aligns with their existing work practices. Indeed, interventions that fail to do this may be disliked, and at worst, they may be dangerous.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/47717.

Outcomes of End-User Testing of a Care Coordination Mobile App With Families of Children With Special Health Care Needs: Simulation Study

BACKGROUND

Care for children with special health care needs relies on a network of providers who work to address the medical, behavioral, developmental, educational, social, and economic needs of the child and their family. Family-directed, manually created visual depictions of care team composition (ie, care mapping) and detailed note-taking curated by caregivers (eg, care binders) have been shown to enhance care coordination for families of these children, but they are difficult to implement in clinical settings owing to a lack of integration with electronic health records and limited visibility of family-generated insights for care providers. Caremap is an electronic health record-integrated digital personal health record mobile app designed to integrate the benefits of care mapping and care binders. Currently, there is sparse literature describing end-user participation in the co-design of digital health tools. In this paper, we describe a project that evaluated the usability and proof of concept of the Caremap app through end-user simulation.

OBJECTIVE

This study aimed to conduct proof-of-concept testing of the Caremap app to coordinate care for children with special health care needs and explore early end-user engagement in simulation testing. The specific aims included engaging end users in app co-design via app simulation, evaluating the usability of the app using validated measures, and exploring user perspectives on how to make further improvements to the app.

METHODS

Caregivers of children with special health care needs were recruited to participate in a simulation exercise using Caremap to coordinate care for a simulated case of a child with complex medical and behavioral needs. Participants completed a postsimulation questionnaire adapted from 2 validated surveys: the Pediatric Integrated Care Survey (PICS) and the user version of the Mobile Application Rating Scale (uMARS). A key informant interview was also conducted with a liaison to Spanish-speaking families regarding app accessibility for non-English-speaking users.

RESULTS

A Caremap simulation was successfully developed in partnership with families of children with special health care needs. Overall, 38 families recruited from 19 different US states participated in the simulation exercise and completed the survey. The average rating for the survey adapted from the PICS was 4.1 (SD 0.82) out of 5, and the average rating for the adapted uMARS survey was 4 (SD 0.83) out of 5. The highest-rated app feature was the ability to track progress toward short-term, patient- and family-defined care goals.

CONCLUSIONS

Internet-based simulation successfully facilitated end-user engagement and feedback for a digital health care coordination app for families of children with special health care needs. The families who completed simulation with Caremap rated it highly across several domains related to care coordination. The simulation study results elucidated key areas for improvement that translated into actionable next steps in app development.