Assessing usability of electronic patient-reported outcome measures in older people with and without a rare dermatologic disorder

BACKGROUND

Robust and well-defined data collection is important when using electronic patient-reported outcome measures (ePROMs) in clinical studies. Questions have been raised as to whether older age may be a barrier to data collection due to patients' unfamiliarity with electronic devices. Older adults may also have underlying health conditions that affect their ability to fill out patient-reported outcome measures (PROMs) on electronic devices. The aim of this observational, qualitative research study was to evaluate the usability of electronic PROMs (ePROMs) on a tablet and smartphone in older participants with and without bullous pemphigoid (BP).

METHODS

Older people with and without BP were recruited in the US and France. They participated in 60-min in-person interviews, with moderators observing their completion of various tasks, including ePROMs, using a tablet and smartphone. Participants were scored on ease of task completion using a scale from 1 to 5.

RESULTS

A total of 12 participants were recruited (≥65 years old; six each with and without BP [all participants without BP were ≥75 years old]). Most participants (83%) could easily and confidently perform most assigned tasks on both the tablet and smartphone. Although select tasks required assistance, all participants were eventually able to complete all tasks. Overall, ePROM usability did not correlate with age, sex, country, or disease state. Feedback on the general usability of both electronic devices was largely positive, and most participants (n = 11; 92%) were willing to use them. Participants were generally pleased with the training modules offered on both devices, describing the training as sufficient, straightforward, and helpful. In total, 25 usability issues were identified, which fell into three categories: incomplete instructions, unclear language, and insufficient technical/visual design. Participants provided feedback on how to enhance device usability.

CONCLUSIONS

The results suggest that older people can confidently use a tablet or smartphone for ePROM completion, particularly with appropriate training. ePROMs should be designed with the needs of the target patient population in mind. These results can be leveraged to improve clinical data recording, optimize device usability, and enhance the user experience for older people and those with functional or physical limitations.

Electronic data capture in resource-limited settings using the lightweight clinical data acquisition and recording system

Our prototype system designed for clinical data acquisition and recording of studies is a novel electronic data capture (EDC) software for simple and lightweight data capture in clinical research. Existing software tools are either costly or suffer from very limited features. To overcome these shortcomings, we designed an EDC software together with a mobile client. We aimed at making it easy to set-up, modifiable, scalable and thereby facilitating research. We wrote the software in R using a modular approach and implemented existing data standards along with a meta data driven interface and database structure. The prototype is an adaptable open-source software, which can be installed locally or in the cloud without advanced IT-knowledge. A mobile web interface and progressive web app for mobile use and desktop computers is added. We show the software's capability, by demonstrating four clinical studies with over 1600 participants and 679 variables per participant. We delineate a simple deployment approach for a server-installation and indicate further use-cases. The software is available under the MIT open-source license. Conclusively the software is versatile, easily deployable, highly modifiable, and extremely scalable for clinical studies. As an open-source R-software it is accessible, open to community-driven development and improvement in the future.

Digital measures in epilepsy in low-resourced environments

INTRODUCTION

Digital measures and digital health-care delivery have been rarely implemented in lower-and-middle-income countries (LMICs), contributing to worsening global disparities and inequities. Sustainable ways to implement and use digital approaches will help to improve time to access, management, and quality of life in persons with epilepsy, goals that remain unreachable in under-resourced communities. As under-resourced environments differ in human and economic resources, no one approach will be appropriate to all LMICs.

AREAS COVERED

Digital health and tools to monitor and measure digital endpoints and metrics of quality of life will need to be developed or adapted to the specific needs of under-resourced areas. Portable technologies may partially address the urban-rural divide. Careful delineation of stakeholders and their engagement and alignment in all efforts is critically important if these initiatives are to be successfully sustained. Privacy issues, neglected in many regions globally, must be purposefully addressed.

EXPERT OPINION

Epilepsy care in under-resourced environments has been limited by the lack of relevant technologies for diagnosis and treatment. Digital biomarkers, and investigative technological advances, may finally make it feasible to sustainably improve care delivery and ultimately quality of life including personalized epilepsy care.

How patients with insomnia interpret and respond to the consensus sleep diary: a cognitive interview study

OBJECTIVE/BACKGROUND

The Consensus Sleep Diary (CSD) is widely used to assess subjective sleep. Psychometric evaluations and focus-groups support its validity and clinical usefulness, but further research into its validity is needed. The aim of the study was to evaluate a Swedish translation of the CSD regarding test content and response processes in patients with insomnia.

PATIENTS/METHODS

In connection with translating the CSD into Swedish, we used cognitive interviewing to evaluate test content and the response process, that is, how people make decisions when responding to survey items. Cognitive interviews were conducted with 13 primary health care patients with insomnia disorder (mean age, 49 years; SD 15.5). Iterative, reparative analysis was used to investigate test content. Descriptive deductive analysis was used to investigate interview transcripts for the themes of the cognitive model: comprehension, retrieval, decision process, and judgement. Together, the themes explain the response process when responding to a patient-reported outcome measure.

RESULTS

The overall comprehension of the CSD could be affected by poor adherence to the instructions (comprehension). Patients had difficulty with recall if they did not complete the diary immediately in the morning and just before bedtime (retrieval). They could have problems deciding how to respond to certain items because they imbued sleep-related concepts with extra meaning (decision process), and had trouble finding response alternatives nuanced enough to describe their experience of sleep and tiredness (judgement).

CONCLUSIONS

This study contributes knowledge on how the instrument is perceived and used by care-seeking patients with insomnia. In this context, the CSD exhibits known flaws such as memory lapses if the diary is not filled in directly in the morning. To increase the accuracy of patients' responses, therapists should support patients in reading the instructions.

Patient-targeted education (ePRO-E) to increase ePRO intent within an Alliance clinical trial (A221805-SI1)

BACKGROUND

The Patient Cloud ePRO app was adopted by the National Cancer Institute National Clinical Trials Network (NCTN) to facilitate capturing electronic patient-reported (ePRO) outcome data, but use has been low. The study objectives were to test whether a patient-targeted ePRO educational resource (ePRO-E) would increase ePRO intent (number of users) and improve data quality (high quality: ≥80% of the required surveys submitted) within an ongoing NCTN study.

METHODS

The ePRO-E intervention, a patient-targeted educational resource (written material and 6-minute animated YouTube video), was designed to address ePRO barriers. ePRO intent and data quality were compared between 2 groups (N = 69): a historical control group and a prospectively recruited intervention group exposed to ePRO-E. Covariates included technology attitudes, age, sex, education, socioeconomic status, and comorbidity.

RESULTS

Intervention group ePRO intent (78.8%) was statistically significantly higher than historical control group intent (47.1%) (P = .03). Patients choosing ePRO versus paper surveys had more positive and higher technology attitudes scores (P = .03). The odds of choosing ePRO were 4.7 times higher (95% Confidence Interval [CI] = 1.2 to 17.8) (P = .02) among intervention group patients and 5.2 times higher (95% CI = 1.3 to 21.6) (P = .02) among patients with high technology attitudes scores, after controlling for covariates. However, the 80% submission rate (percentage submitting ≥80% of required surveys) in the ePRO group (30.6%) was statistically significantly lower than in the paper group (57.9%) (P = .05).

CONCLUSIONS

ePRO-E exposure increased ePRO intent. High technology attitudes scores were associated with ePRO selection. Since the ePRO survey submission rate was low, additional strategies are needed to promote high-quality data submission.

Electronic Patient-Reported Outcome Data Collection Systems in Oncology Clinical Trials: A Survey of Clinical Research Professionals (an Alliance Study)

PURPOSE

To describe clinical research professionals (CRPs)' experiences with electronic patient-reported outcome (ePRO) data collection systems in oncology clinical trials and identify correlates of CRPs' attitude toward technology.

METHODS

An online survey was conducted among 210 CRPs from 125 National Cancer Institute-funded research sites. Measures included CRPs' demographic characteristics, working years, employment locations, and previous experiences with various types of ePROs. Their attitude toward technology was measured by the Technology Attitude Scale-Adapted. The Wilcoxon signed-rank test was used to compare two subdomains of attitude (perceived usefulness [PU] and perceived ease of use [PEU]). Multiple linear regression was used to explore correlates of (1) overall attitude, (2) PU, and (3) PEU. The significance level was 5%.

RESULTS

Participants' median age was 41 years (range, 21-67). Most were female (90%) and White (82%). More than half of the participants had previous experiences with web-based ePROs using patients' own devices (72%) or site-/sponsor-provided on-site devices (eg, kiosks or tablets; 64%). CRPs who were 60 years or older (β = -0.32, P < .05) or worked for 10-20 years (β = -0.11, P < .05) had relatively negative attitudes, controlling for other factors. Previous experiences with more ePRO types were associated with more positive attitudes (β = 0.08, P = .02). Similar correlates were found with PU but not with PEU.

CONCLUSION

This study revealed that CRPs had various experiences with ePRO systems and attitudes toward technology. Age, working years, and previous experiences with ePROs were correlates of overall attitude toward technology and PU. These findings suggest necessary targeted training to facilitate ePRO use in oncology clinical trials by improving CRPs' awareness and attitude toward technology.

An Open-Source, Standard-Compliant, and Mobile Electronic Data Capture System for Medical Research (OpenEDC): Design and Evaluation Study

BACKGROUND

Medical research and machine learning for health care depend on high-quality data. Electronic data capture (EDC) systems have been widely adopted for metadata-driven digital data collection. However, many systems use proprietary and incompatible formats that inhibit clinical data exchange and metadata reuse. In addition, the configuration and financial requirements of typical EDC systems frequently prevent small-scale studies from benefiting from their inherent advantages.

OBJECTIVE

The aim of this study is to develop and publish an open-source EDC system that addresses these issues. We aim to plan a system that is applicable to a wide range of research projects.

METHODS

We conducted a literature-based requirements analysis to identify the academic and regulatory demands for digital data collection. After designing and implementing OpenEDC, we performed a usability evaluation to obtain feedback from users.

RESULTS

We identified 20 frequently stated requirements for EDC. According to the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 25010 norm, we categorized the requirements into functional suitability, availability, compatibility, usability, and security. We developed OpenEDC based on the regulatory-compliant Clinical Data Interchange Standards Consortium Operational Data Model (CDISC ODM) standard. Mobile device support enables the collection of patient-reported outcomes. OpenEDC is publicly available and released under the MIT open-source license.

CONCLUSIONS

Adopting an established standard without modifications supports metadata reuse and clinical data exchange, but it limits item layouts. OpenEDC is a stand-alone web app that can be used without a setup or configuration. This should foster compatibility between medical research and open science. OpenEDC is targeted at observational and translational research studies by clinicians.

Use of Patient-reported Outcome Measures and Patient-reported Experience Measures within Evaluation Studies of Telemedicine Applications: Systematic Review

Background

With the rise of digital health technologies and telemedicine, the need for evidence-based evaluation is growing. Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are recommended as an essential part of the evaluation of telemedicine. For the first time, a systematic review has been conducted to investigate the use of PROMs and PREMs in the evaluation studies of telemedicine covering all application types and medical purposes.

Objective

This study investigates the following research questions: in which scenarios are PROMs and PREMs collected for evaluation purposes, which PROM and PREM outcome domains have been covered and how often, which outcome measurement instruments have been used and how often, does the selection and quantity of PROMs and PREMs differ between study types and application types, and has the use of PROMs and PREMs changed over time.

Methods

We conducted a systematic literature search of the MEDLINE and Embase databases and included studies published from inception until April 2, 2020. We included studies evaluating telemedicine with patients as the main users; these studies reported PROMs and PREMs within randomized controlled trials, controlled trials, noncontrolled trials, and feasibility trials in English and German.

Results

Of the identified 2671 studies, 303 (11.34%) were included; of the 303 studies, 67 (22.1%) were feasibility studies, 70 (23.1%) were noncontrolled trials, 20 (6.6%) were controlled trials, and 146 (48.2%) were randomized controlled trials. Health-related quality of life (n=310; mean 1.02, SD 1.05), emotional function (n=244; mean 0.81, SD 1.18), and adherence (n=103; mean 0.34, SD 0.53) were the most frequently assessed outcome domains. Self-developed PROMs were used in 21.4% (65/303) of the studies, and self-developed PREMs were used in 22.3% (68/303). PROMs (n=884) were assessed more frequently than PREMs (n=234). As the evidence level of the studies increased, the number of PROMs also increased (τ=−0.45), and the number of PREMs decreased (τ=0.35). Since 2000, not only has the number of studies using PROMs and PREMs increased, but the level of evidence and the number of outcome measurement instruments used have also increased, with the number of PREMs permanently remaining at a lower level.

Conclusions

There have been increasingly more studies, particularly high-evidence studies, which use PROMs and PREMs to evaluate telemedicine. PROMs have been used more frequently than PREMs. With the increasing maturity stage of telemedicine applications and higher evidence level, the use of PROMs increased in line with the recommendations of evaluation guidelines. Health-related quality of life and emotional function were measured in almost all the studies. Simultaneously, health literacy as a precondition for using the application adequately, alongside proper training and guidance, has rarely been reported. Further efforts should be pursued to standardize PROM and PREM collection in evaluation studies of telemedicine.

Best Practices for Avoiding Paper Backup When Implementing Electronic Approaches to Patient-Reported Outcome Data Collection in Clinical Trials

Electronic data capture is fast becoming the preferred method of collecting patient-reported outcome (PRO) data in clinical trials. Data collection can be site-based (clinical study site), and typically collected on a tablet, or field-based (subject's typical environment such as home, school, or workplace), and most often accomplished with handheld devices, such as a smartphone. While site and study subject compliance with protocol-specific data collection procedures using these devices is critical to trial success, so is the robustness of the device hardware and the software these devices use to capture the trial data. Technology failures and/or site or subject resistance to the electronic data capture protocol may lead a subject to record data on paper, which can result in undesirable data challenges. As such, both site and subject compliance issues and technology-related factors must be anticipated to adhere to the ePRO data collection plan. The objective of this paper is to provide the technology industry's best practice recommendations for optimizing ePRO data collection in clinical trials by proposing the inclusion of a planned approach to data collection that includes viable electronic backup strategies so that defaulting to a paper-based backup becomes unnecessary.

Training on the Use of Technology to Collect Patient-Reported Outcome Data Electronically in Clinical Trials: Best Practice Recommendations from the ePRO Consortium

Electronic capture of patient-reported outcome (PRO) data has many advantages over paper-based data collection. Regulatory agencies have consistently supported the use of electronic PRO (ePRO) data capture and recommended participant and site staff training on the correct use of electronic data capture systems. The objective of this paper is to outline best practice recommendations for training end users, including site staff and study participants, on the use of ePRO technology in clinical trials to enable consistent, accurate, and complete data collection. Site personnel should be trained on study-specific as well as technology-specific topics and be given instructions on whom to contact to obtain technical support. Optimal training takes place over time using multiple modalities, including hands-on, face-to-face training at an investigator meeting or directly in the clinical site; remote training via webinar or teleconference; interactive on-demand self-paced-training via e-learning modalities; and supplemented by proxy training performed by study clinical research associates. Like site personnel training, study participants should be provided with individual, hands-on training by site staff at the initiation of the trial and in conjunction with interactive electronic training modules that can be accessed on-demand throughout the duration of the trial. The recommendations put forth in this paper provide a structured framework for the training that site personnel and study participants need to optimize the advantages trials can gain from using ePRO data collection systems.