Usability of an innovative and interactive electronic system for collection of patient-reported data in axial spondyloarthritis: comparison with the traditional paper-administered format

Barriers and facilitators to treat-to-target in axial spondyloarthritis in clinical practice: a mixed methods study

This study aimed to explore the perceptions of patients and rheumatologists about a treat-to-target (T2T) strategy in axial spondyloarthritis (axSpA) and identify the barriers and facilitators to its implementation in clinical practice. A mixed methods design was applied. Patients with axSpA who visited the outpatient clinic with active disease (AxSpA Disease Activity Score [ASDAS] ≥ 2.1), but did not receive a treatment adjustment, were identified. These patient cases were discussed in individual semi-structured interviews with the respective treating rheumatologists, and a subgroup of these patients was also interviewed. In parallel, all interviewed participants completed a quantitative survey. Qualitative and quantitative data were analysed thematically and descriptively, respectively. Twenty-three patients were discussed with 11 rheumatologists, and 16 of these patients were interviewed personally. Barriers to T2T included challenges in the measurement of inflammatory disease activity using the ASDAS, and numerous patient-related factors such as concern about treatment adaptations. The limited number of viable treatment options and scarce amount of evidence supporting T2T in axSpA, as well as logistical challenges, were additional obstacles. Facilitators included patients' broad knowledge about axSpA, rheumatologists' awareness of T2T recommendations, and positive doctor-patient relationships with the application of shared decision-making. Moreover, a supporting infrastructure, such as one with high accessibility to the outpatient clinic between scheduled visits, was considered necessary for the application of a T2T strategy. In conclusion, numerous barriers and facilitators to the implementation of a T2T strategy in axSpA are present, which need to be considered when applying this treatment approach in clinical practice.

Remote-Customized Telecontrol for Patients with Rheumatoid Arthritis: The iARPlus (Innovative Approach in Rheumatology) Initiative

Objective. Telecontrol approaches for rheumatoid arthritis (RA) management aim to enhance patient outcomes. This pilot study assessed whether the Rheumatoid Arthritis Impact of Disease (RAID) approach could be used during teleconsultations to monitor RA disease activity through a web-based platform called iARPlus (Innovative Approach in Rheumatology). Methods. Forty RA patients participated in two in-person visits (baseline and 12 months) and seven teleconsultations over 12 months, collected via the iARPlus portal and accessible through an internet browser. Disease activity, at baseline and follow-up, was measured using the Clinical Disease Activity Index (CDAI) and self-reported RAID scores throughout the study. The RAID approach, developed by the European Alliance of Associations for Rheumatology (EULAR), combines key patient-reported outcomes (PROs). Results. Nineteen patients (mean age: 49.3 years) were treated with Janus kinase inhibitors (JAKis), and 21 patients (mean age: 48.1 years) received adalimumab. All patients had active disease (mean CDAI 27.9 ± 4.8). Strong correlations were found between CDAI and RAID scores at baseline (ρ = 0.809, p < 0.0001) and at follow-up (ρ = 0.789, p < 0.0001). JAKi-treated patients showed greater reductions in RAID scores, pain relief, and higher rates of disease remission compared to adalimumab-treated patients. Conclusions. RAID scores were effective in teleconsultations for assessing RA disease activity. JAKi treatment resulted in better pain control and disease activity improvement compared to adalimumab. Further studies are needed to confirm the clinical and economic benefits of telecontrol for RA management.

Standardized Usage of Electronic Patient-Reported Outcome Measurements is Time-Efficient and Feasible

(1) Background: Digitization is of the utmost importance in improving the transfer of medical data. In order to emphasize the need for the greater implementation of digital solutions, we compared analog PROMs (aPROMs) to electronic PROMs (ePROMs) to emphasize the time benefits for clinical everyday life. (2) Methods: This prospective, observational study compared the evaluation of SF-36 in patients between 18 and 80 years old with musculoskeletal pathologies. We performed an age-independent and age-dependent analysis. (3) Results: After the import of aPROMs data, ePROMs took significantly less time (11.97 ± 3.00 min vs. 9.41 ± 3.12 min, p = 0.002, d = 0.797). There were no significant differences associated with age for aPROMs (7.23 ± 2.57 min vs. 8.38 ± 2.71 min, p = 0.061, d = -0.607) or ePROMs (8.72 ± 2.19 min vs. 10.09 ± 3.80 min, p = 0.130, d = -0.436), respectively. (4) Conclusions: This study indicates that ePROMs are a time-feasible method for collecting data to guide patient-personalized treatment approaches.

How to Monitor Disease Activity of Axial Spondyloarthritis in Clinical Practice

PURPOSE OF REVIEW

Treatment guided by periodic and quantitative data assessment results in better outcomes compared to using clinical gestalt. While validated generic as well as specific disease activity measures for axial spondyloarthritis (axSpA) are available, there is vast scope to improve their actual utilization in routine clinical practice. In this review, we discuss available disease activity measures for axSpA, describe results from the survey conducted among general rheumatologists as well as Spondyloarthritis Research and Treatment Network (SPARTAN) members about disease activity measurement in daily practice, and discuss ways to improve axSpA disease activity using technological advances. We also discuss the definitions of active disease and target for the treatment of axSpA.

RECENT FINDINGS

The 2019 American College of Rheumatology (ACR)/Spondylitis Association of America (SAA)/Spondyloarthritis Research and Treatment Network (SPARTAN) axSpA treatment guidelines conditionally recommend the regular monitoring of disease activity using a validated measure such as Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) or Ankylosing Spondylitis Disease Severity Index (ASDAS). Assessment of Spondyloarthritis International Society (ASAS)-European Alliance of Associations for Rheumatology (EULAR) guidelines recommend ASDAS as the most appropriate instrument for the assessment of disease activity, preferably calculated using C-reactive protein (CRP). ASAS has selected a core set of variables which were updated recently and have been endorsed by the Outcome Measures in Rheumatology Clinical Trials (OMERACT) group in order to bring homogeneity in assessment of axSpA. In a recent study, Patient-Reported Outcomes Measurement Information System (PROMIS®) measures were able to discriminate inactive, moderate, and high-very high ASDAS activity groups. A newly developed semi-objective index P4 (pain, physical function, patient global, and physician global) correlates well with BASDAI and ASDAS in axSpA and can also be used for other rheumatic diseases in busy clinical practices. Regular disease activity monitoring is critical for long-term management of axSpA and shared decision-making. The integration of electronic health records and smart devices provides a great opportunity to capture patient-reported data. Automated capture of electronic patient-reported outcome measures (ePROMs) is a highly efficient way and results in consistent regular monitoring and may improve the long-term outcomes. While currently used measures focus only on musculoskeletal symptoms of axSpA, a composite disease activity measure that can also incorporate extra-articular manifestations may provide a better assessment of disease activity.

PsycAssist: A Web-Based Artificial Intelligence System Designed for Adaptive Neuropsychological Assessment and Training

Assessing executive functions in individuals with disorders or clinical conditions can be challenging, as they may lack the abilities needed for conventional test formats. The use of more personalized test versions, such as adaptive assessments, might be helpful in evaluating individuals with specific needs. This paper introduces PsycAssist, a web-based artificial intelligence system designed for neuropsychological adaptive assessment and training. PsycAssist is a highly flexible and scalable system based on procedural knowledge space theory and may be used potentially with many types of tests. We present the architecture and adaptive assessment engine of PsycAssist and the two currently available tests: Adap-ToL, an adaptive version of the Tower of London-like test to assess planning skills, and MatriKS, a Raven-like test to evaluate fluid intelligence. Finally, we describe the results of an investigation of the usability of Adap-ToL and MatriKS: the evaluators perceived these tools as appropriate and well-suited for their intended purposes, and the test-takers perceived the assessment as a positive experience. To sum up, PsycAssist represents an innovative and promising tool to tailor evaluation and training to the specific characteristics of the individual, useful for clinical practice.

Evaluation of electronic patient-reported outcome assessment in inpatient cancer care: a feasibility study

PURPOSE

Patient-reported outcome (PRO) measures are increasingly important in evaluating medical care. The increased integration of technology within the healthcare systems allows for collection of PROs electronically. The objectives of this study were to Ashley et al. J Med Internet Res (2013) implement an electronic assessment of PROs in inpatient cancer care and test its feasibility for patients and Dawson et al. BMJ (2010) determine the equivalence of the paper and electronic assessment.

METHODS

We analyzed two arms from a study that was originally designed to be an interventional, three-arm, and multicenter inpatient trial. A self-administered questionnaire based on validated PRO-measures was applied and completed at admission, 1 week after, and at discharge. For this analysis - focusing on feasibility of the electronic assessment - the following groups will be considered: Group A (intervention arm) received a tablet version, while group B (control arm) completed the questionnaire on paper. A feasibility questionnaire, that was adapted from Ashley et al. J Med Internet Res (2013), was administered to group A.

RESULTS

We analyzed 103 patients that were recruited in oncology wards. ePRO was feasible to most patients, with 84% preferring the electronic over paper-based assessment. The feasibility questionnaire contained questions that were answered on a scale ranging from "1" (illustrating non achievement) to "5" (illustrating achieving goal). The majority (mean 4.24, SD .99) reported no difficulties handling the electronic tool and found it relatively easy finding time for filling out the questionnaire (mean 4.15, SD 1.05). There were no significant differences between the paper and the electronic assessment regarding the PROs.

CONCLUSION

Results indicate that electronic PRO assessment in inpatient cancer care is feasible.

Validation of smartphone app-based digital patient reported outcomes in full-endoscopic spine surgery

PURPOSE

Patient reported outcome measures (PROMs) are important tools to assess patient function, pain, disability, and quality of life. We aim to study the efficiency and validity of digital PROMs collection using a smartphone app compared to traditional paper PROMs.

METHODS

Patients undergoing evaluation for full-endoscopic spine surgery were recruited from the outpatient clinic at Harborview Medical Center. Visual analogue scale (VAS), Oswestry disability index (ODI), and EQ5-5D PROMs were administered on paper and through a smartphone app called SpineHealthie. Compliance rates were collected, and PROM results were assessed for correlation between paper and digital methods.

RESULTS

123 patients were enrolled. 57.7% of patients completed paper PROMs, 82.9% completed digital PROMs, and 48.8% completed both. Of the patients that completed both, Spearman's correlation was greatest for VAS leg, ODI, and EQ5 index scores. Correlation was weaker for VAS back pain, neck pain, and upper extremity pain. Patients tended to report lower disability and higher quality of life on the digital PROM compared to the paper PROM.

CONCLUSION

The SpineHealthie app effectively and accurately collects PROMs digitally, showing strong concordance with traditional paper PROMs. We conclude that digital PROMs constitute a promising strategy for monitoring patients after spine surgery over time.

Exploration of the Feasibility of Remote Assessment of Functioning in Children and Adolescents with Developmental Disabilities: Parents' Perspectives and Related Contextual Factors

The COVID-19 pandemic interrupted face-to-face health services, leveraging telehealth strategies. The aim of this cross-sectional study was to investigate, from a parent's perspective, the feasibility of a remote assessment of functioning in children with developmental disabilities during the pandemic and related contextual factors, based on how parents carry out the assessment. Parents of children with developmental disabilities (mean age = 7.56 ± 3.68) responded to a remote assessment via electronic forms and telephone interview. We analyzed parents' perspectives about the feasibility of the assessment. We also tested the association between feasibility score and sociodemographics/pandemic experience. Regression analysis tested if children's functioning characteristics predicted feasibility. A total of 57 mothers completed the remote assessment, and more than 95% did not report difficulties in accessing/responding to electronic forms. They scored remote assessment as easy and feasible, and reported no difficulties with telephone interview. Greater feasibility rates were related to lower maternal age (rho Spearman = -0.290; p = 0.029). The model shows that children's characteristics predicted 20.4% of feasibility (p < 0.005). Remote assessment showed to be feasible. Younger mothers might consider easier-to-use technologies, beyond considering remote assessment more viable. These results can guide the next steps in research and remote clinical practice.

Monitoring of Disease Activity With a Smartphone App in Routine Clinical Care in Patients With Axial Spondyloarthritis

OBJECTIVE

To investigate the performance of a health app with respect to usability, adherence, and equivalence of data in daily care of patients with axial spondyloarthritis (axSpA).

METHODS

Consecutive patients with axSpA were asked to export patient-reported outcomes (PRO) electronically with the AxSpA Live App regularly every 2 weeks over a period of 6 months. The first clinical visit was followed by 2 further personal visits after 3 and 6 months. Patients completed paper-based PRO at every visit; they also completed the Mobile App Rating Scale and the System Usability Scale after 3 and 6 months.

RESULTS

Of 103 patients with axSpA, 69 agreed to participate (67.0%): age 41.5 (11.3) years, 58.0% male, Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) 4.3 (2.0), and 76.8% treated with biologic disease-modifying antirheumatic drugs. Patients' adherence to regular app exports was 29.0% and 28.4% after 3 and 6 months, respectively. Significant predictors for good adherence were high disease activity (P = 0.02) and older age (P = 0.04). No systematic differences between digital and paper-based BASDAI scores were found (intraclass correlation coefficients 0.99 [95% CI 0.98-0.99]). Performance of the app was rated as good.

CONCLUSION

Collection of digital PROs by AxSpA Live App may be successfully used in patients with axSpA with high disease activity. Our study showed equivalence of digital data, but adherence to the app after 6 months was poor. Higher disease activity and older age resulted in increased adherence to the app. This suggests that the use of health apps like this should concentrate on more severely affected patients.

Treat-to-target in axial spondyloarthritis: an observational study in daily practice

OBJECTIVES

To evaluate the extent to which internationally agreed treat-to-target recommendations were applied in clinical practice in patients with axial spondyloarthritis.

METHODS

Data were used from a web-based patient registry for monitoring SpA in daily practice in the Netherlands. The extent to which treat-to-target was applied was evaluated through four indicators: the proportion of patients (i) with ≥1 Ankylosing Spondylitis Disease Activity Score (ASDAS) assessed during a 1-year period, (ii) having inactive disease/low disease activity (i.e. ASDAS < 2.1), (iii) in whom re-evaluation of ASDAS within recommended intervals occurred, and (iv) with high disease activity (HDA, i.e. ASDAS ≥ 2.1) in whom treatment was adapted ≤6 weeks after obtaining ASDAS ≥ 2.1. Patients with HDA with treatment adaptations were compared with patients with HDA without treatment adaptations.

RESULTS

In 185 out of 219 patients (84%), disease activity was monitored with ≥1 ASDAS during a 1-year period, of whom 71 (38%) patients had a score below the target (ASDAS < 2.1) at first measurement. Re-evaluation of ASDAS ≤3 months occurred in 11% and 23% of the patients with inactive disease/low disease activity and HDA, respectively. Treatment adaptation occurred in 19 out of 114 patients (17%) with HDA. Patients in whom treatment was adapted had significantly higher ASDAS (P < 0.01), CRP levels (P < 0.05) and physician global assessment (P < 0.05) compared with patients without treatment adaptations.

CONCLUSIONS

Treat-to-target was applied to a limited extent in clinical practice in patients with axial spondyloarthritis. Available disease activity scores seemed not to be used for determining the frequency of re-evaluation nor treatment adaptation.

Converting the existing disease surveillance from a paper-based to an electronic-based system using district health information system (DHIS-2) for real-time information: the Lebanese experience

INTRODUCTION

The Ministry of Public Health in Lebanon is in the process of converting the surveillance reporting from a cumbersome paper-based system to a web-based electronic platform (DHIS-2) to have real-time information for early detection of alerts and outbreaks and for initiating a prompt response.

OBJECTIVES

This paper aimed to document the Lebanese experience in implementing DHIS-2 for the disease surveillance system. It also targets to assess the improvement of reporting rates and timeliness of the reported data and to disclose the encountered challenges and opportunities.

METHODOLOGY

This is a retrospective description of processes involved in the implementation of the DHIS-2 tool in Lebanon. Initially, it was piloted for the school-based surveillance in 2014; then its use was extended in May 2017 to cover other specific surveillance systems. This included all surveillance programs collecting aggregate data from hospitals, medical centers, dispensaries, or laboratories at the first stage. As part of the national roll-out process, the online application was developed. The customized aggregated-based datasets, organization units, user accounts, specific and generic dashboards were generated. More than 80 training sessions were conducted throughout the country targeting 1290 end-users including health officers at the national and provincial levels, focal persons who were working in all public and private hospitals, laboratories, and medical centers as well. Completeness and timeliness of reported data were compared before and after the implementation of DHIS-2. The unveiled challenges and the main lessons learned during the roll-out process were discussed.

RESULTS

For laboratory-based surveillance, completeness of reporting increased from 70.8% in May to 89.6% in October. Timeliness has improved from 25 to 74%. For medical centers, an improvement of 8.1% for completeness and 9.4% in timeliness was recorded before and after training sessions. For zero reporting, completeness remains the same (88%) and timeliness has improved from 74 to 87%. The main challenges faced during the implementation of DHIS-2 were mainly infrastructural and system-related in addition to poor internet connectivity, limited workforce, and frequent changes to DHIS-2 versions.

CONCLUSION

Implementation of DHIS-2 improved timeliness and completeness for aggregated data reporting. Continued on-site support, monitoring, and system enhancement are needed to improve the performance of DHIS-2.

Test-retest reliability of outcome measures: data from three trials in radiographic and non-radiographic axial spondyloarthritis

Objectives

Aim of this study was to assess test–retest reliability of candidate instruments for the mandatory domains of the Assessment of Spondyloarthritis international Society (ASAS)-Outcome Measures in Rheumatology core set for axial spondyloarthritis (axSpA).

Methods

Screening and baseline data from COAST-V, COAST-X and RAPID-axSpA was used to evaluate test–retest reliability of each candidate instrument for the mandatory domains (disease activity, pain, morning stiffness, fatigue, physical function, overall functioning and health). A maximum time interval of 28 days between both visits was used for inclusion in this study. Test–retest reliability was assessed by intraclass correlation coefficient (ICC). Bland and Altman plots provided mean difference and 95% limits of agreement, which were used to calculate the smallest detectable change (SDC). Data were analysed for radiographic and non-radiographic axSpA separately.

Results

Good reliability was found for Ankylosing Spondylitis Disease Activity Score (ICC 0.79, SDC 0.6), C reactive protein (ICC 0.72–0.79, SDC 12.3–17.0), Bath Ankylosing Spondylitis Functional Index (ICC 0.87, SDC 1.1) and 36-item Short-Form Health Survey (ICC Physical Component Summary 0.81, SDC 4.7, Mental Component Summary 0.80, SDC 7.3). Moderate reliability was found for Bath Ankylosing Spondylitis Disease Activity Index (ICC 0.72, SDC 1.1), patient global assessment (ICC 0.58, SDC 1.5), total back pain (ICC 0.64, SDC 1.3), back pain at night (ICC 0.67, SDC 1.3), morning stiffness (ICC 0.52–0.63, SDC 1.5–2.2), fatigue (ICC 0.65, SDC 1.3) and ASAS-Health Index (ICC 0.74, SDC 2.5). Reliability and SDC for the radiographic and non-radiographic axSpA subgroups were similar.

Conclusion

Overall reliability was good, and comparable levels of reliability were found for patients with radiographic and non-radiographic axSpA, even though most instruments were developed for radiographic axSpA. Composite measures showed higher reliability than single-item measures in assessing disease activity in patients with axSpA.

Digital Approaches for a Reliable Early Diagnosis of Psoriatic Arthritis

Psoriatic arthritis (PsA) is a chronic inflammatory disease that develops in up to 30% of patients with psoriasis. In the vast majority of cases, cutaneous symptoms precede musculoskeletal complaints. Progression from psoriasis to PsA is characterized by subclinical synovio-entheseal inflammation and often non-specific musculoskeletal symptoms that are frequently unreported or overlooked. With the development of increasingly effective therapies and a broad drug armamentarium, prevention of arthritis development through careful clinical monitoring has become priority. Identifying high-risk psoriasis patients before PsA onset would ensure early diagnosis, increased treatment efficacy, and ultimately better outcomes; ideally, PsA development could even be averted. However, the current model of care for PsA offers only limited possibilities of early intervention. This is attributable to the large pool of patients to be monitored and the limited resources of the health care system in comparison. The use of digital technologies for health (eHealth) could help close this gap in care by enabling faster, more targeted and more streamlined access to rheumatological care for patients with psoriasis. eHealth solutions particularly include telemedicine, mobile technologies, and symptom checkers. Telemedicine enables rheumatological visits and consultations at a distance while mobile technologies can improve monitoring by allowing patients to self-report symptoms and disease-related parameters continuously. Symptom checkers have the potential to direct patients to medical attention at an earlier point of their disease and therefore minimizing diagnostic delay. Overall, these interventions could lead to earlier diagnoses of arthritis, improved monitoring, and better disease control while simultaneously increasing the capacity of referral centers.

Measuring Spinal Mobility Using an Inertial Measurement Unit System: A Reliability Study in Axial Spondyloarthritis

The objectives of this study were to evaluate the reliability of wearable inertial motion unit (IMU) sensors in measuring spinal range of motion under supervised and unsupervised conditions in both laboratory and ambulatory settings. A secondary aim of the study was to evaluate the reliability of composite IMU metrology scores (IMU-ASMI (Amb)). Forty people with axSpA participated in this clinical measurement study. Participant spinal mobility was assessed by conventional metrology (Bath Ankylosing Spondylitis Metrology Index, linear version—BASMI_Lin) and by a wireless IMU sensor-based system which measured lumbar flexion-extension, lateral flexion and rotation. Each sensor-based movement test was converted to a normalized index and used to calculate IMU-ASMI (Amb) scores. Test-retest reliability was evaluated using intra-class correlation coefficients (ICC). There was good to excellent agreement for all spinal range of movements (ICC > 0.85) and IMU-ASMI (Amb) scores (ICC > 0.87) across all conditions. Correlations between IMU-ASMI (Amb) scores and conventional metrology were strong (Pearson correlation ≥ 0.85). An IMU sensor-based system is a reliable way of measuring spinal lumbar mobility in axSpA under supervised and unsupervised conditions. While not a replacement for established clinical measures, composite IMU-ASMI (Amb) scores may be reliably used as a proxy measure of spinal mobility.

Barriers and Facilitators to the International Implementation of Standardized Outcome Measures in Clinical Cleft Practice

Objective:

To identify barriers and facilitators to international implementation of a prospective system for standardized outcomes measurement in cleft care.

Design:

Cleft teams that have implemented the International Consortium for Health Outcomes Measurement Standard Set for cleft care were invited to participate in this 2-part qualitative study: (1) an exploratory survey among clinicians, health information technology professionals, and project coordinators, and (2) semistructured interviews of project leads. Thematic content analysis was performed, with organization of themes according to the dimensions of the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework: reach, effectiveness, adoption, implementation, and maintenance.

Results:

Four cleft teams in Europe and North America participated in this study. Thirteen participants completed exploratory questionnaires and 5 interviewees participated in follow-up interviews. Survey responses and thematic content analysis revealed common facilitators and barriers to implementation at all sites. Teams reach patients either via email or during the clinic visit to capture patient-reported outcomes. Adopting routine data collection is enhanced by aligning priorities at the organizational and cleft team level. Streamlining workflows and developing an efficient data collection platform are necessary early on, followed by pilot testing or stepwise implementation. Regular meetings and financial resources are crucial for implementing, sustaining, analyzing collected data, and providing feedback to health care professionals and patients. Fostering patient-centered care was articulated as a positive outcome, whereas time presented challenges across all RE-AIM dimensions.

Conclusions:

Identified themes can inform ongoing implementation efforts. Intentionally investing time to lay a sound foundation early on will benefit every phase of implementation and help overcome barriers such as lack of support or motivation.

Real-world digital implementation of the Psychosis Polyrisk Score (PPS): A pilot feasibility study

BACKGROUND

The Psychosis Polyrisk Score (PPS) is a potential biomarker integrating non-purely genetic risk/protective factors for psychosis that may improve identification of individuals at risk and prediction of their outcomes at the individual subject level. Biomarkers that are easy to administer are direly needed in early psychosis to facilitate clinical implementation. This study digitally implements the PPS and pilots its feasibility of use in the real world.

METHODS

The PPS was implemented digitally and prospectively piloted across individuals referred for a CHR-P assessment (n = 16) and healthy controls (n = 66). Distribution of PPS scores was further simulated in the general population.

RESULTS

98.8% of individuals referred for a CHR-P assessment and healthy controls completed the PPS assessment with only one drop-out. 96.3% of participants completed the assessment in under 15 min. Individuals referred for a CHR-P assessment had high PPS scores (mean = 6.2, SD = 7.23) than healthy controls (mean = -1.79, SD = 6.78, p < 0.001). In simulated general population data, scores were normally distributed ranging from -15 (lowest risk, RR = 0.03) to 39.5 (highest risk, RR = 8912.51).

DISCUSSION

The PPS is a promising biomarker which has been implemented digitally. The PPS can be easily administered to both healthy controls and individuals at potential risk for psychosis on a range of devices. It is feasible to use the PPS in real world settings to assess individuals with emerging mental disorders. The next phase of research should be to include the PPS in large-scale international cohort studies to evaluate its ability to refine the prognostication of outcomes.

Primary Care Pre-Visit Electronic Patient Questionnaire for Asthma: Uptake Analysis and Predictor Modeling

Background

mHealth tablet-based interventions are increasingly being studied and deployed in various health care settings, yet little knowledge exists regarding patient uptake and acceptance or how patient demographics influence these important implementation metrics.

Objective

To determine which factors influence the uptake and successful completion of an mHealth tablet questionnaire by analyzing its implementation in a primary care setting.

Methods

We prospectively studied a patient-facing electronic touch-tablet asthma questionnaire deployed as part of the Electronic Asthma Management System. We describe tablet uptake and completion rates and corresponding predictor models for these behaviors.

Results

The tablet was offered to and accepted by patients in 891/1715 (52.0%) visits. Patients refused the tablet in 33.0% (439/1330) visits in which it was successfully offered. Patients aged older than 65 years of age (odds ratio [OR] 2.30, 95% CI 1.33-3.95) and with concurrent chronic obstructive pulmonary disease (OR 2.22, 95% CI 1.05-4.67) were more likely to refuse the tablet, and those on an asthma medication (OR 0.55, 95% CI 0.30-0.99) were less likely to refuse it. Once accepted, the questionnaire was completed in 784/891 (88.0%) instances, with those on an asthma medication (OR 0.53, 95% CI 0.32-0.88) being less likely to leave it incomplete.

Conclusions

Older age predicted initial tablet refusal but not tablet questionnaire completion, suggesting that perceptions of mHealth among older adults may negatively impact uptake, independent of usability. The influence of being on an asthma medication suggests that disease severity may also mediate mHealth acceptance. Although use of mHealth questionnaires is growing rapidly across health care settings and diseases, few studies describe their real-world acceptance and its predictors. Our results should be complemented by qualitative methods to identify barriers and enablers to uptake and may inform technological and implementation strategies to drive successful usage.

Validity, reliability, and application of the electronic version of a chronic kidney disease patient awareness questionnaire: a pilot study

OBJECTIVES

A questionnaire which provides desirable reliability and validity has been previously developed to assess the disease awareness of diagnosed chronic kidney disease (CKD) patients. However, conventional paper questionnaires often have disadvantages, including recall bias. To substantially improve this, we therefore aimed to explore the feasibility of developing a smartphone-based electronic version (e-version) based upon its original paper version and subsequently tested its validity, reliability, and applicability.

METHODS

A pilot study was conducted at Guangdong Provincial Hospital of Chinese Medicine in Guangzhou, China, during August 2019. The e-version had identical content to the paper version and was adapted in terms of layout and assisted functions via the Wechat-incorporated Wen-Juan-Xing platform. Eligible patients with diagnosed CKD were invited to participate and were assigned the e-version. Randomly selected respondents received a test-retest of the same e-version 2 weeks after their first completion. In some instances, psychometric properties, including validity and reliability of the e-version, were examined. In others, its clinical application was also tested, which included comparisons among the clinical profiles of patients who had/had not responded to the questionnaire as well as patients with above or below average questionnaire scores.

RESULTS

Of the 225 patients screened, 217 were enrolled to participate, with a response rate of 52.5%. Desirable reliability (Cronbachα = 0.962, ICC for total scores = 0.948), while good convergent validity (Cronbachα = 0.962) and low discriminant validity (one extracted component), of the e-version were detected. Performing inter-group comparisons highlighted statistical differences in terms of higher education level (z = -2.436, P = 0.015) and earlier CKD stages (z = -1.978, P = 0.048), with these patients often preferring to respond. No significant differences were detected in the clinical profiles between respondents who obtained an above or below average questionnaire score.

CONCLUSION

The e-version is reliable but was not shown to be a valid approach. Audiences with higher education levels and less advanced disease condition may prefer to respond to the e-version. Adaptation of this e-questionnaire, from its original paper version, may not be a direct transition and meticulous modifications may be required during the transition process.

TRIAL REGISTRATION

Chinese Clinical Trial Registry (ChiCTR1900024633).

A Mobile Technology for Collecting Patient-Reported Physical Activity and Distress Outcomes: Cross-Sectional Cohort Study

Background

Electronic patient-reported outcome (PROs) provides a fast and reliable assessment of a patient’s health-related quality of life. Nevertheless, using PRO in the traditional paper format is not practical for clinical practice due to the limitations associated with data analysis and management. A questionnaire app was developed to address the need for a practical way to group and use distress and physical activity assessment tools.

Objective

The purpose of this study was to assess the level of agreement between electronic (mobile) and paper-and-pencil questionnaire responses.

Methods

We validated the app version of the distress thermometer (DT), International Physical Activity Questionnaire (IPAQ), and Patient Health Questionnaire–9 (PHQ-9). A total of 102 participants answered the paper and app versions of the DT and IPAQ, and 96 people completed the PHQ-9. The study outcomes were the correlation of the data between the paper-and-pencil and app versions.

Results

A total of 106 consecutive breast cancer patients were enrolled and analyzed for validation of paper and electronic (app) versions. The Spearman correlation values of paper and app surveys for patients who responded to the DT questionnaire within 7 days, within 3 days, and on the same day were .415 (P<.001), .437 (P<.001), and .603 (P<.001), respectively. Similarly, the paper and app survey correlation values of the IPAQ total physical activity metabolic equivalent of task (MET; Q2-6) were .291 (P=.003), .324 (P=.005), and .427 (P=.01), respectively. The correlation of the sum of the Patient Health Questionnaire–9 (Q1-9) according to the time interval between the paper-based questionnaire and the app-based questionnaire was .469 for 14 days (P<.001), .574 for 7 days (P<.001), .593 for 3 days (P<.001), and .512 for the same day (P=.03). These were all statistically significant. Similarly, the correlation of the PHQ (Q10) value according to the time interval between the paper-based questionnaire and the app-based questionnaire was .283 for 14 days (P=.005), .409 for 7 days (P=.001), .415 for 3 days (P=.009), and .736 for the same day (P=.001). These were all statistically significant. In the overall trend, the shorter the interval between the paper-and-pencil questionnaire and the app-based questionnaire, the higher the correlation value.

Conclusions

The app version of the distress and physical activity questionnaires has shown validity and a high level of association with the paper-based DT, IPAQ (Q2-6), and PHQ-9. The app-based questionnaires were not inferior to their respective paper versions and confirm the feasibility for their use in clinical practice. The high correlation between paper and mobile app data allows the use of new mobile apps to benefit the overall health care system.

Trial Registration

ClinicalTrials.gov NCT03072966; https://clinicaltrials.gov/ct2/show/NCT03072966

Benefits and Disadvantages of Electronic Patient-reported Outcome Measures: Systematic Review

Background

Patient-reported outcome measures (PROMs) are important in clinical practice and research. The growth of electronic health technologies provides unprecedented opportunities to systematically collect information via PROMs.

Objective

The aim of this study was to provide an objective and comprehensive overview of the benefits, barriers, and disadvantages of the digital collection of qualitative electronic patient-reported outcome measures (ePROMs).

Methods

We performed a systematic review of articles retrieved from PubMED and Web of Science. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed during all stages. The search strategy yielded a total of 2333 records, from which 32 met the predefined inclusion and exclusion criteria. The relevant ePROM-related information was extracted from each study.

Results

Results were clustered as benefits and disadvantages. Reported benefits of ePROMs were greater patient preference and acceptability, lower costs, similar or faster completion time, higher data quality and response rates, and facilitated symptom management and patient-clinician communication. Tablets were the most used ePROM modality (14/32, 44%), and, as a platform, Web-based systems were used the most (26/32, 81%). Potential disadvantages of ePROMs include privacy protection, a possible large initial financial investment, and exclusion of certain populations or the “digital divide.”

Conclusions

In conclusion, ePROMs offer many advantages over paper-based collection of patient-reported outcomes. Overall, ePROMs are preferred over paper-based methods, improve data quality, result in similar or faster completion time, decrease costs, and facilitate clinical decision making and symptom management. Disadvantages regarding ePROMs have been outlined, and suggestions are provided to overcome the barriers. We provide a path forward for researchers and clinicians interested in implementing ePROMs.

Trial Registration

PROSPERO CRD42018094795; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=94795

Is There Equivalence Between the Electronic and Paper Version of the Questionnaires for Assessment of Patients With Chronic Low Back Pain?

STUDY DESIGN

Cross-sectional study.

OBJECTIVE

To investigate the equivalence of electronic and paper version of self-report questionnaires for the assessment of disability, pain, fear of movement, depression, and physical activity of patients with chronic low back pain (LBP).

SUMMARY OF BACKGROUND DATA

Paper and electronic versions of self-report questionnaires are commonly used for assessment of patients with LBP. However, the equivalence of self-report questionnaires commonly used for assessment of patients with chronic LBP remains unclear.

METHODS

Seventy-nine individuals with chronic LBP seeking physiotherapy care were recruited. Participants attended the clinic twice with an interval of 1 week and completed the self-reported questionnaires in a random order. The following questionnaires were administered: Roland Morris Disability Questionnaire (RMDQ); 11-point numerical rating scale (NRS); Tampa Scale for Kinesiophobia (TSK); Center for Epidemiological Studies-Depression (CES-D), and Baecke Habitual Physical Activity Questionnaire (BPAQ). To investigate the equivalence between the two questionnaire versions, intraclass correlation coefficient with 95% confidence interval and Bland-Altman plotting was used.

RESULTS

The paper and electronic versions of the RMDQ, TSK, and CES-D showed good reliability and the showed moderate reliability. In contrast, the NRS showed poor reliability between the electronic and paper versions.

CONCLUSION

Our findings support that the electronic version of the RMDQ, TSK, CES-D, and BPAQ can be administered in clinical and research settings for assessment of patients with chronic LBP. Nevertheless, electronic version of the NRS for assessment of pain intensity should not be used interchangeably in clinical practice in patients with chronic LBP.

LEVEL OF EVIDENCE

3.

Using Health Information Technology to Support Use of Patient-Reported Outcomes in Rheumatology

Technology can help health care providers understand their patients' experience of illness in a way that was previously impossible. Experience in using health information technology (IT) to capture this information through PROs within rheumatology suggests that careful attention to human centered design, including detailed workflow planning, consideration of patient and physician burden, integration into the health IT ecosystem, and delivering information to the right person at the right time are all important. Technology applications must be tested in diverse health systems and populations to ensure they are simple to interpret, useful for clinical decision making and effective in impacting outcomes.

Mobile Health Apps for Self-Management of Rheumatic and Musculoskeletal Diseases: Systematic Literature Review

BACKGROUND

Although the increasing availability of mobile health (mHealth) apps may enable people with rheumatic and musculoskeletal diseases (RMDs) to better self-manage their health, there is a general lack of evidence on ways to ensure appropriate development and evaluation of apps.

OBJECTIVE

This study aimed to obtain an overview on existing mHealth apps for self-management in patients with RMDs, focusing on content and development methods.

METHODS

A search was performed up to December 2017 across 5 databases. For each publication relevant to an app for RMDs, information on the disease, purpose, content, and development strategies was extracted and qualitatively assessed.

RESULTS

Of 562 abstracts, 32 were included in the analysis. Of these 32 abstracts, 11 (34%) referred to an app linked to a connected device. Most of the apps targeted rheumatoid arthritis (11/32, 34%). The top three aspects addressed by the apps were pain (23/32, 71%), fatigue (15/32, 47%), and physical activity (15/32, 47%). The development process of the apps was described in 84% (27/32) of the articles and was of low to moderate quality in most of the cases. Despite most of the articles having been published within the past two years, only 5 apps were still commercially available at the time of our search. Moreover, only very few studies showed improvement of RMD outcome measures.

CONCLUSIONS

The development process of most apps was of low or moderate quality in many studies. Owing to the increasing RMD patients' willingness to use mHealth apps for self-management, optimal standards and quality assurance of new apps are mandatory.

Comparison between an electronic version of the foot and ankle outcome score and the standard paper version: A randomized multicenter study

To prove the equivalence of the Korean version of the Foot and Ankle Outcome Score (FAOS) in the printed (PFAOS) vs the electronic (EFAOS) form in a multicenter randomized study.Overall, 227 patients with ages ranging from 20 to 79 years from 16 dedicated foot and ankle centers were included. Patients were randomized into either a 'paper first' group (P-F group, n = 113) or an 'electronic device (tablet computer) first' group (E-F group, n = 114). The first evaluation either by paper (P-F group) or tablet (E-F group) was followed by a second evaluation the following day. The difference between the PFAOS and EFAOS results in each group was calculated and analyzed. To evaluate the benefit of each methodology, the time consumed per evaluation was compared and patients were asked which methodology they preferred and which was the easiest to use.There were no significant differences in age or sex between the groups. An intraclass correlation coefficient (ICC) value of 0.934 (95% confidence interval [CI]: 0.912-0.950, P < .001) was confirmed in PFAOS and EFAOS, showing a significant correlation between the 2 methodologies. EFAOS was completed in a shorter amount of time than PFAOS. The majority of patients agreed that EFAOS was easier to complete than PFAOS.The paper or electronic forms of the Korean adaptation of FAOS were considered equivalent. The shorter time of completion and the preference for the electronic version over paper by patients deems the electronic FAOS a promising option to consider in future.

EULAR points to consider for the development, evaluation and implementation of mobile health applications aiding self-management in people living with rheumatic and musculoskeletal diseases

Background

Mobile health applications (apps) are available to enable people with rheumatic and musculoskeletal diseases (RMDs) to better self-manage their health. However, guidance on the development and evaluation of such apps is lacking.

Objectives

The objective of this EULAR task force was to establish points to consider (PtC) for the development, evaluation and implementation of apps for self-management of RMDs.

Methods

A systematic literature review of app content and development strategies was conducted, followed by patient focus group and an online survey. Based on this information and along with task force expert opinion, PtC were formulated in a face-to-face meeting by a multidisciplinary task force panel of experts, including two patient research partners. The level of agreement among the panel in regard to each PtC was established by anonymous online voting.

Results

Three overarching principles and 10 PtC were formulated. Three PtC are related to patient safety, considered as a critical issue by the panel. Three are related to relevance of the content and functionalities. The requirement for transparency around app development and funding sources, along with involvement of relevant health professionals, were also raised. Ease of app access across ages and abilities was highlighted, in addition to considering the cost benefit of apps from the outset. The level of agreement was from 8.8 to 9.9 out of 10.

Conclusion

These EULAR PtC provide guidance on important aspects that should be considered for the development, evaluation and implementation of existing and new apps.

Development, usability and acceptability of an integrated eHealth system for spondyloarthritis in the Netherlands (SpA-Net)

Objective

To develop and test the usability and acceptability of a disease-specific integrated electronic health (eHealth) system for spondyloarthritis (SpA) in the Netherlands (‘SpA-Net’).

Methods

SpA-Net was developed in four phases. First, content and design were discussed with experts on SpA and patients. Second, the database, electronic medical record (EMR) and quality management system were developed. Third, multiple rounds of testing were performed. Fourth, the eHealth system was implemented in practice and feasibility was tested among patients through semistructured focus interviews (n=16 patients) and among care providers through feedback meetings (n=11 rheumatologists/fellows and 5 nurses).

Results

After completion of the first three steps of development in 2015, SpA-Net was implemented in 2016. All patients included have a clinical diagnosis of SpA. Information on domains relevant to clinical record-keeping is prospectively collected at routine outpatient consultations and readily available to care providers, presented in a clear dashboard. Patients complete online questionnaires prior to outpatient visits. In February 2019, 1069 patients were enrolled (mean [SD] age 54.9 [14.1] years, 52.4% men). Patients interviewed (n=16) considered SpA-Net an accessible system that was beneficial to disease insight and patient–physician communication, and had additional value to current care. Care providers appreciated the additional information for (preparing) consultations. Barriers were the initial time required to adopt the EMR and the quantity of data entry.

Conclusion

SpA-Net enables monitoring of patients with SpA and real-life data collection, and could help improve knowledge and optimise communication between patients and care providers. Both considered SpA-Net a valuable addition to current care.

Trial registration number

NTR6740.

Development process and patient usability preferences for a touch screen tablet-based questionnaire

We sought to design a touch tablet asthma questionnaire while identifying patient preferences for usability features of such questionnaires. We created an evidence-based prototype and employed rapid-cycle design (semi-structured focus group testing, analysis, corresponding modifications, re-testing) with asthma patients aged ⩾16 years. We analyzed transcripts using deductive and inductive content analysis. Quantitative measures included Likert-type-scale responses, the System Usability Scale, and questionnaire completion times. There were 20 participants across five focus groups (15/20 female, age 49.1 ± 15.6 years). Usability-related themes included (1) "Touch Technology" (hygiene, touch technology familiarity, ease of use) and (2) "Questionnaire Design" (visual characteristics, navigation). Completion time was 11.7 ± 5.9 min. Summative Likert-type scale responses suggested high system usability, as did a System Usability Scale score of 84.2 ± 14.7. In summary, Attention to specific technology- and design-related preferences can result in a highly usable patient-facing touch tablet questionnaire. Our findings can inform touch questionnaire design across other diseases.

Smartphone APPlications in the clinical care and management of Rheumatic Diseases

BACKGROUND

Rheumatic diseases (RDs) are the most common cause of severe long-term pain and physical disability, affecting hundreds of millions of people around the world. Smartphones technology have the potential to become an important tool that rheumatologist can employ in the clinical care management of RD.

METHODS

Research of the published literature on the principle electronic databases available as Ovid MEDLINE, Health Technology Assessment Database, Embase, and PsycINFO was conducted, and the studies evaluated eligible were reviewed.

RESULTS

Our search produced 120 results from which 47 eligible articles were identified reporting studies of smartphone apps for patients with RD. All examined feasibility and five assessed the efficacy of a smartphone intervention for clinical care management.

CONCLUSIONS

It has been demonstrated a strong evidence for the feasibility of using smartphone to enhance care of patients with RD.  Based on the available literature and our personal experiences, we consider useful the development of some mobile phone apps, to simplify and assist the rheumatologist during his clinical practice. Still remains limited data on the efficacy of such interventions.

Physiotherapy Questionnaires App to Deliver Main Musculoskeletal Assessment Questionnaires: Development and Validation Study

BACKGROUND

Patient-reported outcomes (PROs) translate subjective outcomes into objective data that can be quantified and analyzed. Nevertheless, the use of PROs in their traditional paper format is not practical for clinical practice due to limitations associated with the analysis and management of the data. To address the need for a viable way to group and utilize the main functioning assessment tools in the field of musculoskeletal disorders, the Physiotherapy Questionnaires app was developed.

OBJECTIVE

This study aims to explain the development of the app, to validate it using two questionnaires, and to analyze whether participants prefer to use the app or the paper version of the questionnaires.

METHODS

In the first stage, the app for an Android operational system was developed. In the second stage, the aim was to select questionnaires that were most often used in musculoskeletal clinical practice and research. The Foot and Ankle Outcome Score (FAOS) and American Orthopaedic Foot and Ankle Society (AOFAS) questionnaire were selected to validate the app. In total, 50 participants completed the paper and app versions of the AOFAS and 50 completed the FAOS. The study's outcomes were the correlation of the data between the paper and app versions as well as the preference of the participants between the two versions.

RESULTS

The app was approved by experts after the adaptations of the layout for mobile phones and a total of 18 questionnaires were included in the app. Moreover, the app allows the generation of PDF and Excel files with the patients' data. In regards to validity, the mean of the total scores of the FAOS were 91.54% (SD 8.86%) for the paper version and 91.74% (SD 9.20%) for the app. There was no statistically significant differences in the means of the total scores or the subscales (P=.11-.94). The mean total scores for the AOFAS were 93.94 (SD 8.47) for the paper version and 93.96 (SD 8.48) for the app. No statistically significant differences were found for the total scores for the AOFAS or the subscales (P>.99). The app showed excellent agreement with the paper version of the FAOS, with an ICC value of 0.98 for the total score (95% CI 0.98-0.99), which was also found for the AOFAS with the ICC for the total score of 0.99 (95% CI 0.98-0.99). For compliance, 72% (36/50) of the participants in the FAOS group and 94% (47/50) in the AOFAS group preferred the app version.

CONCLUSIONS

The Physiotherapy Questionnaires app showed validity and high levels of compliance for the FAOS and AOFAS, which indicates it is not inferior to the paper version of these two questionnaires and confirms its viability and feasibility for use in clinical practice.

Mobile electronic versus paper case report forms in clinical trials: a randomized controlled trial

BACKGROUND

Regulations, study design complexity and amounts of collected and shared data in clinical trials render efficient data handling procedures inevitable. Recent research suggests that electronic data capture can be key in this context but evidence is insufficient. This randomized controlled parallel group study tested the hypothesis that time efficiency is superior when electronic (eCRF) instead of paper case report forms (pCRF) are used for data collection. We additionally investigated predictors of time saving effects and data integrity.

METHODS

This study was conducted on top of a clinical weight loss trial performed at a clinical research facility over six months. All study nurses and patients participating in the clinical trial were eligible to participate and randomly allocated to enter cross-sectional data obtained during routine visits either through pCRF or eCRF. A balanced randomization list was generated before enrolment commenced. 90 and 30 records were gathered for the time that 27 patients and 2 study nurses required to report 2025 and 2037 field values, respectively. The primary hypothesis, that eCRF use is faster than pCRF use, was tested by a two-tailed t-test. Analysis of variance and covariance were used to evaluate predictors of entry performance. Data integrity was evaluated by descriptive statistics.

RESULTS

All randomized patients were included in the study (eCRF group n = 13, pCRF group n = 14). eCRF, as compared to pCRF, data collection was associated with significant time savings  across all conditions (8.29 ± 5.15 min vs. 10.54 ± 6.98 min, p = .047). This effect was not defined by participant type, i.e. patients or study nurses (F_(1,112) = .15, p = .699), CRF length (F_(2,112) = .49, p = .609) or patient age (Beta = .09, p = .534). Additional 5.16 ± 2.83 min per CRF were saved with eCRFs due to data transcription redundancy when patients answered questionnaires directly in eCRFs. Data integrity was superior in the eCRF condition (0 versus 3 data entry errors).

CONCLUSIONS

This is the first study to prove in direct comparison that using eCRFs instead of pCRFs increases time efficiency of data collection in clinical trials, irrespective of item quantity or patient age, and improves data quality.

TRIAL REGISTRATION

Clinical Trials NCT02649907 .

Detecting Acute Otitis Media Symptom Episodes Using a Mobile App: Cohort Study

BACKGROUND

Population cohort studies are useful to study infectious diseases episodes not attended by health care services, but conventional paper diaries and questionnaires to capture cases are prone to noncompliance and recall bias. Use of smart technology in this setting may improve case finding.

OBJECTIVE

The objective of our study was to validate an interactive mobile app for monitoring occurrence of acute infectious diseases episodes in individuals, independent of health care seeking, using acute otitis media (AOM) symptom episodes in infants as a case study. We were interested in determining participant compliance and app performance in detecting and ascertaining (parent-reported) AOM symptom episodes with this novel tool compared with traditional methods used for monitoring study participants.

METHODS

We tested the InfectieApp research app to detect AOM symptom episodes. In 2013, we followed 155 children aged 0 to 3 years for 4 months. Parents recorded the presence of AOM symptoms in a paper diary for 4 consecutive months and completed additional disease questionnaires when AOM symptoms were present. In 2015 in a similar cohort of 69 children, parents used an AOM diary and questionnaire app instead.

RESULTS

During conventional and app-based recording, 93.13% (17,244/18,516) and 94.56% (7438/7866) of symptom diaries were returned, respectively, and at least one symptom was recorded for 32.50% (n=5606) and 43.99% (n=3272) of diary days (P<.01). The incidence of AOM symptom episodes was 605 and 835 per 1000 child-years, respectively. Disease questionnaires were completed for 59% (17/29) of episodes when participants were using conventional recording, compared with 100% (18/18) for app-based recording.

CONCLUSIONS

The use of the study's smart diary app improved AOM case finding and disease questionnaire completeness. For common infectious diseases that often remain undetected by health care services, use of this technology can substantially improve the accurateness of disease burden estimates.

Patient-Reported Outcomes Measurement Information System Tools for Collecting Patient-Reported Outcomes in Children With Juvenile Arthritis

OBJECTIVE

To evaluate the precision and construct validity of pediatric Patient-Reported Outcomes Measurement Information System (PROMIS) instruments in a population of juvenile idiopathic arthritis (JIA) patients and parent proxies.

METHODS

A convenience sample of JIA patients and parents of JIA patients completed PROMIS instruments for 8 domains: anger, anxiety, depressive symptoms, fatigue, mobility, pain interference, peer relationships, and upper-extremity function. Short form and computerized adaptive test (CAT) scores were derived from item bank responses. Raw scores were translated to standardized T scores with corresponding SEs. Discrimination between inactive versus active disease was evaluated as an indicator of each measure's construct validity. SEs were plotted to evaluate each instrument's relative precision. Patient-parent concordance was assessed using intraclass correlations (ICCs).

RESULTS

A total of 228 patients and 223 parents participated, providing 71-78 responses per domain. Patient- and parent-reported anger, fatigue, mobility, and pain interference scores significantly differed between those with inactive and active disease. Anxiety, depressive symptoms, and peer relationships differed by disease activity levels for parent-report only. Short forms and CATs provided comparable reliability to the full item banks across the full range of each outcome. Patient-parent agreement ranged from ICC 0.3 to 0.8. CATs did not reduce the number of items for any domain compared to the short form.

CONCLUSION

Precision and discriminatory abilities of PROMIS instruments depend on health domain and report type (self-report versus parent proxy-report) for children with JIA. Varying levels of patient-parent concordance reinforces the importance of considering both perspectives in comprehensive health outcomes assessments.

Electronic case report forms and electronic data capture within clinical trials and pharmacoepidemiology

AIMS

Researchers in clinical and pharmacoepidemiology fields have adopted information technology (IT) and electronic data capture, but these remain underused despite the benefits. This review discusses electronic case report forms and electronic data capture, specifically within pharmacoepidemiology and clinical research.

METHODS

The review used PubMed and the Institute of Electrical and Electronic Engineers library. Search terms used were agreed by the authors and documented. PubMed is medical and health based, whereas Institute of Electrical and Electronic Engineers is technology based. The review focuses on electronic case report forms and electronic data capture, but briefly considers other relevant topics; consent, ethics and security.

RESULTS

There were 1126 papers found using the search terms. Manual filtering and reviewing of abstracts further condensed this number to 136 relevant manuscripts. The papers were further categorized: 17 contained study data; 40 observational data; 27 anecdotal data; 47 covering methodology or design of systems; one case study; one literature review; two feasibility studies; and one cost analysis.

CONCLUSION

Electronic case report forms, electronic data capture and IT in general are viewed with enthusiasm and are seen as a cost-effective means of improving research efficiency, educating participants and improving trial recruitment, provided concerns about how data will be protected from misuse can be addressed. Clear operational guidelines and best practises are key for healthcare providers, and researchers adopting IT, and further work is needed on improving integration of new technologies with current systems. A robust method of evaluation for technical innovation is required.

Cloudy with a Chance of Pain: Engagement and Subsequent Attrition of Daily Data Entry in a Smartphone Pilot Study Tracking Weather, Disease Severity, and Physical Activity in Patients With Rheumatoid Arthritis

BACKGROUND

The increasing ownership of smartphones provides major opportunities for epidemiological research through self-reported and passively collected data.

OBJECTIVE

This pilot study aimed to codesign a smartphone app to assess associations between weather and joint pain in patients with rheumatoid arthritis (RA) and to study the success of daily self-reported data entry over a 60-day period and the enablers of and barriers to data collection.

METHODS

A patient and public involvement group (n=5) and 2 focus groups of patients with RA (n=9) supported the codesign of the app collecting self-reported symptoms. A separate "capture app" was designed to collect global positioning system (GPS) and continuous raw accelerometer data, with the GPS-linking providing local weather data. A total of 20 patients with RA were then recruited to collect daily data for 60 days, with entry and exit interviews. Of these, 17 were loaned an Android smartphone, whereas 3 used their own Android smartphones.

RESULTS

Of the 20 patients, 6 (30%) withdrew from the study: 4 because of technical challenges and 2 for health reasons. The mean completion of daily entries was 68% over 2 months. Patients entered data at least five times per week 65% of the time. Reasons for successful engagement included a simple graphical user interface, automated reminders, visualization of data, and eagerness to contribute to this easily understood research question. The main barrier to continuing engagement was impaired battery life due to the accelerometer data capture app. For some, successful engagement required ongoing support in using the smartphones.

CONCLUSIONS

This successful pilot study has demonstrated that daily data collection using smartphones for health research is feasible and achievable with high levels of ongoing engagement over 2 months. This result opens important opportunities for large-scale longitudinal epidemiological research.

Patient Satisfaction with Collection of Patient-Reported Outcome Measures in Routine Care

INTRODUCTION

Systematic collection of patient-reported outcome measures (PROMs) during ambulatory clinic visits can enhance communication between patient and provider, and provide the ability to evaluate outcomes of care. Little is known about patient satisfaction of PROM data collection in routine clinical care. To evaluate patient reaction to the routine collection of PROMs in the ambulatory setting.

METHODS

Before all ambulatory clinic visits at our neurological institute, patients electronically complete health status questionnaires. We administered an 8-question patient satisfaction survey to a sample of patients seen across the institute after their clinical visit.

RESULTS

Of 343 patients approached, 323 agreed to participate. The majority responded that the questionnaire system was easy to use, was an appropriate length, and benefited their care overall (strongly agree or agree = 92.3%, 87.6%, and 77.3%, respectively). Provider review of the PROMs with the patient during the clinic visit was associated with significantly higher positive responses to all questions, even those regarding logistical aspects of the collection process. There were significant age and race differences in response to perceived benefit: those in the Black/other race category had a markedly lower probability of viewing the process favorably with increasing age.

CONCLUSIONS

Systematic collection of PROMs via an electronic questionnaire appears to be well accepted by patients. A minority of patients did not feel the questionnaire content applied to their appointment or that the system was a beneficial feature of the clinical practice. The provider can significantly improve the patient's perception of PROM collection and the patient-physician encounter by reviewing the questionnaire results with the patient.

An investigation of completion times on the Screener and Opioid Assessment for Patients with Pain - revised (SOAPP-R)

Background

Respondents’ scores to the Screener and Opioid Assessment for Patients with Pain – revised (SOAPP-R) have been shown to be predictive of aberrant drug-related behavior (ADB). However, research is lacking on whether an individual’s completion time (the amount of time that he/she takes to finish the screener) has utility in predicting ADB, despite the fact that response speed has been useful in predicting behavior in other fields. The purpose of this study was to evaluate the degree to which SOAPP-R completion time is predictive of ADB.

Materials and methods

This retrospective study analyzed completion-time data from 82 adult emergency department patients who completed the SOAPP-R on a tablet computer. The utility of SOAPP-R completion times in predicting ADB was assessed via logistic regression and the area under the curve (AUC) statistic. An external measure of ADB using Prescription Drug Monitoring Program data defined ADB to have occurred in individuals with at least four opioid prescriptions and at least four prescribers in 12 months.

Results

Although there was a slight trend for individuals with greater completion times to have greater odds of ADB (odds ratio 1.004 in simple logistic regression), the association between SOAPP-R completion time and ADB was not statistically significant in either simple logistic regression (P=0.307) or multiple logistic regression adjusting for SOAPP-R score (P=0.419). AUC values for the prediction of ADB using completion time alone, SOAPP-R score alone, and both completion time and SOAPP-R score were 0.63, 0.64, and 0.65, respectively.

Conclusion

There was no significant evidence that SOAPP-R completion times were predictive of ADB among emergency department patients. However, the AUC value for completion times was only slightly less than that for SOAPP-R total scores.

Willingness and Ability of Older Adults in the Emergency Department to Provide Clinical Information Using a Tablet Computer

OBJECTIVES

To estimate the proportion of older adults in the emergency department (ED) who are willing and able to use a tablet computer to answer questions.

DESIGN

Prospective, ED-based cross-sectional study.

SETTING

Two U.S. academic EDs.

PARTICIPANTS

Individuals aged 65 and older.

MEASUREMENTS

As part of screening for another study, potential study participants were asked whether they would be willing to use a tablet computer to answer eight questions instead of answering questions orally. A custom user interface optimized for older adults was used. Trained research assistants observed study participants as they used the tablets. Ability to use the tablet was assessed based on need for assistance and number of questions answered correctly.

RESULTS

Of 365 individuals approached, 248 (68%) were willing to answer screening questions, 121 of these (49%) were willing to use a tablet computer; of these, 91 (75%) were able to answer at least six questions correctly, and 35 (29%) did not require assistance. Only 14 (12%) were able to answer all eight questions correctly without assistance. Individuals aged 65 to 74 and those reporting use of a touchscreen device at least weekly were more likely to be willing and able to use the tablet computer. Of individuals with no or mild cognitive impairment, the percentage willing to use the tablet was 45%, and the percentage answering all questions correctly was 32%.

CONCLUSION

Approximately half of this sample of older adults in the ED was willing to provide information using a tablet computer, but only a small minority of these were able to enter all information correctly without assistance. Tablet computers may provide an efficient means of collecting clinical information from some older adults in the ED, but at present, it will be ineffective for a significant portion of this population.

Evaluation of the Swedish Web-Version of Quality of Recovery (SwQoR): Secondary Step in the Development of a Mobile Phone App to Measure Postoperative Recovery

BACKGROUND

The majority of all surgeries are performed on an outpatient basis (day surgery). The Recovery Assessment by Phone Points (RAPP) app is an app for the Swedish Web-version of Quality of Recovery (SwQoR), developed to assess and follow-up on postoperative recovery after day surgery.

OBJECTIVES

The objectives of this study are (1) to estimate the extent to which the paper and app versions of the SwQoR provide equivalent values; (2) to contribute evidence as to the feasibility and acceptability of a mobile phone Web-based app for measuring postoperative recovery after day surgery and enabling contact with a nurse; and (3) to contribute evidence as to the content validity of the SwQoR.

METHODS

Equivalence between the paper and app versions of the SwQoR was measured using a randomized crossover design, in which participants used both the paper and app version. Feasibility and acceptability was evaluated by a questionnaire containing 16 questions regarding the value of the app for follow-up care after day surgery. Content validity evaluation was based on responses by day surgery patients and the staff of the day surgery department.

RESULTS

A total of 69 participants completed the evaluation of equivalence between the paper and app versions of the SwQoR. The intraclass correlation coefficient (ICC) for the SwQoR was .89 (95% CI 0.83-0.93) and .13 to .90 for the items. Of the participants, 63 continued testing the app after discharge and completed the follow-up questionnaire. The median score was 69 (inter-quartile range, IQR 66-73), indicating a positive attitude toward using an app for follow-up after day surgery. A total of 18 patients and 12 staff members participated in the content validity evaluation. The item-level content validity index (I-CVI) for the staff group was in the 0.64 to 1.0 range, with a scale-level content validity index (S-CVI) of 0.88. For the patient group, I-CVI was in the range 0.30 to 0.92 and S-CVI was 0.67. The content validity evaluation of the SwQoR, together with three new items, led to a reduction from 34 to 24 items.

CONCLUSIONS

Day surgery patients had positive attitudes toward using the app for follow-up after surgery, and stated a preference for using the app again if they were admitted for a future day surgery procedure. Equivalence between the app and paper version of the SwQoR was found, but at the item level, the ICC was less than .7 for 9 items. In the content validity evaluation of the SwQoR, staff found more items relevant than the patients, and no items found relevant by either staff or patients were excluded when revising the SwQoR.

Feasibility of an Electronic Survey on iPads with In-Person Data Collectors for Data Collection with Health Care Professionals and Health Care Consumers in General Emergency Departments

BACKGROUND

Translating Emergency Knowledge for Kids was established to bridge the research-practice gap in pediatric emergency care by bringing the best evidence to Canadian general emergency departments (EDs). The first step in this process was to conduct a national needs assessment to determine the information needs and preferences of health professionals and parents in this clinical setting.

OBJECTIVE

To describe the development and implementation of two electronic surveys, and determine the feasibility of collecting electronic survey data on iPads with in-person data collectors in a busy clinical environment.

METHODS

Two descriptive surveys were conducted in 32 general EDs. Specific factors were addressed in four survey development and implementation stages: survey design, survey delivery, survey completion, and survey return. Feasibility of the data collection approach was determined by evaluating participation rates, completion rates, average survey time to completion, and usability of the platform. Usability was assessed with the in-person data collectors on five key variables: interactivity, portability, innovativeness, security, and proficiency.

RESULTS

Health professional participation rates (1561/2575, 60.62%) and completion rates (1471/1561, 94.23%) were strong. Parental participation rates (974/1099, 88.63%) and completion rates (897/974, 92.09%) were excellent. Mean time to survey completion was 28.08 minutes for health professionals and 43.23 minutes for parents. Data collectors rated the platform "positively" to "very positively" on all five usability variables.

CONCLUSIONS

A number of design and implementation considerations were explored and integrated into this mixed-mode survey data collection approach. Feasibility was demonstrated by the robust survey participation and completion rates, reasonable survey completion times, and very positive usability evaluation results.

The Psoriatic Arthritis Impact of Disease 12-item questionnaire: equivalence, reliability, validity, and feasibility of the touch-screen administration versus the paper-and-pencil version

BACKGROUND

Over the last few years, there has been a shift toward a more patient-centered perspective of the disease by adopting patient-reported outcomes. Touch-screen formats are increasingly being used for data collection in routine care and research.

OBJECTIVES

The aim of this study is to examine the equivalence, reliability, validity and respondent preference for a computerized touch-screen version of the Psoriatic Arthritis Impact of Disease 12-item (PsAID-12) questionnaire in comparison with the original paper-and-pencil version, in a cohort of patients with psoriatic arthritis (PsA).

METHODS

One hundred and fifty-nine patients with PsA completed both the touch screen- and the conventional paper-and-pencil administered PsAID-12 questionnaire. Agreement between formats was assessed by intraclass correlation coefficients. Spearman's rho correlation coefficient was used to test convergent validity of the touch screen format of PsAID-12, while receiver operating characteristic curve analysis was performed to test discriminant validity. In order to assess the patient's preference, the participants filled in an additional questionnaire. The time taken to complete both formats was measured.

RESULTS

A high concordance between the responses to the two modes of the PsAID-12 tested was found, with no significant mean differences. Intraclass correlation coefficients between data obtained for touch-screen and paper versions ranged from 0.801 to 0.962. There was a very high degree of correlation between the touch-screen format of PsAID-12 and composite disease activity indices (all at a P level <0.0001), Health Assessment Questionnaire, and Physician Assessment of disease activity. The discriminatory power of the touch-screen format of PsAID-12, assessed using the minimal disease activity - Outcome Measurements in Rheumatology Clinical Trials criteria, was very good, with an area under the receiver operating characteristic curve of 0.937 and a resulting cutoff value of 2.5. The touch-screen questionnaire was readily accepted and preferred. The mean time spent for completing the questionnaire on touch screen was 2 minutes and on paper was 2.7 minutes.

CONCLUSION

The touch-screen mode of administration of PsAID-12 can be a feasible and suitable alternative to the paper-and-pencil mode for the assessment of patients with PsA.

Effectiveness of a telemonitoring intensive strategy in early rheumatoid arthritis: comparison with the conventional management approach

BACKGROUND

The advent of Internet and World Wide Web has created new perspectives toward interaction between patients and healthcare professionals. Telemonitoring patients with rheumatoid arthritis (RA) is an emerging concept to guide the collaborative management treatment and improve outcomes in patients. The objective of this study was to investigate whether an intensive treatment strategy, according to a telemonitoring protocol, is more effective than conventional management strategy in reaching remission and comprehensive disease control (CDC) after 1 year in early rheumatoid arthritis (ERA) patients.

METHODS

Forty-four ERA patients were randomly allocated into two groups: the telemonitoring intensive strategy (TIS) group (group 1) or the conventional strategy (CS) group (group 2). Three patients refused to participate. In group 1 (n = 21), a remote monitoring system of disease activity, in combination with protocolised treatment adjustments aiming for remission was applied. In group 2 (n = 20), patients were treated according to daily clinical practice, with regular evaluation of disease activity, but without protocolised treatment adjustments. A telemedical care called "REmote TElemonitoring for MAnaging Rheumatologic Condition and HEaltcare programmes" (RETE-MARCHE), was developed to perform the remote monitoring.

RESULTS

A higher percentage of patients in the TIS group achieved CDAI remission vs patients in the CS group (38.1 % vs 25 % at year 1, p <0.01). Time to achieve remission was significantly shorter in the group 1 than in the group 2, with a median of 20 weeks vs a median over 36-weeks (p <0.001). Concordantly, the patients in group 1 showed a greater improvement (p <0.001), compared with group 2 in terms of functional impairment (71.4 % vs 35 %) and radiological damage progression (23.8 % vs 10 %), resulting in a greater rate of CDC (19.4 % vs 5 %).

CONCLUSIONS

According to our results, an intensive treatment strategy by telemonitoring leads to more effective disease remission and more rapid CDC than treatment according to conventional management strategy in ERA.

TRIAL REGISTRATION

TRIAL REGISTRATION NUMBER

ISRCTN13142685 Date of registration: March, 17(th) 2016.

Implementation of a mobile inpatient quality of life (QoL) assessment for oncology nursing

PURPOSE

Cancer patients suffer from a variety of symptoms, but little is known about changes during hospitalization and symptom burden at discharge. We implemented an electronic quality of life (QoL) assessment used by the nursing team in routine inpatient care. Feasibility, acceptance, and the course of QoL were investigated.

METHODS

A self-administered electronic questionnaire based on the EQ-5D and the EORTC QLQ-C30 was applied in clinical routine. Cancer patients were approached by the nursing staff to complete the QoL assessment twice, at admission and at the day of discharge. Both the feedback of the nursing staff as well as characteristics of participants were used to evaluate the electronic assessment.

RESULTS

Out of 210 patients from an oncologic ward, 85 patients (40 %) were invited to participate, 95 % of whom (n = 81) agreed to participate. Participation rate depended on the day of admission, the presence of the coordinating nurse, the overall morbidity assessed by patient clinical complexity level, and the patient age. Forty-six patients (56 %) asked for assistance in completing the questionnaire. Patients older than 53 years and male patients were more likely to need assistance. Twenty-two percent of the nursing staff (n = 5) use the information assessed for individual patient care. Fifty-two percent (n = 12) rated the additional workload as very little or little and 68 % (n = 15) agreed that handling for the patient was easy. Global QoL improved during the stay. Most severe symptoms at admission included fatigue, pain, appetite loss, and insomnia.

CONCLUSIONS

The results of this study indicate that it is feasible to implement and use an electronic QoL assessment by the nursing staff in routine inpatient cancer care. Obstacles and worries of staff members have to be considered when further developing this program.

High Level of Agreement between Electronic and Paper Mode of Administration of a Thyroid-Specific Patient-Reported Outcome, ThyPRO

INTRODUCTION AND PURPOSE

Use of electronic questionnaires to collect health-related quality-of-life data has evolved as an alternative to paper questionnaires. For the electronic questionnaire to be used interchangeably with the validated paper questionnaire, measurement properties similar to the original must be demonstrated. The aim of the present study was to assess the equivalence between the paper version and the electronic version of the thyroid-related quality-of-life questionnaire ThyPRO.

METHODS

Patients with Graves' hyperthyroidism or autoimmune hypothyroidism in a clinically stable phase were included. The patients were recruited from two endocrine outpatient centers. All patients completed both versions in a randomized test-retest set-up. Scores were compared using intraclass correlation coefficients (ICCs), paired t tests and Bland-Altman plots. Limits of agreement were compared with data from a previous paper-paper test-retest study.

RESULTS

104 patients were included. ICCs were generally high for the 13 scales, ranging from 0.76 to 0.95. There was a small but significant difference in the scale score between paper and electronic administration for the Cosmetic complaints scale, but no differences were found for any other scale. Bland-Altman plots showed similar limits of agreement compared to the earlier test-retest study of the paper version of ThyPRO.

CONCLUSION

Based on our analyses using ICCs, paired t tests and Bland-Altman plots, we found adequate agreement between the paper and electronic questionnaires. The statistically significant difference in score found in the Cosmetic complaints scale is small and probably clinically insignificant.

Subjects with osteoarthritis can easily use a handheld touch screen electronic device to report medication use: qualitative results from a usability study

OBJECTIVES

Electronic data capture is increasingly used to improve collection of patient-reported outcome measures in clinical trials and care. The validation of electronic patient-reported outcome devices requires information on patient preference and ease of use. This study conducted usability testing for a General Symptom Questionnaire and Medication Module™ on a handheld device for subjects with osteoarthritis (OA) to determine whether subjects can report on their symptoms and medication use using an electronic diary.

METHODS

Nine subjects with OA participating in a large US mode equivalency study were surveyed independently in this study. Subjects completed a General Symptom Questionnaire and Medication Module™ using the LogPad^® LW handheld device. Demographic and technology use information was collected, and the subjects were queried on device usability.

RESULTS

Subjects reported that the handheld device was easy to use and that they were able to complete a General Symptom Questionnaire and Medication Module™ with little or no assistance. They did not report any issues with the screen or size of the device. Subjects were willing to travel with the device to complete electronic diaries at home or in public. Participants indicated that they would be able to use the handheld device to answer questions during a clinical trial. Subjects with OA experienced no physical discomfort during completion of either questionnaire.

CONCLUSION

The General Symptom Questionnaire and Medication Module™ were usable and acceptable to subjects with OA on a handheld device. This was consistent regardless of previous experience and confidence with technology, despite the potential physical restrictions for an OA cohort.

Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis

PURPOSE

Technological advances in recent decades have led to the availability of new modes to administer patient-reported outcomes (PROs). To aid selecting optimal modes of administration (MOA), we undertook a systematic review to determine whether differences in bias (both size and direction) exist among modes.

METHODS

We searched five electronic databases from 2004 (date of last comprehensive review on this topic) to April 2014, cross-referenced and searched reference lists. Studies that compared two or more MOA for a health-related PRO measure in adult samples were included. Two reviewers independently applied inclusion and quality criteria and extracted findings. Meta-analyses and meta-regressions were conducted using random-effects models.

RESULTS

Of 5100 papers screened, 222 were considered potentially relevant and 56 met eligibility criteria. No evidence of bias was found for: (1) paper versus electronic self-complete; and (2) self-complete versus assisted MOA. Heterogeneity for paper versus electronic comparison was explained by type of construct (i.e. physical vs. psychological). Heterogeneity for self-completion versus assisted modes was in part explained by setting (clinic vs. home); the largest bias was introduced when assisted completion occurred in the clinic and follow-up was by self-completion (either electronic or paper) in the home.

CONCLUSIONS

Self-complete paper and electronic MOA can be used interchangeably for research in clinic and home settings. Self-completion and assisted completion produce equivalent scores overall, although heterogeneity may be induced by setting. These results support the use of mixed MOAs within a research study, which may be a useful strategy for reducing missing PRO data.

Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods

BACKGROUND

Self-administered survey questionnaires are an important data collection tool in clinical practice, public health research and epidemiology. They are ideal for achieving a wide geographic coverage of the target population, dealing with sensitive topics and are less resource-intensive than other data collection methods. These survey questionnaires can be delivered electronically, which can maximise the scalability and speed of data collection while reducing cost. In recent years, the use of apps running on consumer smart devices (i.e., smartphones and tablets) for this purpose has received considerable attention. However, variation in the mode of delivering a survey questionnaire could affect the quality of the responses collected.

OBJECTIVES

To assess the impact that smartphone and tablet apps as a delivery mode have on the quality of survey questionnaire responses compared to any other alternative delivery mode: paper, laptop computer, tablet computer (manufactured before 2007), short message service (SMS) and plastic objects.

SEARCH METHODS

We searched MEDLINE, EMBASE, PsycINFO, IEEEXplore, Web of Science, CABI: CAB Abstracts, Current Contents Connect, ACM Digital, ERIC, Sociological Abstracts, Health Management Information Consortium, the Campbell Library and CENTRAL. We also searched registers of current and ongoing clinical trials such as ClinicalTrials.gov and the World Health Organization (WHO) International Clinical Trials Registry Platform. We also searched the grey literature in OpenGrey, Mobile Active and ProQuest Dissertation & Theses. Lastly, we searched Google Scholar and the reference lists of included studies and relevant systematic reviews. We performed all searches up to 12 and 13 April 2015.

SELECTION CRITERIA

We included parallel randomised controlled trials (RCTs), crossover trials and paired repeated measures studies that compared the electronic delivery of self-administered survey questionnaires via a smartphone or tablet app with any other delivery mode. We included data obtained from participants completing health-related self-administered survey questionnaire, both validated and non-validated. We also included data offered by both healthy volunteers and by those with any clinical diagnosis. We included studies that reported any of the following outcomes: data equivalence; data accuracy; data completeness; response rates; differences in the time taken to complete a survey questionnaire; differences in respondent's adherence to the original sampling protocol; and acceptability to respondents of the delivery mode. We included studies that were published in 2007 or after, as devices that became available during this time are compatible with the mobile operating system (OS) framework that focuses on apps.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data from the included studies using a standardised form created for this systematic review in REDCap. They then compared their forms to reach consensus. Through an initial systematic mapping on the included studies, we identified two settings in which survey completion took place: controlled and uncontrolled. These settings differed in terms of (i) the location where surveys were completed, (ii) the frequency and intensity of sampling protocols, and (iii) the level of control over potential confounders (e.g., type of technology, level of help offered to respondents). We conducted a narrative synthesis of the evidence because a meta-analysis was not appropriate due to high levels of clinical and methodological diversity. We reported our findings for each outcome according to the setting in which the studies were conducted.

MAIN RESULTS

We included 14 studies (15 records) with a total of 2275 participants; although we included only 2272 participants in the final analyses as there were missing data for three participants from one included study.Regarding data equivalence, in both controlled and uncontrolled settings, the included studies found no significant differences in the mean overall scores between apps and other delivery modes, and that all correlation coefficients exceeded the recommended thresholds for data equivalence. Concerning the time taken to complete a survey questionnaire in a controlled setting, one study found that an app was faster than paper, whereas the other study did not find a significant difference between the two delivery modes. In an uncontrolled setting, one study found that an app was faster than SMS. Data completeness and adherence to sampling protocols were only reported in uncontrolled settings. Regarding the former, an app was found to result in more complete records than paper, and in significantly more data entries than an SMS-based survey questionnaire. Regarding adherence to the sampling protocol, apps may be better than paper but no different from SMS. We identified multiple definitions of acceptability to respondents, with inconclusive results: preference; ease of use; willingness to use a delivery mode; satisfaction; effectiveness of the system informativeness; perceived time taken to complete the survey questionnaire; perceived benefit of a delivery mode; perceived usefulness of a delivery mode; perceived ability to complete a survey questionnaire; maximum length of time that participants would be willing to use a delivery mode; and reactivity to the delivery mode and its successful integration into respondents' daily routine. Finally, regardless of the study setting, none of the included studies reported data accuracy or response rates.

AUTHORS' CONCLUSIONS

Our results, based on a narrative synthesis of the evidence, suggest that apps might not affect data equivalence as long as the intended clinical application of the survey questionnaire, its intended frequency of administration and the setting in which it was validated remain unchanged. There were no data on data accuracy or response rates, and findings on the time taken to complete a self-administered survey questionnaire were contradictory. Furthermore, although apps might improve data completeness, there is not enough evidence to assess their impact on adherence to sampling protocols. None of the included studies assessed how elements of user interaction design, survey questionnaire design and intervention design might influence mode effects. Those conducting research in public health and epidemiology should not assume that mode effects relevant to other delivery modes apply to apps running on consumer smart devices. Those conducting methodological research might wish to explore the issues highlighted by this systematic review.