A within-subjects trial to test the equivalence of online and paper outcome measures: the Roland Morris disability questionnaire

Measurement equivalence of the paper-based and electronic version of the Integrated Palliative care Outcome Scale (IPOS): A randomised crossover trial

BACKGROUND

The Integrated Palliative Care Outcome Scale (IPOS) validly and reliably measures symptoms and concerns of those receiving palliative care.

AIM

To determine the equivalence of the paper version with an electronic version of the IPOS (eIPOS).

DESIGN

Multicentre randomised crossover trial (NCT03879668) with a within-subject comparison of the two modes (washout period 30 min).

SETTING/PARTICIPANTS

Convenience sample of specialist inpatient and palliative home care patients aged over 18 years with cancer and non-cancer conditions was recruited. Scores were compared using intraclass correlation coefficients (ICC), Bland-Altman plots and via a mixed-effects analysis of variance.

RESULTS

Fifty patients were randomised to complete paper-electronic (n = 24) and electronic-paper (n = 26) IPOS with median age 69 years (range 24-95), 56% male, 16% non-cancer. The ICCs showed very high concordance for the total score (ICC 0.99, 95% CI 0.98-1.00), lowest ICCs being observed for symptoms 'Appetite loss' and 'Drowsiness' (ICC 0.95, 95% CI 0.92-0.97). Nine of seventeen items had ICCs above 0.98, as did all subscales. No statistically significant mode, order, age, and interaction effects were observed for IPOS total score and subscales, except for 'Communication' (F_mode = 5.9, p = 0.019). Fifty-eight percent preferred the electronic version. In the group 75+ years, 53% preferred the paper version. Only three entries in the free-text main problems differed between the versions.

CONCLUSION

The very high equivalence in scores and free text between the IPOS and the eIPOS demonstrates that eIPOS is feasible and reliable in an older palliative population.

Test reliability and comparability of paper and Chinese electronic version of the western Ontario and McMaster University osteoarthritis index: protocol for a randomised controlled clinical trial

INTRODUCTION

The Western Ontario and McMaster University osteoarthritis index (WOMAC) is the most commonly used indicator of disease-specific outcome in knee osteoarthritis for its convenience and reliability. It has two formats the paper-based WOMAC (p-WOMAC) and the electronic WOMAC (e-WOMAC). In China, the p-WOMAC has been widely used though e-WOMAC is yet untested. This study aims to test whether e-WOMAC is consistent with the p-WOMAC before and after the intervention.

METHODS AND ANALYSIS

A total of 70 patients from Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine will be randomly assigned in two groups, named, group A and group B. This study is divided into three stages. In the first stage, patients in group A will be evaluated first by p-WOMAC and then by e-WOMAC. Patients in group B will be evaluated by e-WOMAC and then by p-WOMAC. In the second stage of the study, drug interventions will be implemented. 200 mg celecoxib will be administered orally once a day starting from the second day of enrolment for a period of 21 days. In the third stage, postintervention evaluation will be conducted after administration. Patients in group A will be evaluated first by e-WOMAC and then by p-WOMAC. Patients in group B will be evaluated first by p-WOMAC and then by e-WOMAC. In order to avoid the possible bias because of patients' potential memory, e-WOMAC and p-WOMAC will be taken for each patient at 15 min apart. The primary outcome of the study is the mean score difference in WOMAC, and the secondary outcomes are the score differences in WOMAC subscales: pain, stiffness and physical function.

ETHICS AND DISSEMINATION

The protocol has been approved by the Independent Review Board of SGH (approval number: 2020-814-21-01). The results of the trial will be submitted for publication in a peer-reviewed journal.

TRIAL REGISTRATION NUMBER

ChiCTR2100050914.

Direct and mediated effects of treatment context on low back pain outcome: a prospective cohort study

OBJECTIVES

Contextual components of treatment previously associated with patient outcomes include the environment, therapeutic relationship and expectancies. Questions remain about which components are most important, how they influence outcomes and comparative effects across treatment approaches. We aimed to identify significant and strong contextual predictors of patient outcomes, test for psychological mediators and compare effects across three treatment approaches.

DESIGN

Prospective cohort study with patient-reported and practitioner-reported questionnaire data (online or paper) collected at first consultation, 2 weeks and 3 months.

SETTING

Physiotherapy, osteopathy and acupuncture clinics throughout the UK.

PARTICIPANTS

166 practitioners (65 physiotherapists, 46 osteopaths, 55 acupuncturists) were recruited via their professional organisations. Practitioners recruited 960 adult patients seeking treatment for low back pain (LBP).

PRIMARY AND SECONDARY OUTCOMES

The primary outcome was back-related disability. Secondary outcomes were pain and well-being. Contextual components measured were: therapeutic alliance; patient satisfaction with appointment systems, access, facilities; patients' treatment beliefs including outcome expectancies; practitioners' attitudes to LBP and practitioners' patient-specific outcome expectancies. The hypothesised mediators measured were: patient self-efficacy for pain management; patient perceptions of LBP and psychosocial distress.

RESULTS

After controlling for baseline and potential confounders, statistically significant predictors of reduced back-related disability were: all three dimensions of stronger therapeutic alliance (goal, task and bond); higher patient satisfaction with appointment systems; reduced patient-perceived treatment credibility and increased practitioner-rated outcome expectancies. Therapeutic alliance over task (η_p²=0.10, 95% CI 0.07 to 0.14) and practitioner-rated outcome expectancies (η_p²=0.08, 95% CI 0.05 to 0.11) demonstrated the largest effect sizes. Patients' self-efficacy, LBP perceptions and psychosocial distress partially mediated these relationships. There were no interactions with treatment approach.

CONCLUSIONS

Enhancing contextual components in musculoskeletal healthcare could improve patient outcomes. Interventions should focus on helping practitioners and patients forge effective therapeutic alliances with strong affective bonds and agreement on treatment goals and how to achieve them.

Online questionnaire, clinical and biomechanical measurements for outcome prediction of plantar heel pain: feasibility for a cohort study

BACKGROUND

Plantar heel pain (PHP) accounts for 11-15% of foot symptoms requiring professional care in adults. Recovery is variable, with no robust prognostic guides for sufferers, clinicians or researchers. Therefore, we aimed to determine the validity, reliability and feasibility of questionnaire, clinical and biomechanical measures selected to generate a prognostic model in a subsequent cohort study.

METHODS

Thirty-six people (19 females & 17 males; 20-63 years) were recruited with equal numbers in each of three groups: people with PHP (PwPHP), other foot pain (PwOP) and healthy (H) controls. Eighteen people performed a questionnaire battery twice in a randomised order to determine online and face-to-face agreement. The remaining 18 completed the online questionnaire once, plus clinical measurements including strength and range of motion, mid-foot mobility, palpation and ultrasound assessment of plantar fascia. Nine of the same people underwent biomechanical assessment in the form of a graded loaded challenge augmenting walking with added external weight and amended step length on two occasions. Outcome measures were (1) feasibility of the data collection procedure, measurement time and other feedback; (2) establishing equivalence to usual procedures for the questionnaire battery; known-group validity for clinical and imaging measures; and initial validation and reliability of biomechanical measures.

RESULTS

There were no systematic differences between online and face-to-face administration of questionnaires (p-values all > .05) nor an administration order effect (d = - 0.31-0.25). Questionnaire reliability was good or excellent (ICC2,1_absolute)(ICC 0.86-0.99), except for two subscales. Full completion of the survey took 29 ± 14 min. Clinically, PwPHP had significantly less ankle-dorsiflexion and hip internal-rotation compared to healthy controls [mean (±SD) for PwPHP-PwOP-H = 14°(±6)-18°(±8)-28°(±10); 43°(±4)- 45°(±9)-57°(±12) respectively; p < .02 for both]. Plantar fascia thickness was significantly higher in PwPHP (3.6(0.4) mm vs 2.9(0.4) mm, p = .01) than the other groups. The graded loading challenge demonstrated progressively increasing ground reaction forces.

CONCLUSION

Online questionnaire administration was valid therefore facilitating large cohort recruitment and being relevant to remote service evaluation and research. The physical and ultrasound examination revealed the expected differences between groups, while the graded loaded challenge progressively increases load and warrants future research. Clinician and researchers can be confident about these methodological approaches and the cohort study, from which useful clinical tools should result, is feasible.

LEVEL OF EVIDENCE

IV.

Electronic Data Capture Versus Conventional Data Collection Methods in Clinical Pain Studies: Systematic Review and Meta-Analysis

BACKGROUND

The most commonly used means to assess pain is by patient self-reported questionnaires. These questionnaires have traditionally been completed using paper-and-pencil, telephone, or in-person methods, which may limit the validity of the collected data. Electronic data capture methods represent a potential way to validly, reliably, and feasibly collect pain-related data from patients in both clinical and research settings.

OBJECTIVE

The aim of this study was to conduct a systematic review and meta-analysis to compare electronic and conventional pain-related data collection methods with respect to pain score equivalence, data completeness, ease of use, efficiency, and acceptability between methods.

METHODS

We searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica Database (EMBASE), and Cochrane Central Register of Controlled Trials (CENTRAL) from database inception until November 2019. We included all peer-reviewed studies that compared electronic (any modality) and conventional (paper-, telephone-, or in-person-based) data capture methods for patient-reported pain data on one of the following outcomes: pain score equivalence, data completeness, ease of use, efficiency, and acceptability. We used random effects models to combine score equivalence data across studies that reported correlations or measures of agreement between electronic and conventional pain assessment methods.

RESULTS

A total of 53 unique studies were included in this systematic review, of which 21 were included in the meta-analysis. Overall, the pain scores reported electronically were congruent with those reported using conventional modalities, with the majority of studies (36/44, 82%) that reported on pain scores demonstrating this relationship. The weighted summary correlation coefficient of pain score equivalence from our meta-analysis was 0.92 (95% CI 0.88-0.95). Studies on data completeness, patient- or provider-reported ease of use, and efficiency generally indicated that electronic data capture methods were equivalent or superior to conventional methods. Most (19/23, 83%) studies that directly surveyed patients reported that the electronic format was the preferred data collection method.

CONCLUSIONS

Electronic pain-related data capture methods are comparable with conventional methods in terms of score equivalence, data completeness, ease, efficiency, and acceptability and, if the appropriate psychometric evaluations are in place, are a feasible means to collect pain data in clinical and research settings.

Online Validation of a Battery of Questionnaires for the Assessment of Family Functioning and Related Factors

BACKGROUND

Family functioning relies on different factors that are related to the individual characteristics of each member, the social context in which the family nucleus is integrated, and the internal and interpersonal family factors. The Short Version of the Family Assessment Measure-III, Dyadic Adjustment Scale, Inventory of Parent and Peer Attachment, Twenty-item Toronto Alexithymia Scale, Hospital Anxiety and Depression Scale, and Multidimensional Scale of Perceived Social Support are among the most commonly employed self-report measures for the assessment of family functioning and related factors. Traditionally, these scales have been administered using paper-and-pencil versions. However, with increased access to the Internet, online administration of questionnaires has become more common. The present study aimed to validate an online version of each of the above-mentioned questionnaires in a heterogeneous sample of Italian healthy individuals.

METHODS

One-hundred participants were recruited for each questionnaire. A crossover design was used in each validation. The minimum important difference (MID) was applied to evaluate the differences in the variances of the paper-and-pencil and online format scores. A MID >0.5 is a reasonable first approximation of a threshold of important change. Taking into account the cross over design, mean difference between pencil-and-paper and online versions, and Intraclass Correlation Coefficient were also estimated by mixed models.

RESULTS

The MID was <0.5 for all the instruments used. Therefore, no significant difference was observed between the score variances of the paper-and-pencil and online formats of all the questionnaires. Moreover, for each questionnaire the difference between the means of online and paper-and-pencil administrations scores (mean O-P) was calculated. We reported 95% confidence intervals that did not include the 0; therefore, mean (O-P) was not statistically significant.

CONCLUSIONS

The current findings indicate that the online versions of all the questionnaires we administered can be considered reliable tools for the assessment of family functioning and related factors.

Responsiveness, Reliability, and Minimally Important and Minimal Detectable Changes of 3 Electronic Patient-Reported Outcome Measures for Low Back Pain: Validation Study

Background

The Roland Morris Disability Questionnaire (RMDQ), visual analog scale (VAS) of pain intensity, and numerical rating scale (NRS) are among the most commonly used outcome measures in trials of interventions for low back pain. Their use in paper form is well established. Few data are available on the metric properties of electronic counterparts.

Objective

The goal of our research was to establish responsiveness, minimally important change (MIC) thresholds, reliability, and minimal detectable change at a 95% level (MDC₉₅) for electronic versions of the RMDQ, VAS, and NRS as delivered via iOS and Android apps and Web browser.

Methods

We recruited adults with low back pain who visited osteopaths. We invited participants to complete the eRMDQ, eVAS, and eNRS at baseline, 1 week, and 6 weeks along with a health transition question at 1 and 6 weeks. Data from participants reporting recovery were used in MIC and responsiveness analyses using receiver operator characteristic (ROC) curves and areas under the ROC curves (AUCs). Data from participants reporting stability were used for analyses of reliability (intraclass correlation coefficient [ICC] agreement) and MDC₉₅.

Results

We included 442 participants. At 1 and 6 weeks, ROC AUCs were 0.69 (95% CI 0.59 to 0.80) and 0.67 (95% CI 0.46 to 0.87) for the eRMDQ, 0.69 (95% CI 0.58 to 0.80) and 0.74 (95% CI 0.53 to 0.95) for the eVAS, and 0.73 (95% CI 0.66 to 0.80) and 0.81 (95% CI 0.69 to 0.92) for the eNRS, respectively. Associated MIC thresholds were estimated as 1 (0 to 2) and 2 (–1 to 5), 13 (9 to 17) and 7 (–12 to 26), and 2 (1 to 3) and 1 (0 to 2) points, respectively. Over a 1-week period in participants categorized as “stable” and “about the same” using the transition question, ICCs were 0.87 (95% CI 0.66 to 0.95) and 0.84 (95% CI 0.73 to 0.91) for the eRMDQ with MDC₉₅ of 4 and 5, 0.31 (95% CI –0.25 to 0.71) and 0.61 (95% CI 0.36 to 0.77) for the eVAS with MDC₉₅ of 39 and 34, and 0.52 (95% CI 0.14 to 0.77) to 0.67 (95% CI 0.51 to 0.78) with MDC₉₅ of 4 and 3 for the eNRS.

Conclusions

The eRMDQ was reliable with borderline adequate responsiveness. The eNRS was responsive with borderline reliability. While the eVAS had adequate responsiveness, it did not have an attractive reliability profile. Thus, the eNRS might be preferred over the eVAS for measuring pain intensity. The observed electronic outcome measures’ metric properties are within the ranges of values reported in the literature for their paper counterparts and are adequate for measuring changes in a low back pain population.

Enhancing the Validity of a Quality of Life Measure for Autistic People

Accurate measurement of quality of life (QoL) is important for evaluation of autism services and trials of interventions. We undertook psychometric validation of the World Health Organisation measure-WHOQoL-BREF, examined construct validity of the WHO Disabilities module and developed nine additional autism-specific items (ASQoL) from extensive consultation with the autism community. The sample of 309 autistic people was recruited from the Adult Autism Spectrum Cohort-UK. The WHOQoL-BREF had good psychometric properties, including criterion, convergent, divergent and discriminant validity. The WHO Disabilities module showed adequate construct validity and reliability. The ASQoL items form a unitary factor of QoL, with one global item. Future studies can use the WHO measures alongside the ASQoL items to measure QoL of autistic people.

Agreement between electronic and paper Epworth Sleepiness Scale responses in obstructive sleep apnoea: secondary analysis of a randomised controlled trial undertaken in a specialised tertiary care clinic

OBJECTIVES

Originally developed as a paper questionnaire, the electronic Epworth Sleepiness Scale (ESS) is widely used in sleep clinics and sleep population research. Despite potential differences between computer-based and conventional questionnaire delivery, studies have not evaluated the agreement between electronic and paper versions of the ESS. Given the widespread use of the ESS, a bias between results would present considerable data concerns. Thus, the aim of this study was to examine agreement between electronic and paper ESS responses in obstructive sleep apnoea (OSA).

DESIGN

We undertook a secondary analysis of baseline data from a randomised controlled trial (ANZCTR: ACTRN12611000847910).

SETTING

Data were collected in a tertiary sleep research laboratory located in Sydney, Australia.

PARTICIPANTS

Data were analysed from 112 adult patients with OSA.

MEASUREMENTS

Patients were given the English version of the ESS as part of a battery of sleep laboratory questionnaires. They completed electronic and subsequently paper ESS questionnaires on the same day.

RESULTS

We found no significant difference between electronic and paper ESS questionnaires (mean=0.1, SD=2.1, 95% CI -0.3 to 0.5, P=0.57) or heteroscedasticity. There was no evidence of bias along the range of the measure. 95% limits of agreement at 4.3 and -4.1 were comparable with previous data.

CONCLUSIONS

We found no evidence of bias between electronic and paper ESS questionnaires in this sample of patients with OSA, as the two formats displayed sufficient agreement to be clinically comparable. Regardless of severity, patients reported the same level of daytime sleepiness with the same level of accuracy across both measures.

TRIAL REGISTRATION NUMBER

ACTRN12611000847910; Pre-results.

Adherence to Web-Based Self-Assessments in Long-Term Direct-to-Patient Research: Two-Year Study of Multiple Sclerosis Patients

Background

Direct-to-patient research via Web-based questionnaires is increasingly being used. Missed data or delayed reporting of data may negatively affect the quality of study results. It is insufficiently known to what degree patients adhere to agreed self-assessment schedule over the long term and whether questionnaires are filled out in a timely manner.

Objective

The objective of this study was to investigate patients’ adherence to a self-assessment schedule with low-frequency long questionnaires versus that with a high-frequency short questionnaire.

Methods

In this study, the 36-item MS Impact Profile (MSIP) questionnaire measured (perceived) disabilities and the 54-item MS Quality of Life-54 (MSQoL-54) questionnaire measured health-related quality of life at 6-month intervals. Additionally, the 2-item Medication and Adherence (MA) questionnaire documented medication and adherence to disease-modifying medication every month. An experienced MS nurse assessed the Expanded Disability Status Scale (EDSS) score via phone. For both the self-assessment schedules, we calculated the percentage of patients who had completed all the questionnaires in the first 2 years (completion adherence), the percentage of patients who completed all the questionnaires within set time frames (interval adherence), the relationship between adherence and the EDSS score, and the timing of EDSS assessment.

Results

Of the 331 patients who enrolled themselves, 301 patients completed at least one questionnaire. At month six (M6), M12, M18, and M24, the MSIP was completed by 83.4% (251/301), 71.8% (216/301), 68.1% (205/301), and 58.5% (176/301) of the patients, respectively; the MSQoL-54 by 82.1% (247/301), 71.8% (216/301), 66.8% (201/301), and 57.1% (172/301), respectively; and the MA questionnaire by 80.1% (241/301), 70.4% (212/301), 62.1% (187/301), and 53.5% (161/301), respectively. For the MSIP, 56.8% (171/301) of the patients were 2-year completion adherent; 55.5% (167/301) and 53.5% (161/301) of the patients were completion adherent for the MSQoL-54 and MA questionnaires, respectively. Whereas 85.5% (142/166) of the patients were interval adherent for the MSIP and MSQoL-54, 25.5% (41/161) were interval adherent for the MA questionnaire, with 73.9% (119/161) exceeding the maximum MA monthly interassessment interval. Completion adherence for the monthly short MA questionnaire was higher in patients with moderately high disability (EDSS 5.0-5.5) than for those with no or minimal disability (EDSS 0-2.5) (OR 5.47, 95% CI 1.08-27.69; P=.040). Completion adherence was also higher in patients with EDSS assessment within 6 months after baseline than in those with later assessment (OR 1.810, 95% CI 0.999-3.280; P=.050).

Conclusions

The 2-year completion adherence to Web-based self-assessments did not differ between the low-frequency long questionnaires and a high-frequency short questionnaire, but the interval adherence was substantially higher for the low-frequency long questionnaires. Personal contact with a member of the research team regarding a clinically relevant professional-reported outcome early in the study might positively affect the long-term completion adherence in direct-to-patient studies.

Correlates of partner support to abstain from prenatal alcohol use: a cross-sectional survey among Dutch partners of pregnant women

Partners can play an important role, but are often ignored in interventions targeting the prevention of prenatal alcohol use. A better understanding of the correlates of partner support to abstain from prenatal alcohol use can help to make a better use of partner support. The aim of this study was to identify correlates of this support by analysing differences between partners reporting low versus high support. An online cross-sectional study among 237 Dutch partners of pregnant women was conducted. Respondents were recruited through Dutch midwifery practices in September-October 2009. Questionnaires were based on the I-Change Model. Chi-square and t-test showed that partners reporting high support were more likely to desire their partner to abstain from alcohol use and to have received advice from their pregnant spouse or midwife that abstinence was desirable. They also had stronger perceptions that the baby would experience harm from prenatal alcohol use and that harm could be more severe, and they saw more advantages and fewer disadvantages of providing support. They also reported more influence from their social environment encouraging their support, had greater self-efficacy and had a stronger intention to support their partner during the remainder of the pregnancy compared to partners reporting low support. Health professionals may improve their alcohol advice by discussing the advantages and disadvantages of support with the partner and by encouraging couples to discuss and propose solutions for the situations in which partners find it difficult not to support alcohol abstinence. By providing an insight into important correlates of partner support, this study expands the research area aiming to reduce prenatal alcohol use.

Back pain beliefs in adolescents and adults in Australasia: A cross-sectional pilot study of selected psychometric properties of paper-based and web-based questionnaires in two diverse countries

BACKGROUND

It is unknown whether questionnaires measuring psychosocial constructs related to low back pain (LBP) that were originally designed for adults are suitable for adolescents, and if paper and web-versions have similar measurement properties.

OBJECTIVES

To examine selected psychometric properties for the paper- and web-based Back-Beliefs Questionnaire (BBQ) and the Fear Avoidance Beliefs Questionnaire (FABQ-phys) among adults and adolescents in two diverse countries and to determine whether differences existed between countries and pain groups.

METHODS

A sample of 156 adults (Hong Kong, n= 75; Australia, n= 81) and 96 adolescents (Hong Kong, n= 61; Australia, n= 35) participated in this cross-sectional study.

RESULTS

Main effects for country and administration mode were observed in adult BBQ scores, where Australian adults reported significantly higher BBQ scores than Hong Kong adults (mean difference (MD); 95% CI: 2.85; 0.96-4.74) and significantly higher scores were recorded on the web mode compared to the paper mode (MD 0.74; 0.10-1.38). Similarly, Hong Kong adults and adolescents reported higher FABQ-phys scores than Australian adults and adolescents (MD; 95% CI: 3.40; 1.37-5.43 and 4.88; 0.53-9.23, respectively). Internal consistency values were mostly acceptable (α ≥ 0.7).

CONCLUSION

Differences exist between cultures for LBP-related beliefs. The BBQ and FABQ-phys have acceptable measurement properties in both administration modes.

Non-specific mechanisms in orthodox and CAM management of low back pain (MOCAM): theoretical framework and protocol for a prospective cohort study

INTRODUCTION

Components other than the active ingredients of treatment can have substantial effects on pain and disability. Such 'non-specific' components include: the therapeutic relationship, the healthcare environment, incidental treatment characteristics, patients' beliefs and practitioners' beliefs. This study aims to: identify the most powerful non-specific treatment components for low back pain (LBP), compare their effects on patient outcomes across orthodox (physiotherapy) and complementary (osteopathy, acupuncture) therapies, test which theoretically derived mechanistic pathways explain the effects of non-specific components and identify similarities and differences between the therapies on patient-practitioner interactions.

METHODS AND ANALYSIS

This research comprises a prospective questionnaire-based cohort study with a nested mixed-methods study. A minimum of 144 practitioners will be recruited from public and private sector settings (48 physiotherapists, 48 osteopaths and 48 acupuncturists). Practitioners are asked to recruit 10-30 patients each, by handing out invitation packs to adult patients presenting with a new episode of LBP. The planned multilevel analysis requires a final sample size of 690 patients to detect correlations between predictors, hypothesised mediators and the primary outcome (self-reported back-related disability on the Roland-Morris Disability Questionnaire). Practitioners and patients complete questionnaires measuring non-specific treatment components, mediators and outcomes at: baseline (time 1: after the first consultation for a new episode of LBP), during treatment (time 2: 2 weeks post-baseline) and short-term outcome (time 3: 3 months post-baseline). A randomly selected subsample of participants in the questionnaire study will be invited to take part in a nested mixed-methods study of patient-practitioner interactions. In the nested study, 63 consultations (21/therapy) will be audio-recorded and analysed quantitatively and qualitatively, to identify communication practices associated with patient outcomes.

ETHICS AND DISSEMINATION

The protocol is approved by the host institution's ethics committee and the NHS Health Research Authority Research Ethics Committee. Results will be disseminated via peer-reviewed journal articles, conferences and a stakeholder workshop.

Applying Rasch analysis to evaluate measurement equivalence of different administration formats of the Activity Limitation scale of the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR)

BACKGROUND

Electronic formats of patient-reported outcome (PRO) measures are now routinely used in clinical research studies. When changing from a validated paper and pen to electronic administration it is necessary to establish their equivalence. This study reports on the value of Rasch analysis in this process.

METHODS

Three groups of US pulmonary hypertension (PH) patients participated. The first completed an electronic version of the CAMPHOR Activity Limitation scale (e-sample) and this was compared with two pen and paper administrated samples (pp1 and pp2). The three databases were combined and analysed for fit to the Rasch model. Equivalence was evaluated by differential item functioning (DIF) analyses.

RESULTS

The three datasets were matched randomly in terms of sample size (n = 147). Mean age (years) and percentage of male respondents were as follows: e-sample (51.7, 16.0 %); pp1 (50.0, 14.0 %); pp2 (55.5, 40.4 %). The combined dataset achieved fit to the Rasch model. Two items showed evidence of borderline DIF. Further analyses showed the inclusion of these items had little impact on Rasch estimates indicating the DIF identified was unimportant.

CONCLUSIONS

Differences between the performance of the electronic and pen and paper administrations of the CAMPHOR Activity Limitation scale were minor. The results were successful in showing how the Rasch model can be used to determine the equivalence of alternative formats of PRO measures.

High Level of Agreement between Electronic and Paper Mode of Administration of a Thyroid-Specific Patient-Reported Outcome, ThyPRO

INTRODUCTION AND PURPOSE

Use of electronic questionnaires to collect health-related quality-of-life data has evolved as an alternative to paper questionnaires. For the electronic questionnaire to be used interchangeably with the validated paper questionnaire, measurement properties similar to the original must be demonstrated. The aim of the present study was to assess the equivalence between the paper version and the electronic version of the thyroid-related quality-of-life questionnaire ThyPRO.

METHODS

Patients with Graves' hyperthyroidism or autoimmune hypothyroidism in a clinically stable phase were included. The patients were recruited from two endocrine outpatient centers. All patients completed both versions in a randomized test-retest set-up. Scores were compared using intraclass correlation coefficients (ICCs), paired t tests and Bland-Altman plots. Limits of agreement were compared with data from a previous paper-paper test-retest study.

RESULTS

104 patients were included. ICCs were generally high for the 13 scales, ranging from 0.76 to 0.95. There was a small but significant difference in the scale score between paper and electronic administration for the Cosmetic complaints scale, but no differences were found for any other scale. Bland-Altman plots showed similar limits of agreement compared to the earlier test-retest study of the paper version of ThyPRO.

CONCLUSION

Based on our analyses using ICCs, paired t tests and Bland-Altman plots, we found adequate agreement between the paper and electronic questionnaires. The statistically significant difference in score found in the Cosmetic complaints scale is small and probably clinically insignificant.

Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

OBJECTIVE

To conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al's 2008 review.

METHODS

A systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined.

RESULTS

Four hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95% CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95% CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95% CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001).

CONCLUSION

The present study supports the conclusion of Gwaltney's previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studies, additional work is necessary to establish this. Furthermore, there is the need to standardize migration practices and reporting practices (i.e. include copies of tested instrument versions and screenshots) so that clear recommendations regarding equivalence testing can be made in the future.raising questions about the necessity of conducting equivalence testing moving forward.

Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

OBJECTIVE

To conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al's 2008 review.

METHODS

A systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined.

RESULTS

Four hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95% CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95% CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95% CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001).

CONCLUSION

The present study supports the conclusion of Gwaltney's previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studies, additional work is necessary to establish this. Furthermore, there is the need to standardize migration practices and reporting practices (i.e. include copies of tested instrument versions and screenshots) so that clear recommendations regarding equivalence testing can be made in the future.raising questions about the necessity of conducting equivalence testing moving forward.

Reliability, validity and sensitivity of a computerized visual analog scale measuring state anxiety

BACKGROUND AND OBJECTIVES

Assessment of state anxiety is frequently required in clinical and research settings, but its measurement using standard multi-item inventories entails practical challenges. Such inventories are increasingly complemented by paper-and-pencil, single-item visual analog scales measuring state anxiety (VAS-A), which allow rapid assessment of current anxiety states. Computerized versions of VAS-A offer additional advantages, including facilitated and accurate data collection and analysis, and applicability to computer-based protocols. Here, we establish the psychometric properties of a computerized VAS-A.

METHODS

Experiment 1 assessed the reliability, convergent validity, and discriminant validity of the computerized VAS-A in a non-selected sample. Experiment 2 assessed its sensitivity to increase in state anxiety following social stress induction, in participants with high levels of social anxiety.

RESULTS

Experiment 1 demonstrated the computerized VAS-A's test-retest reliability (r = .44, p < .001); convergent validity with the State-Trait Anxiety Inventory's state subscale (STAI-State; r = .60, p < .001); and discriminant validity as indicated by significantly lower correlations between VAS-A and different psychological measures relative to the correlation between VAS-A and STAI-State. Experiment 2 demonstrated the VAS-A's sensitivity to changes in state anxiety via a significant pre- to during-stressor rise in VAS-A scores (F(1,48) = 25.13, p < .001).

LIMITATIONS

Set-order administration of measures, absence of clinically-anxious population, and gender-unbalanced samples.

CONCLUSIONS

The adequate psychometric characteristics, combined with simple and rapid administration, make the computerized VAS-A a valuable self-rating tool for state anxiety. It may prove particularly useful for clinical and research settings where multi-item inventories are less applicable, including computer-based treatment and assessment protocols. The VAS-A is freely available: http://people.socsci.tau.ac.il/mu/anxietytrauma/visual-analog-scale/.

Pedometer-based internet-mediated intervention for adults with chronic low back pain: randomized controlled trial

BACKGROUND

Chronic pain, especially back pain, is a prevalent condition that is associated with disability, poor health status, anxiety and depression, decreased quality of life, and increased health services use and costs. Current evidence suggests that exercise is an effective strategy for managing chronic pain. However, there are few clinical programs that use generally available tools and a relatively low-cost approach to help patients with chronic back pain initiate and maintain an exercise program.

OBJECTIVE

The objective of the study was to determine whether a pedometer-based, Internet-mediated intervention can reduce chronic back pain-related disability.

METHODS

A parallel group randomized controlled trial was conducted with 1:1 allocation to the intervention or usual care group. 229 veterans with nonspecific chronic back pain were recruited from one Department of Veterans Affairs (VA) health care system. Participants randomized to the intervention received an uploading pedometer and had access to a website that provided automated walking goals, feedback, motivational messages, and social support through an e-community (n=111). Usual care participants (n=118) also received the uploading pedometer but did not receive the automated feedback or have access to the website. The primary outcome was measured using the Roland Morris Disability Questionnaire (RDQ) at 6 months (secondary) and 12 months (primary) with a difference in mean scores of at least 2 considered clinically meaningful. Both a complete case and all case analysis, using linear mixed effects models, were conducted to assess differences between study groups at both time points.

RESULTS

Baseline mean RDQ scores were greater than 9 in both groups. Primary outcome data were provided by approximately 90% of intervention and usual care participants at both 6 and 12 months. At 6 months, average RDQ scores were 7.2 for intervention participants compared to 9.2 for usual care, an adjusted difference of 1.6 (95% CI 0.3-2.8, P=.02) for the complete case analysis and 1.2 (95% CI -0.09 to 2.5, P=.07) for the all case analysis. A post hoc analysis of patients with baseline RDQ scores ≥4 revealed even larger adjusted differences between groups at 6 months but at 12 months the differences were no longer statistically significant.

CONCLUSIONS

Intervention participants, compared with those receiving usual care, reported a greater decrease in back pain-related disability in the 6 months following study enrollment. Between-group differences were especially prominent for patients reporting greater baseline levels of disability but did not persist over 12 months. Primarily, automated interventions may be an efficient way to assist patients with managing chronic back pain; additional support may be needed to ensure continuing improvements.

TRIAL REGISTRATION

ClinicalTrials.gov NCT00694018; http://clinicaltrials.gov/ct2/show/NCT00694018 (Archived by WebCite at http://www.webcitation.org/6IsG4Y90E).

Integrating cancer survivors' experiences into UK cancer registries: design and development of the ePOCS system (electronic Patient-reported Outcomes from Cancer Survivors)

Background:

Understanding the psychosocial challenges of cancer survivorship, and identifying which patients experience ongoing difficulties, is a key priority. The ePOCS (electronic patient-reported outcomes from cancer survivors) project aims to develop and evaluate a cost-efficient, UK-scalable electronic system for collecting patient-reported outcome measures (PROMs), at regular post-diagnostic timepoints, and linking these with clinical data in cancer registries.

Methods:

A multidisciplinary team developed the system using agile methods. Design entailed process mapping the system's constituent parts, data flows and involved human activities, and undertaking usability testing. Informatics specialists built new technical components, including a web-based questionnaire tool and tracking database, and established component-connecting data flows. Development challenges were overcome, including patient usability and data linkage and security.

Results:

We have developed a system in which PROMs are completed online, using a secure questionnaire administration tool, accessed via a public-facing website, and the responses are linked and stored with clinical registry data. Patient monitoring and communications are semiautomated via a tracker database, and patient correspondence is primarily Email-based. The system is currently honed for clinician-led hospital-based patient recruitment.

Conclusions:

A feasibility test study is underway. Although there are possible challenges to sustaining and scaling up ePOCS, the system has potential to support UK epidemiological PROMs collection and clinical data linkage.