Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires

Electronic Patient Reported Outcome Measures and quality of life in cancer (E-PROMISE): systematic review of the evidence and meta-analysis

OBJECTIVE

The objective of this systematic review and meta-analysis (SRMA) was to evaluate the impact of electronic patient-reported outcomes (ePROs) on health-related quality of life (HRQoL) in patients with cancer.

DESIGN

We performed SRMA of randomised controlled trials (RCTs) comparing ePRO interventions with usual care in patients with cancer. The primary outcome was HRQoL. We used a random effects model a priori due to the anticipated clinical heterogeneity. Subgroup analyses and meta-regressions were performed to explore sources of heterogeneity. After assessing the risk of bias using risk-of-bias tool (RoB V.2), we rated the evidence certainty using the Grading of Recommendations, Assessment, Development and Evaluations framework.

ELIGIBILITY CRITERIA

We included studies meeting the following criteria: (1) RCTs; (2) patients diagnosed with any type of cancer, undergoing or having completed treatment; (3) comparing ePROs with usual care without ePRO interventions; (4) assessing the effect on HRQoL.

INFORMATION SOURCES

We systematically searched PubMed, Embase, and the Cochrane Central Register of Controlled Trials up to April 2024.

RESULTS

We screened 7706 records to include 36 RCTs with 9608 patients. ePRO interventions showed a standardised mean difference (SMD) of 0.35; 95% CI 0.18 to 0.51 compared with usual care. Patients receiving ongoing therapy had an SMD of 0.39 (95% CI 0.21 to 0.58), while those who had completed therapy had an SMD of 0.12 (95% CI 0.01 to 0.22), with a significant subgroup difference (p=0.01). No statistically significant differences were observed across the method of ePRO assessment, cancer site, metastasis status, therapy status, average age or duration of ePRO use. The results remained consistent with Bayesian and other sensitivity analyses.

CONCLUSIONS

ePRO interventions improve HRQoL more than usual care in patients with cancer, with greater effect in those currently undergoing therapy. This improvement is independent of cancer type, duration of ePRO use or patient age. Future research should address sources of heterogeneity, explore long-term impacts and develop strategies to increase patient engagement and adherence to ePRO systems.

PROSPERO REGISTRATION NUMBER

CRD42024531708.

Reliability of the electronic patient reported outcome measures for assessing xerostomia, dysphagia and quality of life in Spanish patients with head and neck cancer: a randomised crossover design

PURPOSE

To analyse reliability in terms of concordance (agreement) and equivalence of the Patient Reported Outcome Measures (PROM) with an electronic modality (ePROM) of the recognised questionnaires assessing of xerostomia, dysphagia and quality of life (QoL) in Spanish patients with head and neck cancer (HNC). We hypothesised notable reliability and equivalence between the two modalities.

METHODS

A total of 24 patients (median age 63.00 years, undergone radiotherapy, either alone or in combination with surgery and/or chemotherapy, and suffering xerostomia) were randomised to either paper-based (PROM) or ePROM in a two-arm crossover design with a within-subject comparison of the two modalities (washout period 90 min). Outcome measures of interest were xerostomia: severity itself (Xerostomia Inventory, XI), perceived xerostomia (visual analogue scale, VAS), regional oral dryness (Regional Oral Dryness Inventory, RODI) and dry mouth/sticky saliva (specific head and neck module European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Head and Neck Module, EORTC QLQ-H&N35 and updated EORTC QLQ-H&N43); dysphagia: swallowing burden (Eating Assessment Tool-10, EAT-10) and swallowing (EORTC QLQ-H&N35 and EORTC QLQ-H&N43); and QoL: global health (EORTC QLQ-Core 30, EORTC QLQ-C30). Data concerning the concordance between modalities was evaluated using Spearman correlation coefficients, intraclass correlation coefficients (ICCs) and Bland Altman plots with limits of agreement. In addition, a two one-sided test to check equivalence with clinical importance changes. Finally, 1-week time span separated test and retest of ePROM (only electronic modality) using Wilcoxon test and ICCs.

RESULTS

There was excellent concordance (PROM versus ePROM 0.79-0.96) with most differences fell within the limits of agreement. The equivalence analysis showed that the difference between both modalities was not more than a tolerably small amount (P < 0.05), except for dysphagia and QoL. Analysis over time exhibited from good to excellent (0.81-0.93) test-retest stability for the majority of outcome measures.

CONCLUSION

The newly developed ePROMs embedded into LAXER application have showed high level of reliability that supports their implementation in clinical practice, offering a convenient and efficient alternative to paper-based questionnaires. This study shows that electronic adaptations are possible despite the challenging older target population.

TRIAL REGISTRATION

The study is part of the LAXER study (2021-11-04 / ClinicalTrials.gov: NCT05106608).

Error rates of data processing methods in clinical research: A systematic review and meta-analysis of manuscripts identified through PubMed

BACKGROUND

In clinical research, prevention of data errors is paramount to ensuring reproducibility of trial results and the safety and efficacy of the resulting interventions. Over the last 40 years, empirical assessments of data accuracy in clinical research have been reported, however, there has been little systematic synthesis of these results. Although notable exceptions exist, little evidence exists regarding the relative accuracy of different data processing methods.

METHODS

A systematic review of the literature identified through PubMed was performed to identify studies that evaluated the quality of data obtained through data processing methods typically used in clinical trials. Quantitative information on data accuracy was abstracted from the manuscripts and pooled. Meta-analysis of single proportions based on the Freeman-Tukey transformation method and the generalized linear mixed model approach were used to derive an overall estimate of error rates across data processing methods used in each study for comparison.

RESULTS

A total of 93 papers (published from 1978 to 2008) meeting our inclusion criteria were categorized according to their data processing methods. The accuracy associated with data processing methods varied widely, with error rates ranging from 2 errors per 10,000 fields to 2,784 errors per 10,000 fields. MRA was associated with both high and highly variable error rates, having a pooled error rate of 6.57% (95% CI: 5.51, 7.72). In comparison, the pooled error rates for optical scanning, single-data entry, and double-data entry methods were 0.74% (0.21, 1.60), 0.29% (0.24, 0.35) and 0.14% (0.08, 0.20), respectively.

CONCLUSIONS

Data processing methods may explain a significant amount of the variability in data accuracy. MRA error rates, for example, were high enough to impact decisions made using the data and could necessitate increases in sample sizes to preserve statistical power. Thus, the choice of data processing methods can likely impact process capability and, ultimately, the validity of trial results.

Real-World Quality-of-Life Data in Metastatic Breast Cancer Patients Treated with CDK4/6 Inhibitors Using Four Assessment Tools

PURPOSE

To evaluate the impact of the type of CDK4/6 inhibitor administered and comorbidities on the quality of life in patients with metastatic breast cancer, as well as the correlation between quality of life and patient outcomes.

MATERIALS AND METHODS

This prospective single-center study utilized four internationally validated questionnaires: the EORTC QLQ-C30 (European Organization for Research and Treatment of Cancer Core Cancer Quality of Life Questionnaire), the Depression, Anxiety, and Stress Scale-21 (DASS-21), the Multidimensional Fatigue Inventory (MFI), and the Pittsburgh Sleep Quality Index (PSQI), administered to a cohort of 76 patients undergoing treatment for metastatic breast cancer with CDK4/6 inhibitors.

RESULTS

Ribociclib is associated with fewer insomnia problems and appears to provide better sleep quality compared to other CDK4/6 inhibitors. Mental fatigue and loss of appetite negatively influence patient survival. Patients with comorbidities reported more severe insomnia and constipation. Comorbidities are associated with a lower quality of life, reflected in greater fatigue, insomnia, and constipation, as well as increased scores for depression and stress.

CONCLUSIONS

Assessing global quality of life in metastatic breast cancer patients is essential, as it can predict patient progression and should be integrated into every breast cancer unit.

Testing an Electronic Patient-Reported Outcome Platform in the Context of Traumatic Brain Injury: PRiORiTy Usability Study

Background

Traumatic brain injury (TBI) is a significant public health issue and a leading cause of death and disability globally. Advances in clinical care have improved survival rates, leading to a growing population living with long-term effects of TBI, which can impact physical, cognitive, and emotional health. These effects often require continuous management and individualized care. Traditional paper-based assessments can be cumbersome, potentially impeding regular monitoring of patient-reported outcomes (PROs). Electronic PROs (ePROs) offer a promising alternative by enabling real-time symptom tracking, which can facilitate early identification of issues, support shared decision-making, and improve outcomes for patients with TBI.

Objective

This study evaluates the usability of an ePRO platform-Atom5-for individuals with TBI. By analyzing how patients use the system to report their symptoms, the study aims to identify usability issues, assess user satisfaction, and determine the potential of Atom5 to support ongoing patient-centered care.

Methods

Atom5 was customized to enable individuals with TBI to report their symptoms. Usability testing was conducted through one-on-one sessions with participants recruited from Headway UK-an organization supporting brain injury survivors. Each participant took part in cognitive interviews using with the "Think Aloud" method, encouraging them to verbalize their thoughts and experiences while using the platform. This approach provided qualitative insights into areas of difficulty, usability strengths, and accessibility barriers. User satisfaction was quantitatively assessed with a brief 4-item questionnaire based on the System Usability Scale. Usability outcomes were analyzed for critical and noncritical errors, focusing on user experience and overall satisfaction.

Results

In total, 9 participants completed a single usability testing session using Atom5, including 4 men, 4 women, and 1 nonbinary individual; 4 participants were under 55 years old, and 6 had their TBI <10 years ago. Finally, 8 participants used an Android device. The platform included measures for anxiety (Generalized Anxiety Disorder-2 item), depression (Patient Health Questionnaire-2), posttraumatic stress disorder (Posttraumatic Stress Disorder checklist 2), and TBI-specific quality of life (Traumatic Brain Injury - Quality of Life Short form) and a total of 26 questions. Overall, all participants were satisfied with the system, noting that it was easy to navigate and accessible despite difficulties in understanding some questions. Further, 6 participants encountered no errors, while 1 participant reported one critical error and 2 others reported one noncritical error each. The participants rated their overall satisfaction with the platform at an average score of 3.9 (SD 0.49) out of 5.

Conclusions

This usability study suggests that individuals living with TBI can effectively report symptoms using the Atom5 ePRO platform, with generally high satisfaction and few usability issues, thereby enabling continuous monitoring and proactive symptom management. Future ePRO development should focus on inclusivity and adaptability to address the diverse needs of patients with TBI, ensuring these tools can effectively support a wide range of users.

Patient Voices: Multimethod Study on the Feasibility of Implementing Electronic Patient-Reported Outcome Measures in a Comprehensive Cancer Center

BACKGROUND

"Patient Voices" is a software developed to promote the systematic collection of electronic patient-reported outcome measures (ePROMs) in routine oncology clinical practice.

OBJECTIVE

This study aimed to assess compliance with and feasibility of the Patient Voices ePROM system and analyze patient-related barriers in an Italian comprehensive cancer center.

METHODS

Consecutive patients with cancer attending 3 outpatient clinics and 3 inpatient wards were screened for eligibility (adults, native speakers, and being able to fill in the ePROMs) and enrolled in a quantitative and qualitative multimethod study. Compliance, reasons for not administering the ePROMs, patients' interaction needs, and patient-perceived System Usability Scale (range 0-100) were collected; semistructured interviews were carried out in a subsample of patients.

RESULTS

From June 2020 to September 2021, a total of 435 patients were screened, 421 (96.7%) were eligible, and 309 completed the ePROMs (309/421, 73.4%; 95% CI 69.8%-77.5%; mean age 63.3, SD 13.7 years). Organization problems and patient refusal were the main reasons for not administering the ePROMs (outpatients: 40/234, 17.1% and inpatients: 44/201, 21.9%). Help for tablet use was needed by 27.8% (47/169) of outpatients and 10.7% (15/140) of inpatients, while the support received for item interpretation was similar in the 2 groups (outpatients: 36/169, 21.3% and inpatients: 26/140, 18.6%). Average System Usability Scale scores indicated high usability in both groups (outpatients: mean 86.8, SD 15.8 and inpatients: mean 83.9, SD 18.8). Overall, repeated measurement compliance was 76.9% (173/225; outpatients only). Interviewed patients showed positive attitudes toward ePROMs. However, there are barriers to implementation related to the time and cognitive effort required to complete the questionnaires. There is also skepticism about the usefulness of ePROMs in interactions with health care professionals.

CONCLUSIONS

This study provides useful information for future ePROM implementation strategies, aimed at effectively supporting the routine clinical management and care of patients with cancer. In addition, these findings may be relevant to other organizations willing to systematically collect PROMs or ePROMs in their clinical routines.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03968718; https://clinicaltrials.gov/study/NCT03968718.

Night-time hot spring bathing is associated with improved blood pressure control: A mobile application and paper questionnaire study

Hot spring bathing practice helps to manage hypertension. However, the details of the relationship between hot spring bathing and hypertension remain unknown. Older people are thought to be less adept than younger people at using digital devices such as mobile applications. Whether mobile application questionnaires, which have been increasing in recent years, can be used by older people is unclear. To address the knowledge gap regarding the management of older patients with hypertension, we prospectively evaluated mobile application and paper questionnaires regarding night-time hot spring bathing in respondents who had a choice of which to use. Changes in blood pressure because of hot spring bathing were evaluated. To investigate the effects of night-time hot spring bathing on blood pressure in adults, 1116 volunteers at 14 institutions in Beppu completed the study, including 562 in the mobile application questionnaire group and 556 in the paper questionnaire group. A total of 474 of 477 (99.3%) respondents aged ≥65 years used paper questionnaires. There was a significantly lower drop in both systolic and diastolic blood pressure after using hot springs in respondents aged ≥65 years than in respondents aged <65 years (p<0.001). An age ≥65 years, hypertension with medication, arrhythmia, depression, and using a chloride hot spring were independently and significantly associated with a lower drop in both systolic and diastolic blood pressure after night-time hot spring bathing (p<0.001). Night-time hot spring bathing was significantly associated with reduced blood pressure in older adults (p<0.001). Extending this research by examining how psychosocial factors in respondents aged ≥65 years influence preferences for mobile and paper questionnaires may be beneficial, and further investigation is warranted.

The impact of electronic versus paper-based data capture on data collection logistics and on missing scores in thyroid cancer patients

PURPOSE

The purpose of this study was to investigate the impact of the type of data capture on the time and help needed for collecting patient-reported outcomes as well as on the proportion of missing scores.

METHODS

In a multinational prospective study, thyroid cancer patients from 17 countries completed a validated questionnaire measuring quality of life. Electronic data capture was compared to the paper-based approach using multivariate logistic regression.

RESULTS

A total of 437 patients were included, of whom 13% used electronic data capture. The relation between data capture and time needed was modified by the emotional functioning of the patients. Those with clinical impairments in that respect needed more time to complete the questionnaire when they used electronic data capture compared to paper and pencil (ORadj 24.0; p = 0.006). This was not the case when patients had sub-threshold emotional problems (ORadj 1.9; p = 0.48). The odds of having the researcher reading the questions out (instead of the patient doing this themselves) (ORadj 0.1; p = 0.01) and of needing any help (ORadj 0.1; p = 0.01) were lower when electronic data capture was used. The proportion of missing scores was equivalent in both groups (ORadj 0.4, p = 0.42).

CONCLUSIONS

The advantages of electronic data capture, such as real-time assessment and fewer data entry errors, may come at the price of more time required for data collection when the patients have mental health problems. As this is not uncommon in thyroid cancer, researchers need to choose the type of data capture wisely for their particular research question.

Error Rates of Data Processing Methods in Clinical Research: A Systematic Review and Meta-Analysis of Manuscripts Identified Through PubMed

Background

In clinical research, prevention of systematic and random errors of data collected is paramount to ensuring reproducibility of trial results and the safety and efficacy of the resulting interventions. Over the last 40 years, empirical assessments of data accuracy in clinical research have been reported in the literature. Although there have been reports of data error and discrepancy rates in clinical studies, there has been little systematic synthesis of these results. Further, although notable exceptions exist, little evidence exists regarding the relative accuracy of different data processing methods. We aim to address this gap by evaluating error rates for 4 data processing methods.

Methods

A systematic review of the literature identified through PubMed was performed to identify studies that evaluated the quality of data obtained through data processing methods typically used in clinical trials: medical record abstraction (MRA), optical scanning, single-data entry, and double-data entry. Quantitative information on data accuracy was abstracted from the manuscripts and pooled. Meta-analysis of single proportions based on the Freeman-Tukey transformation method and the generalized linear mixed model approach were used to derive an overall estimate of error rates across data processing methods used in each study for comparison.

Results

A total of 93 papers (published from 1978 to 2008) meeting our inclusion criteria were categorized according to their data processing methods. The accuracy associated with data processing methods varied widely, with error rates ranging from 2 errors per 10,000 fields to 2,784 errors per 10,000 fields. MRA was associated with both high and highly variable error rates, having a pooled error rate of 6.57% (95% CI: 5.51, 7.72). In comparison, the pooled error rates for optical scanning, single-data entry, and double-data entry methods were 0.74% (0.21, 1.60), 0.29% (0.24, 0.35) and 0.14% (0.08, 0.20), respectively.

Conclusions

Data processing and cleaning methods may explain a significant amount of the variability in data accuracy. MRA error rates, for example, were high enough to impact decisions made using the data and could necessitate increases in sample sizes to preserve statistical power. Thus, the choice of data processing methods can likely impact process capability and, ultimately, the validity of trial results.

Comparison between Swedish EORTC QLQ-C30 general population norm data published in 2000 and 2019

BACKGROUND

Normative health-related quality of life (HRQoL) data from the general population are regularly used to facilitate the interpretation of HRQoL as reported by cancer patients participating in cancer clinical trials, especially when conducting long-term follow-up studies after treatment. The aim of the present study is to compare two Swedish normative data sets, published in 2000 and 2019 respectively, and explore whether HRQoL as reported by the Swedish general population has changed over time.

MATERIAL AND METHODS

'Sample 2000' was comprised of normative data from the Swedish general population who responded to the EORTC QLQ-C30 in a Swedish mail survey in 1999 (n = 3069). 'Sample 2019' consisted of data from the Swedish general population collected as part of a European norm data study using online panels, published in 2019 (n = 1027). Data were analyzed stratified by sex and age (40-49; 50-59; 60-69; 70-79 years).

RESULTS

For most of the subscales and single items, no age group differences between the two samples were found, with the exception of the oldest age group (70-79 years), where Sample 2019 generally showed better HRQoL as compared to Sample 2000. Lower (worse) levels of Global quality of life and higher (worse) levels of Dyspnoea were found in Sample 2019 for most age groups.

CONCLUSION

There were no differences found between the samples for most EORTC QLQ-C30 subscales and single items, with the exception of the oldest age group of both sexes in Sample 2019 who reported better HRQoL on many variables. When deciding which normative dataset to use, the mode of data collection and age group have to be considered.

Selecting patient-reported outcome measures for a patient-facing technology

Objective

This article provides insight into our process and considerations for selecting patient-reported outcome measures (PROMs) designed for self-reporting symptoms and quality-of-life among breast cancer (BCA) patients undergoing oral anticancer agent treatment via a patient-facing technology (PFT) platform.

Methods

Following established guidelines, we conducted a thorough assessment of a specific set of PROMs, comparing their content to identify the most suitable options for studying BCA patients.

Results

We recommend utilizing the combination of EORTC QLQ-C30 + EORTC QLQ-BR45 as the preferred instrument, especially when developing a dedicated "breast cancer-only" application.

Discussion

When developing and maintaining a dashboard for a PFT platform that includes multiple cancer types, it is important to consider the feasibility of interface design and workload. To achieve this, we recommend using PRO-CTCAE+PROMIS 10 GH for the PFT. Moreover, it is important to consider adding ad hoc items to complement the chosen PROM(s).

Conclusion

This article describes our efforts to identify PROMs for self-reported data while considering patient and developer burdens, providing guidance to PFT developers facing similar challenges in PROM selection.

Development and usability testing of an electronic patient-reported outcome (ePRO) solution for patients with inflammatory diseases in an Advanced Therapy Medicinal Product (ATMP) basket trial

BACKGROUND

Electronic patient-reported outcome (ePRO) systems are increasingly used in clinical trials to provide evidence of efficacy and tolerability of treatment from the patient perspective. The aim of this study is twofold: (1) to describe how we developed an electronic platform for patients to report their symptoms, and (2) to develop and undertake usability testing of an ePRO solution for use in a study of cell therapy seeking to provide early evidence of efficacy and tolerability of treatment and test the feasibility of the system for use in later phase studies.

METHODS

An ePRO system was designed to be used in a single arm, multi-centre, phase II basket trial investigating the safety and activity of the use of ORBCEL-C™ in the treatment of patients with inflammatory conditions. ORBCEL-C™ is an enriched Mesenchymal Stromal Cells product isolated from human umbilical cord tissue using CD362+ cell selection. Usability testing sessions were conducted using cognitive interviews and the 'Think Aloud' method with patient advisory group members and Research Nurses to assess the usability of the system.

RESULTS

Nine patient partners and seven research nurses took part in one usability testing session. Measures of fatigue and health-related quality of life, the PRO-CTCAE™ and FACT-GP5 global tolerability question were included in the ePRO system. Alert notifications to the clinical team were triggered by PRO-CTCAE™ and FACT-GP5 scores. Patient participants liked the simplicity and responsiveness of the patient-facing app. Two patients were unable to complete the testing session, due to technical issues. Research Nurses suggested minor modifications to improve functionality and the layout of the clinician dashboard and the training materials.

CONCLUSION

By testing the effectiveness, efficiency, and satisfaction of our novel ePRO system (PROmicsR), we learnt that most people with an inflammatory condition found it easy to report their symptoms using an app on their own device. Their experiences using the PROmicsR ePRO system within a trial environment will be further explored in our upcoming feasibility testing. Research nurses were also positive and found the clinical dashboard easy-to-use. Using ePROs in early phase trials is important in order to provide evidence of therapeutic responses and tolerability, increase the evidence based, and inform methodology development.

TRIAL REGISTRATION

ISRCTN, ISRCTN80103507. Registered 01 April 2022, https://www.isrctn.com/ISRCTN80103507.

Patient-Reported Outcomes in Clinical Trials: From an Endpoint to an Intervention in Cancer Care

Underreporting of patient symptoms by clinicians is a common and well-documented phenomenon that has led to integrating patient-reported outcomes (PROs) as endpoints into clinical trials. While PROs are often used to measure disease symptoms, cancer therapy toxicities, and quality of life, they can also assess patients' general experiences and preferences. With the increasing use of electronic medical records and the digital health revolution in oncology, conversion from paper to electronic PROs (ePROs) has also facilitated the integration of PROs into routine care. Evidence from clinical trials is rapidly emerging to support ePROs as a care delivery innovation, given the potential for ePROs to improve patient outcomes through timely evaluation and response to patient needs. Meanwhile, work is ongoing to understand and address ePRO use and challenges to equitable integration, including technical and language barriers for patients, clinicians, and health systems. Nonetheless, the health system and regulatory bodies continue to develop stipulations to promote the use of ePROs. Herein, we review the evolution of PROs from an endpoint to an intervention in prospective clinical trials in oncology.

Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force

The ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) has updated the good practice recommendations from the 2009 ISPOR electronic patient-reported outcome and 2014 patient-reported outcome mixed modes Good Research Practices Task Force reports in light of accumulated evidence of measurement comparability among different modes of PROM data collection. Furthermore, with the increasing use of electronic formats of clinical outcome assessments in clinical trials and the US Food and Drug Administration's encouragement of electronic data collection, this new task force report provides stakeholders with best practice recommendations reflecting the current body of evidence and enables them to respond to future developments in research and technology. This task force recommends an evidence-based approach to determine whether new research is needed to evaluate measurement comparability for a given questionnaire or technology. The suitability of existing evidence depends upon whether it satisfactorily demonstrates that the change in data collection mode has not affected the PROM's measurement properties. In cases where sufficient evidence of measurement comparability exists and best practices for faithful migration are followed, this task force concludes that further testing of measurement comparability among the data collection modes is unnecessary, including cases of "mixing modes" within clinical trials such as bring your own device designs.

Measurement equivalence of the paper-based and electronic version of the Integrated Palliative care Outcome Scale (IPOS): A randomised crossover trial

BACKGROUND

The Integrated Palliative Care Outcome Scale (IPOS) validly and reliably measures symptoms and concerns of those receiving palliative care.

AIM

To determine the equivalence of the paper version with an electronic version of the IPOS (eIPOS).

DESIGN

Multicentre randomised crossover trial (NCT03879668) with a within-subject comparison of the two modes (washout period 30 min).

SETTING/PARTICIPANTS

Convenience sample of specialist inpatient and palliative home care patients aged over 18 years with cancer and non-cancer conditions was recruited. Scores were compared using intraclass correlation coefficients (ICC), Bland-Altman plots and via a mixed-effects analysis of variance.

RESULTS

Fifty patients were randomised to complete paper-electronic (n = 24) and electronic-paper (n = 26) IPOS with median age 69 years (range 24-95), 56% male, 16% non-cancer. The ICCs showed very high concordance for the total score (ICC 0.99, 95% CI 0.98-1.00), lowest ICCs being observed for symptoms 'Appetite loss' and 'Drowsiness' (ICC 0.95, 95% CI 0.92-0.97). Nine of seventeen items had ICCs above 0.98, as did all subscales. No statistically significant mode, order, age, and interaction effects were observed for IPOS total score and subscales, except for 'Communication' (F_mode = 5.9, p = 0.019). Fifty-eight percent preferred the electronic version. In the group 75+ years, 53% preferred the paper version. Only three entries in the free-text main problems differed between the versions.

CONCLUSION

The very high equivalence in scores and free text between the IPOS and the eIPOS demonstrates that eIPOS is feasible and reliable in an older palliative population.

The Women's Health Initiative cancer survivorship clinic incorporating electronic patient-reported outcomes: a study protocol for the Linking You to Support and Advice (LYSA) randomized controlled trial

BACKGROUND

The improved survival rate for many cancers in high-income countries demands a coordinated multidisciplinary approach to survivorship care and service provision to ensure optimal patient outcomes and quality of life. This study assesses the feasibility of introducing a Women's Health Initiative cancer survivorship clinic in Ireland.

METHODS

The trial https://spcare.bmj.com/content/9/2/209.short comprises an intervention and control arm. Two hundred participants will be recruited. Key eligibility (1) women with early-stage hormone receptor-positive breast or gynecologic cancer (cervix or endometrial), within 12 months of completion of primary curative therapy, and (2) access to the Internet. The complex intervention comprises a nurse-led clinic targeting symptom management through a trigger alert system, utilizing electronic patient-reported outcome (ePRO) assessments at baseline, and 2, 4, 6, 8, 10, and 12 months. It also includes input from a dietitian monitoring diet and nutritional status. The control group will receive their usual care pathway standard of care and attend the cancer survivorship clinic and complete ePRO assessments at the start and end of the study. The primary endpoint (feasibility) includes the proportion of enrolled participants who complete baseline and follow-up ePRO surveys and partake in health professional consultations after ePRO data triggers. Secondary endpoints include changes in cancer-related symptom scores assessed by ePROs, health-related Quality of Life Questionnaire (QLQ) scores, Appraisal Self-Care Agency-R scores, and adjuvant endocrine therapy medication adherence. A process evaluation will capture the experiences of participation in the study, and the healthcare costs will be examined as part of the economic analysis. Ethical approval was granted in December 2020, with accrual commencing in March 2021.

DISCUSSION

This protocol describes the implementation of a parallel arm randomized controlled trial (RCT) which examines the feasibility of delivering a Cancer Survivorship Clinic. The ePRO is an innovative symptom monitoring system which detects the treatment-related effects and provides individualized support for cancer survivors. The findings will provide direction for the implementation of future survivorship care.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT05035173 . Retrospectively registered on September 5, 2021.

Inclusion of information technology-based assessments of health-related quality of life in routine oncology practice in Uruguay

BACKGROUND

Previous research has shown that the inclusion of patient-reported outcomes measures in the patient's visit to the oncologists might improve the quality of global health care. The aim of the study was to assess the feasibility, acceptance, and utility perceived by patients and oncologists of health-related quality of life (HRQL) assessments obtained prior to clinical visits, and to evaluate if this has an impact on patient's well-being in a sample of Spanish-speaking patients from Uruguay.

METHODS

Patients assisted regularly in the Oncology Clinic were randomized into two groups: an intervention group that completed a set of questionnaires (European Organisation for Research and Treatment of Cancer-Quality of Life Questionnaire-C30 (EORTC QLQ-C30) and Hospital Anxiety and Depression Scale using a touch screen device and a control group that did not respond to these questionnaires. At 2 months, the responses of all the participants to the Functional Assessment of Cancer Therapy-General (FACT-G) were collected over a telephone to determine whether there were differences in the HRQL between the intervention and control groups. The graphed scores of the intervention group were included in the clinical history of the patient during consultation. Patients and physicians completed the questionnaires on the usefulness of these measurements.

RESULTS

In total, 58 patients participated in this study: 36 in the intervention group and 22 in the control group; 65% of the participants were female, and median age was 59 years (18-79). Regarding patients, 97% found the questionnaires easy to complete and thought that they included important questions. As for oncologists, 68.8% used the information and 87.5% found it useful for the consultation. There were no significant differences in the FACT-G scores between the intervention and control groups.

CONCLUSIONS

The routine HRQL assessments using an electronic device prior to the consultations were positively valued by almost all patients and physicians. This could significantly contribute to a better understanding of the patient's overall problems during consultation. These results confirm the benefits of integrating the patient's self-reported quality of life outcomes into consultations.

Supporting Medical Staff from Psycho-Oncology with Smart Mobile Devices: Insights into the Development Process and First Results

Cancer is a very distressing disease, not only for the patients themselves, but also for their family members and relatives. Therefore, patients are regularly monitored to decide whether psychological treatment is necessary and applicable. However, such monitoring processes are costly in terms of required staff and time. Mobile data collection is an emerging trend in various domains. The medical and psychological field benefits from such an approach, which enables experts to quickly collect a large amount of individual health data. Mobile data collection applications enable a more holistic view of patients and assist psychologists in taking proper actions. We developed a mobile application, FeelBack, which is designed to support data collection that is based on well-known and approved psychological instruments. A controlled pilot evaluation with 60 participants provides insights into the feasibility of the developed platform and it shows the initial results. 31 of these participants received paper-based questionnaire and 29 followed the digital approach. The results reveal an increase of the overall acceptance by 58.5% in the mean when using a digital screening as compared to the paper-based. We believe that such a platform may significantly improve cancer patients’ and relatives’ psychological treatment, as available data can be used to optimize treatment.

Validity of five foot and ankle specific electronic patient-reported outcome (ePRO) instruments in patients undergoing elective orthopedic foot or ankle surgery

BACKGROUND

Patient-reported outcomes (PROs) are widely accepted measures for evaluating outcomes of surgical interventions. As patient-reported information is stored in electronic health records, it is essential that there are valid electronic PRO (ePRO) instruments available for clinicians and researchers. The aim of this study was to evaluate the validity of electronic versions of five widely used foot and ankle specific PRO instruments.

METHODS

Altogether 111 consecutive elective foot/ankle surgery patients were invited face-to-face to participate in this study. Patients completed electronic versions of the Foot and Ankle Ability Measure (FAAM), the Foot and Ankle Outcome Score (FAOS), the modified Lower Extremity Function Scale (LEFS), the Manchester-Oxford Foot Questionnaire (MOXFQ), and the Visual Analogue Scale Foot and Ankle (VAS-FA) on the day of elective foot and/or ankle surgery. Construct validity, coverage, and targeting of the scales were assessed.

RESULTS

Based on general and predefined thresholds, construct validity, coverage, and targeting of the ePRO versions of the FAAM, the FAOS, the MOXFQ, and the VAS-FA were acceptable. Major issues arose with score distribution and convergent validity of the modified LEFS instrument.

CONCLUSIONS

The ePRO versions of the FAAM, the FAOS, the MOXFQ, and the VAS-FA provide valid scores for foot and ankle patients. However, our findings do not support the use of the modified LEFS as an electronic outcome measure for patients with orthopedic foot and/or ankle pathologies.

Follow-Up of Cancer Patients Receiving Anti-PD-(L)1 Therapy Using an Electronic Patient-Reported Outcomes Tool (KISS): Prospective Feasibility Cohort Study

BACKGROUND

Immune checkpoint inhibitors (ICIs) have become a standard of care for various tumor types. Their unique spectrum of side effects demands continuous and long-lasting assessment of symptoms. Electronic patient-reported outcome (ePRO) follow-up has been shown to improve survival and quality of life of cancer patients treated with chemotherapy.

OBJECTIVE

This study aimed to investigate whether ePRO follow-up of cancer patients treated with ICIs is feasible. The study analyzed (1) the variety of patient reported symptoms, (2) etiology of alerts, (3) symptom correlations, and (4) patient compliance.

METHODS

In this prospective, one-arm, multi-institutional study, we recruited adult cancer patients whose advanced cancer was treated with anti-programmed cell death protein 1 (PD)- ligand (L)1 agents in outpatient settings. The ePRO tool consisted of a weekly questionnaire evaluating the presence of typical side effects, with an algorithm assessing the severity of the symptom according to National Cancer Institute Common Terminology Criteria for Adverse Events and an urgency algorithm sending alerts to the care team. A patient experience survey was conducted monthly. The patients were followed up to 6 months or until disease progression.

RESULTS

A total of 889 symptom questionnaires was completed by 37 patients (lung cancer, n=15; melanoma, n=9; genitourinary cancer, n=9; head and neck cancer, n=4). Patients showed good adherence to ePRO follow-up. The most common grade 1 symptoms were fatigue (28%) and itching (13%), grade 2 symptoms were loss of appetite (12%) and nausea (12%), and grade 3-4 symptoms were cough (6%) and loss of appetite (4%). The most common reasons for alerts were loss of appetite and shortness of breath. In the treatment benefit analysis, positive correlations were seen between clinical benefit and itching as well as progressive disease and chest pain.

CONCLUSIONS

According to the results, ePRO follow-up of cancer patients receiving ICIs is feasible. ePROs capture a wide range of symptoms. Some symptoms correlate to treatment benefit, suggesting that individual prediction models could be generated.

TRIAL REGISTRATION

Clinical Trials Register, NCT3928938; https://clinicaltrials.gov/ct2/show/NCT03928938.

Intention to use electronic medical record and its predictors among health care providers at referral hospitals, north-West Ethiopia, 2019: using unified theory of acceptance and use technology 2(UTAUT2) model

BACKGROUND

Electronic Medical Records (EMRs) are systems to store patient information like medical histories, test results, and medications electronically. It helps to give quality service by improving data handling and communication in healthcare setting. EMR implementation in developing countries is increasing exponentially. But, only few of them are successfully implemented. Intention to use EMRs by health care provider is crucial for successful implementation and adoption of EMRs. However, intention of health care providers to use EMR in Ethiopia is unknown.

OBJECTIVE

The aim of this study was to assess health care provider's intention to use and its predictors towards Electronic Medical Record systems at three referral hospitals in north-west, Ethiopia, 2019.

METHODS

Institutional based cross-sectional explanatory study design was conducted from March to September among 420 health care providers working at three referral hospitals in north-west Ethiopia. Data were analyzed using structural equation model (SEM). Simple and multiple SEM were used to assess the determinants of health care providers intention to use EMRs. Critical ratio and standardized coefficients were used to measure the association of dependent and independent variables, 95% confidence intervals and P-value were calculated to evaluate statistical significance. Qualitative data was analyzed using thematic analysis.

RESULT

The mean age of the study subjects was 32.4 years ±8.3 SD. More than two-third 293(69.8%) of the participants were male. Among 420 health care providers, only 167 (39.8%) were scored above the mean of intention to use EMRs. Factors positively associated with intention to use EMRs were performance expectancy (β = 0.39, p < 0.001), effort expectancy (β = 0.24,p < 0.001),social influence (β = 0.18,p < 0.001),facilitating condition (β = 0.23,p < 0.001), and computer literacy (β = 0.08,p < 0.001). Performance expectancy was highly associated with intention to use EMRs.

CONCLUSION

Generally, about 40 % of health care providers were scored above the mean of intention to use EMRs. Performance expectancy played a major role in determining health care providers' intention to use EMRs. The intention of health care providers to use EMRs was attributed by social influence, facilitating condition in the organization, effort expectancy, performance expectancy and computer literacy. Therefore, identifying necessary prerequisites before the actual implementation of EMRs will help to improve the implementation status.

Collection of Post-treatment PRO Data in Oncology Clinical Trials

As patient-reported outcome (PRO) measures are being included more frequently in oncology clinical trials, regulatory and health technology assessment agencies have begun to request long-term, post-treatment PRO data to supplement traditional survival/progression endpoints. These data may be collected as part of cohort extension or registry studies to describe long-term outcomes of study participants after concluding their cancer treatment. While post-treatment PRO data may be expected to satisfy regulatory and payer expectations, significant practical barriers exist for the efficient incorporation of these data into oncology clinical trials, such as subject attrition, protocol deviations, and treatment crossover. The incorporation of post-treatment PRO assessments is a resource-intensive task requiring clear objectives for how the data will be analyzed and interpreted by both sponsors and regulators. Incorporating PRO data collection via electronic modalities (e.g., smartphone, web) may be a less expensive and more feasible option for incorporating long-term follow-up, reducing the frequency of manual study staff follow-up and expensive clinic visits. It is essential to include well-defined estimands for the statistical analysis, as well as to document limitations associated with the long-term follow-up data-collection approach. Analytical techniques will likely rely on descriptive and model-based statistics, and conclusions about treatment differences will likely be limited to preliminary findings of effectiveness (instead of efficacy). Finally, communications with health authorities and regulatory agencies regarding the LTFU study design and analysis should occur as early as possible to ensure that the PRO data to be collected offer an opportunity to properly evaluate the research question(s) of interest.

Quality of Life in Patients with Gynecological Cancers: A Web-Based Study

Introduction:

Gynecological cancers are common in adult women. One of the most important goals in the management of these patients is to improve quality of life, along with survival as a traditional outcome. The aim of this study was to evaluate quality of life in gynecological cancers in Iran.

Methods:

This cross-sectional study was performed on a sample of patients with gynecological cancers including uterine, ovarian, cervical, and vulvovaginal attending a teaching hospital affiliated to Tehran University of Medical Sciences between 2014 and 2019. The data was collected by a web-based platform with validated self-administered questionnaires including demographic information, the EORTC QLQ-C30 and the Hospital Anxiety and Depression (HADS). The data were analyzed using appropriate tests.

Results:

In all 251 patients were studied. The mean age of patients was 52.8±12.4 years and 43% had uterine, 30% had ovarian, 25% had cervical, and 2% had vulvovaginal cancer. The mean global quality of life score as measured by the EORTC QLQ-C30 was 59.8 ± 24.9. Women with ovarian cancer had the highest and women with cervical cancer had the lowest global quality of life score. There were significant differences in emotional, cognitive and global quality of life by cancer diagnosis (p <0.05). Although not significant, overall physical, role, cognitive and social functioning was found to be better in women who had been treated with surgery. The mean anxiety and depression score were 8.7± 5.0 and 7.1 ± 5.2, respectively.

Conclusion:

The results demonstrated that patients with gynecological cancers had a low quality of life, and experience higher anxiety and depression.

<p>Measurement Equivalence of “Touch-Screen” versus “Paper-Based” Assessments of OHRQoL: A Randomized Crossover Trial</p>

Purpose

To determine the measurement equivalence of computer touch screen assessment (CTSA) and paper based assessment (PBA) of the oral health impact profile (OHIP-14).

Patients and Methods

A randomized crossover trial was conducted. Sixty participants were randomized to either i) Arm A: completed CTSA then PBA of OHIP-14, or ii) Arm B: PBA and then CTSA of OHIP-14 within the same day. User preference and time taken to complete the assessments were recorded. Agreement between CTSA and PBA was determined using directional difference (DD), absolute difference (AD), and intraclass correlation coefficient (ICC).

Results

There was no significant difference in CTSA and PBA OHIP-14 scores (P>0.05). The magnitude of the DD in scores between assessment methods was small for overall scores and all domains (<0.3). The AD in OHIP-14 scores was small (~6% for overall score, between 8–16% for domains). Agreement between CTSA and PBA was high (ICC=0.9; 95% CI=0.8–0.9) for overall OHIP-14 scores, but ICC values varied across domains. Most (78%) preferred CTSA. There was no significant difference in time taken to complete assessments (P=0.09). Regression analyses did not identify any significant socio-demographic factor associated with absolute difference between CTSA and PBA scores.

Conclusion

There is equivalence of measurements in OHRQoL assessments from CTSA and PBA, and the time taken to complete assessment by either means is similar. There is a greater preference for CTSA. This has implications to support the use of CTSA in OHRQoL assessments.

Feasibility of perinatal mood screening and text messaging on patients' personal smartphones

Screens and adjunctive treatments for perinatal mood are available, but barriers prevent many women from receiving them. Mobile technology may help bypass barriers. The purpose of this study was to evaluate the feasibility of screening and texting perinatal women via their personal smartphones. This prospective cohort study enrolled 203 pregnant and postpartum women receiving obstetric care at a Midwestern US academic medical center. Participants received one electronic mood screen and three text messages per week for two weeks. Texts were based on the Mothers and Babies Course, a CBT-based preventative program that addresses limited social support, lack of pleasant activities, and harmful thought patterns. Feasibility was defined as the ability to take the mood screen and receive texts without technical difficulties. Demographic variables were paired with results. Insurance type (private or public) was used as a proxy for socioeconomic status. Pearson chi-squared tests were used to analyze the data. A text-based satisfaction survey was also administered. The sample was 72% privately insured and 28% publicly insured. Sixty-seven percent completed electronic screening. Screen completion was significantly associated with private insurance (OR = 3.8, 95% CI 2.00-7.30) and "married" status (OR = 1.93, 95% CI 1.01-3.70). Most survey respondents (92%) found it easy to receive the texts, and 76% responded with very favorable comments about the texts. Smartphone mood screening and supportive texting were technically feasible. Screen completion was lower among single women with public insurance.

Development of EirV3: A Computer-Based Tool for Patient-Reported Outcome Measures in Cancer

PURPOSE

Immediate transfer of patient-reported outcome measures (PROMs) for use in medical consultations is facilitated by electronic assessments. We aimed to describe the rationale and development of Eir version 3 (EirV3), a computer-based symptom assessment tool for cancer, with emphasis on content and user-friendliness.

METHODS

EirV3's specifications and content were developed through multiprofessional, stepwise, and iterative processes (from 2013 to 2016), with literature reviews on traditional and electronic assessment and classification methods, formative iterative usability tests with end-users, and assessment of patient preferences for paper versus electronic assessments.

RESULTS

EirV3 has the following two modules: Eir-Patient for PROMs registration on tablets and Eir-Doctor for presentation of PROMs in a user-friendly interface on computers. Eir-Patient starts with 19 common cancer symptoms followed by specific, in-depth questions for endorsed symptoms. The pain section includes a body map for pain location and intensity, whereas physical functioning, nutritional intake, and well-being are standard questions for all. Data are wirelessly transferred to Eir-Doctor. Symptoms with intensity scores ≥ 3 (on a 0 to 10 scale) are marked in red, with brighter colors corresponding to higher intensity, and supplemented with graphs displaying symptom development over time. Usability results showed that patients and health care providers found EirV3 to be intuitive, easy to use, and relevant. When comparing PROM assessments on paper versus tablets (n = 114), 19% of patients preferred paper, 41% preferred tablets, and 40% had no preference. Median intraclass correlation coefficient between paper and tablets (0.815) was excellent.

CONCLUSION

Iterative test rounds followed by continuous improvements led to a user-friendly, applicable symptom assessment tool, EirV3, developed for and by end-users. EirV3 is undergoing international testing of clinical and cross-cultural adaptability.

Measurement agreement of the self-administered questionnaire of the Belgian Health Interview Survey: Paper-and-pencil versus web-based mode

Before organizing mixed-mode data collection for the self-administered questionnaire of the Belgian Health Interview Survey, measurement effects between the paper-and-pencil and the web-based questionnaire were evaluated. A two-period cross-over study was organized with a sample of 149 employees of two Belgian research institutes (age range 22–62 years, 72% female). Measurement agreement was assessed for a diverse range of health indicators related to general health, mental and psychosocial health, health behaviors and prevention with kappa coefficients and intraclass correlation (ICC). The quality of the data collected by both modes was evaluated by quantifying the missing, ‘don’t know’ and inconsistent values and data entry mistakes. Good to very good agreement was found for all categorical indicators with kappa coefficients superior to 0.60, except for two mental and psychosocial health indicators namely the presence of a sleeping disorder and of a depressive disorder (kappa≥0.50). For the continuous indicators high to acceptable agreement was observed with ICC superior to 0.70. Inconsistent answers and data-entry mistakes were only occurring in the paper-and-pencil mode. There were no less missing values in the web-based mode compared to the paper-and-pencil mode. The study supports the idea that web-based modes provide, in general, equal responses to paper-and-pencil modes. However, health indicators based upon factual and objective items tend to have higher measurement agreement than indicators requiring an assessment of personal subjective feelings. A web-based mode greatly facilitates the data-entry process and guides the completing of a questionnaire. However, item non-response was not positively affected.

Are Electronic and Paper Questionnaires Equivalent to Assess Patients with Overactive Bladder?

PURPOSE

Overactive bladder syndrome is defined as urinary urgency, usually accompanied by frequency and nocturia, with or without urgency urinary incontinence in the absence of urinary tract infection or another obvious pathological condition. Electronic questionnaires have been used in a few specialties with the hope of improving treatment outcomes and patient satisfaction. However, they have not been widely used in the urological field. When treating overactive bladder, the main outcome is to improve patient quality of life. The primary objective of this study was to evaluate whether electronic questionnaires would be equally accepted as or preferred to paper questionnaires. The secondary objective was to look at the preference in relation to patient age, education and iPad® tablet familiarity.

MATERIALS AND METHODS

We prospectively evaluated the iList® electronic questionnaire application using a friendly iPad tablet in patients with overactive bladder who presented to the urology clinic at our institution. Each of the 80 patients who were recruited randomly completed the validated OABSS (Overactive Bladder Symptom Score) and the PPBC (Patient Perception of Bladder Condition) questionnaires in paper and electronic format on the tablet. Variables potentially associated with the outcomes of interest included demographic data, questionnaire method preference, patient response rate and iPad familiarity. We used the 2-sided Z-test to determine whether the proportion of patients who considered the tablet to be the same, better or much better than paper was significantly greater than 50%. The 2-sided chi-square test was applied to assess whether the intervention effect significantly differed among the demographic subgroups.

RESULTS

A total of 80 patients 21 to 87 years old were enrolled in the study from November 2015 to August 2016. Of the patients 53% were female and 49% were 65 years or younger. The incidence of those who considered the tablet to be the same or better than paper was 82.5% (95% CI 74.2-90.8, p <0.001). The incidence of patients who considered the tablet to be the same or better than paper ranged from 76% to 97% regardless of age, gender and education subgroup as well as in those with any familiarity with the tablet (each p <0.001). Of the 20 patients who were not familiar with the tablet 45% preferred the electronic questionnaire (p = 0.654).

CONCLUSIONS

We found that the proportion of patients who considered electronic questionnaires to be equivalent to or better than paper versions was higher than those who preferred paper questionnaires regardless of age, gender or education level.

From the Child's Word to Clinical Intervention: Novel, New, and Innovative Approaches to Symptoms in Pediatric Palliative Care

Despite vast improvements in disease-based treatments, many children live with life-threatening disorders that cause distressing symptoms. These symptoms can be difficult to comprehensively assess and manage. Yet, frequent and accurate symptom reporting and expert treatment is critical to preserving a patient's physical, psychological, emotional, social, and existential heath. We describe emerging methods of symptom and health-related quality-of-life (HRQOL) assessment through patient-reported outcomes (PROs) tools now used in clinical practice and novel research studies. Computer-based and mobile apps can facilitate assessment of symptoms and HRQOL. These technologies can be used alone or combined with therapeutic strategies to improve symptoms and coping skills. We review technological advancements, including mobile apps and toys, that allow improved symptom reporting and management. Lastly, we explore the value of a pediatric palliative care interdisciplinary team and their role in assessing and managing distressing symptoms and minimizing suffering in both the child and family. These methods and tools highlight the way that novel, new, and innovative approaches to symptom assessment and management are changing the way that pediatrics and pediatric palliative care will be practiced in the future.

Evaluating the impact of a 'virtual clinic' on patient experience, personal and provider costs of care in urinary incontinence: A randomised controlled trial

OBJECTIVE

To evaluate the impact of using a 'virtual clinic' on patient experience and cost in the care of women with urinary incontinence.

MATERIALS AND METHODS

Women, aged > 18 years referred to a urogynaecology unit were randomised to either (1) A Standard Clinic or (2) A Virtual Clinic. Both groups completed a validated, web-based interactive, patient-reported outome measure (ePAQ-Pelvic Floor), in advance of their appointment followed by either a telephone consultation (Virtual Clinic) or face-to-face consultation (Standard Care). The primary outcome was the mean 'short-term outcome scale' score on the Patient Experience Questionnaire (PEQ). Secondary Outcome Measures included the other domains of the PEQ (Communications, Emotions and Barriers), Client Satisfaction Questionnaire (CSQ), Short-Form 12 (SF-12), personal, societal and NHS costs.

RESULTS

195 women were randomised: 98 received the intervention and 97 received standard care. The primary outcome showed a non-significant difference between the two study arms. No significant differences were also observed on the CSQ and SF-12. However, the intervention group showed significantly higher PEQ domain scores for Communications, Emotions and Barriers (including following adjustment for age and parity). Whilst standard care was overall more cost-effective, this was minimal (£38.04). The virtual clinic also significantly reduced consultation time (10.94 minutes, compared with a mean duration of 25.9 minutes respectively) and consultation costs compared to usual care (£31.75 versus £72.17 respectively), thus presenting potential cost-savings in out-patient management.

CONCLUSIONS

The virtual clinical had no impact on the short-term dimension of the PEQ and overall was not as cost-effective as standard care, due to greater clinic re-attendances in this group. In the virtual clinic group, consultation times were briefer, communication experience was enhanced and personal costs lower. For medical conditions of a sensitive or intimate nature, a virtual clinic has potential to support patients to communicate with health professionals about their condition.

Reading on Paper and Screen among Senior Adults: Cognitive Map and Technophobia

While the senior population has been increasingly engaged with reading on mobile technologies, research that specifically documents the impact of technologies on reading for this age group has still been lacking. The present study investigated how different reading media (screen versus paper) might result in different reading outcomes among older adults due to both cognitive and psychological factors. Using a laboratory experiment with 81participants aged 57 to 85, our results supported past research and showed the influence of cognitive map formation on readers’ feelings of fatigue. We contributed empirical evidence to the contention that reading on a screen could match that of reading from paper if the presentation of the text on screen resemble that of the print. Our findings also suggested that individual levels of technophobia was an important barrier to older adults’ effective use of mobile technologies for reading. In the post hoc analyses, we further showed that technophobia was correlated with technology experience, certain personality traits, and age. The present study highlights the importance of providing tailored support that helps older adults overcome psychological obstacles in using technologies.

Large-scale clinical implementation of PROMIS computer adaptive testing with direct incorporation into the electronic medical record

The objective of this research was to assess the implementation of collecting patient-reported outcomes data in the outpatient clinics of a large academic hospital and identify potential barriers and solutions to such an implementation. Three PROMIS computer adaptive test instruments, (1) physical function, (2) pain interference, and (3) depression, were administered at 23,813 patient encounters using a novel software platform on tablet computers. The average time to complete was 3.50 ± 3.12 min, with a median time of 2.60 min. Registration times for new patients did not change significantly, 6.87 ± 3.34 to 7.19 ± 2.69 min. Registration times increased for follow-up (p = .007) from 2.94 ± 1.57 (p < .01) min to 3.32 ± 1.78 min. This is an effective implementation strategy to collect patient-reported outcomes and directly import the results into the electronic medical record in real time for use during the clinical visit.

Delivery of Patient-Reported Outcome Instruments by Automated Mobile Phone Text Messaging

BACKGROUND

Patient-reported outcome (PRO) instruments allow patients to interpret their health and are integral in evaluating orthopedic treatments and outcomes. The purpose of this study was to define: (1) correlation between PROs collected by automated delivery of text messages on mobile phones compared with paper delivery; and (2) patient use characteristics of a technology platform utilizing automated delivery of text messages on mobile phones.

METHODS

Paper versions of the 12-Item Short Form Health Survey (SF-12) and the short form of the Disabilities of the Arm, Shoulder and Hand (QuickDASH) were completed by patients in orthopedic hand and upper extremity clinics. Over the next 48 hours, the same patients also completed the mobile phone portion of the study outside of the clinic which included text message delivery of the SF-12 and QuickDASH, assigned in a random order. Correlations between paper and text message delivery of the 2 PROs were assessed.

RESULTS

Among 72 patients, the intraclass correlation coefficient (ICC) between the written and mobile phone delivery of QuickDASH was 0.91 (95% confidence interval [CI], 0.85-0.95). The ICC between the paper and mobile phone delivery of the SF-12 physical health composite score was 0.88 (95% CI, 0.79-0.93) and 0.86 (95% CI, 0.75-0.92) for the SF-12 mental health composite score.

CONCLUSIONS

We find that text message delivery using mobile phones permits valid assessment of SF-12 and QuickDASH scores. The findings suggest that software-driven automated delivery of text communication to patients via mobile phones may be a valid method to obtain other PRO scores in orthopedic patients.

Acceptability of a Touch Screen Tablet Psychosocial Survey Administered to Radiation Therapy Patients in Japan

BACKGROUND

Studies in western clinical settings suggest that touch screen computer surveys are an acceptable mode of collecting information about cancer patients' wellbeing

PURPOSE

We examined the acceptability of a touch screen tablet survey among cancer patients in Japan.

METHODS

Eligible patients (n = 262) attending a university hospital radiation therapy (RT) department were invited to complete a touch screen tablet survey about psychosocial communication and care. Survey consent and completion rates, the proportion and characteristics of patients who completed the touch screen survey unassisted, and patient-reported acceptability were assessed.

RESULTS

Of 158 consenting patients (consent rate 60 % [95 % CI 54, 66 %] of eligible patients), 152 completed the touch screen computer survey (completion rate 58 % [95 % CI 52, 64 %] of eligible patients). The survey was completed without assistance by 74 % (n = 113; 95 % CI 67, 81 %) of respondents. Older age was associated with higher odds of having assistance with survey completion (OR 1.09; 95 % CI 1.04, 1.14 %). Ninety-two percent of patients (95 % CI 86, 96 %) felt that the touch screen survey was easy to use and 95 % (95 % CI 90, 98 %) agreed or strongly agreed that they were comfortable answering the questions. Overall, 65 % (95 % CI 57, 73 %) of respondents would be willing to complete such a survey more than once while waiting for RT treatment.

CONCLUSIONS

Although patient self-reported acceptability of the touch screen survey was high, self-administered touch screen tablet surveys may not be entirely appropriate for older cancer patients or possibly for patients with lower educational attainment.

Patient-reported outcomes in cancer care - hearing the patient voice at greater volume

Recording of patient-reported outcomes (PROs) enables direct measurement of the experiences of patients with cancer. In the past decade, the use of PROs has become a prominent topic in health-care innovation; this trend highlights the role of the patient experience as a key measure of health-care quality. Historically, PROs were used solely in the context of research studies, but a growing body of literature supports the feasibility of electronic collection of PROs, yielding reliable data that are sometimes of better quality than clinician-reported data. The incorporation of electronic PRO (ePRO) assessments into standard health-care settings seems to improve the quality of care delivered to patients with cancer. Such efforts, however, have not been widely adopted, owing to the difficulties of integrating PRO-data collection into clinical workflows and electronic medical-record systems. The collection of ePRO data is expected to enhance the quality of care received by patients with cancer; however, for this approach to become routine practice, uniquely trained people, and appropriate policies and analytical solutions need to be implemented. In this Review, we discuss considerations regarding measurements of PROs, implementation challenges, as well as evidence of outcome improvements associated with the use of PROs, focusing on the centrality of PROs as part of 'big-data' initiatives in learning health-care systems.

A cluster-randomised, controlled proof-of-concept study to explore the feasibility and effect of a patient-directed intervention on quality of life in patients with advanced soft tissue sarcoma

INTRODUCTION

Even with evolving and expanding therapeutical options for the treatment of advanced sarcomas over recent years, the balance between efficacy and toxicity still remains a major concern. Moreover, the symptom burden in patients with sarcoma remains high compared with other malignant diseases. It is, therefore, crucial to assess treatment effectiveness not only in terms of disease-related outcomes (eg, overall survival) but also from an individual and patient-centred perspective using the assessment of patient-reported outcomes (PROs). By focusing on PROs as a primary study endpoint, we aim to address key issues for patients with advanced soft tissue sarcoma (STS) undergoing palliative treatment.

METHODS AND ANALYSIS

The protocol of the YonLife study describes a multicentre, cluster-randomised, controlled, open-label proof-of-concept study conducted in patients with advanced or metastatic STS treated with trabectedin in seven German hospitals. The primary objective of the study is to exploratively compare overall quality of life between the patients receiving a multidimensional intervention based on individual PROs and those receiving usual supportive treatment. This complex intervention consists of the (1) electronic assessment of PRO, (2) creation of a case vignette based on PRO and clinical data and (3) treatment suggestions based on the discussion of these vignettes in a regularly meeting expert panel. Additionally, the YonLife trial assesses the applicability of a tablet-based assessment of PROs. Patients' and physicians' acceptance and challenges concerning the implementation process will be evaluated.

ETHICS AND DISSEMINATION

The YonLife trial has been approved by the Ethics Committee of the University Hospital Dresden as well as by the relevant institutions of each participating centre before patient enrolment. The findings will be reported via relevant peer-reviewed journals as well as through presentation at international conferences.

TRIAL REGISTRATION NUMBER

NCT02204111, pre-results.

Assessing the Comparability of Paper and Electronic Versions of the EORTC QOL Module for Head and Neck Cancer: A Qualitative Study

BACKGROUND

Patient-reported outcome (PRO) instruments are important tools for monitoring disease activity and response to treatment in clinical trials and clinical practice. In recent years, there have been movements away from traditional pen-and-paper PROs towards electronic administration. When using electronic PROs (ePROs), evidence that respondents complete ePROs in a similar way to their paper counterparts provides assurance that the two modes of administration are comparable or equivalent. The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire-Core 30 item (EORTC QLQ-C30) and associated disease-specific modules are among the most widely used PROs in oncology. Although studies have evaluated the comparability and equivalence of electronic and original paper versions of the EORTC QLQ-C30, no such studies have been conducted to date for the head and neck cancer specific module (EORTC QLQ-H&N35).

OBJECTIVE

This study aimed to qualitatively assess the comparability of paper and electronic versions of the EORTC QLQ-H&N35.

METHODS

Ten head and neck cancer patients in the United States underwent structured cognitive debriefing and usability interviews. An open randomized crossover design was used in which participants completed the two modes of administration allocated in a randomized order. Using a "think-aloud" process, participants were asked to speak their thoughts aloud while completing the EORTC QLQ-H&N35. They were thoroughly debriefed on their responses to determine consistency in interpretation and cognitive process when completing the instrument in both paper and electronic format.

RESULTS

Participants reported that the EORTC QLQ-H&N35 demonstrated excellent qualitative comparability between modes of administration. The proportion of noncomparable responses (ie, where the thought process used by participants for selecting responses appeared to be different) observed in the study was low (11/350 response pairs [35 items x 10 participants]; 3.1%). Evidence of noncomparability was observed for 9 of the 35 items of the EORTC QLQ-H&N35 and in no more than 2 participants per item. In addition, there were no apparent differences in level of comparability between individual participants or between modes of administration.

CONCLUSIONS

Mode of administration does not affect participants' response to, or interpretation of, items in the EORTC QLQ-H&N35. The findings from this study add to the existing evidence supporting the use of electronic versions of the EORTC instruments when migrated to electronic platforms according to best practice guidelines.

Implementation and first results of a tablet-based assessment referring to patient-reported outcomes in an inpatient cancer care unit

BACKGROUND

Inclusion of patient-reported outcomes (PROs) in routine cancer care is of key importance for individualized treatment, shared decision making and patient satisfaction.

OBJECTIVE

To describe the implementation under routine conditions of an electronic self-administered PRO assessment and comparison of PROs before and after inpatient treatment in oncologic care.

METHODS

In a tablet-based survey PROs on symptom burden, global health status/ quality of life (QoL) and health utility were collected twice (at hospital admission and discharge) in an inpatient oncological setting over a 17-month period using the EORTC QLQ-C30 and EQ-5D questionnaires. Data were linked to the hospital information system (HIS). Patient acceptability, recruitment rates, symptom burden, and clinically meaningful changes in PROs over time were analyzed.

RESULTS

From a total of 384 hospitalized patients invited to participate at admission 371 (96.6 %) participated. At discharge, 195 patients were approached for a follow-up assessment, and 192 patients (98.5 %) participated. Despite strong acceptance among patients, recruitment rates were decreasing over time. During the hospital stay clinically meaningful improvements were observed for health utility (33.3 %, n = 64) and global health status/QoL (43.2 %, n = 83). Patients reported a variety of symptoms at admission and discharge.

CONCLUSIONS

Implementation of PRO assessment in routine care and data integration into the HIS provides valuable information for the entire medical staff as symptom burden is present during the entire hospital stay.

IMPLICATIONS FOR PRACTICE

Long-term maintenance of PRO assessment in a clinical setting as a prerequisite of value-based healthcare requires continuous involvement of the nursing team, which can only be achieved by allocating resources to this task.

Evaluation of Noncompletion Bias and Long-Term Adherence in a 10-Year Patient-Reported Outcome Monitoring Program in Clinical Routine

BACKGROUND

Currently there is little knowledge on real-life sustainability of routine patient-reported outcome (PRO) measurement and the representativeness of collected data.

OBJECTIVES

The investigation of routine PRO with regard to noncompletion bias and long-term adher- ence, considering the potential impact of mode of assessment (MOA) (paper-pencil vs. electronic PRO [ePRO]) and patient characteristics.

METHODS

At our department, routine PRO measurement in oncological patients is being done since 2005 using different MOA (paper-pencil assessment until 2011 and ePRO assessment from 2011 onward). We analyzed two different patient groups: patients eligible in both periods (both-MOA group) and patients eligible in only one period (one-MOA group). The primary outcome was PRO noncompletion (100% missing questionnaires). The secondary outcome was poor PRO adherence (>20% missing questionnaires). Multivariate logistic regression models were developed, testing the impact of MOA and patient characteristics on the outcomes in the different patient groups.

RESULTS

Data from 1484 eligible patients were included in the analyses. Most of the patients could be included in PRO assessment at least once. PRO noncompletion rates were clearly higher during paper-pencil assessment (odds ratios between 2.72 and 4.31), as were poor PRO adherence rates (odd ratio 2.23). Analyses of potential bias by patient characteristics showed that male patients had a higher risk of poor adherence. Other factors with significant impact were age, country, and cancer diagnosis, but results were indecisive.

CONCLUSIONS

ePRO increased the feasibility of our clinical routine PRO data for retrospective analyses by increasing completion rates. In general, potential completion bias regarding certain patient characteristics requires attention before generalizing results to the respective populations.

A Computerized Family Planning Counseling Aid: A Pilot Study Evaluation of Smart Choices

CONTEXT

Resource constraints may make it challenging for family planning clinics to provide comprehensive contraceptive counseling; technological tools that help providers follow recommended practices without straining resources merit evaluation.

METHODS

A pilot study using a two-group, posttest-only experimental design evaluated Smart Choices, a computer-based tool designed to help providers offer more patient-centered counseling and enable patients to participate proactively in the counseling session. In two North Carolina family planning clinics, 214 women received usual counseling in March-May 2013, and 126 women used Smart Choices in May-July 2013. Exit interviews provided data for the evaluation. Multivariate Poisson and multinomial logistic regression analyses were performed to examine group differences in counseling outcomes.

RESULTS

Three of 12 hypotheses tested were supported: Compared with controls, women in the intervention group knew more contraceptive methods (adjusted mean, 11.1 vs. 10.7); discussed more topics related to sexual health during counseling (1.2 vs. 0.9 among those reporting any discussion); and rated counseling as more patient-centered, an indication of how well they felt providers understood their family planning circumstances and ideas (3.9 vs. 3.7 on a scale of 1-4). Contrary to another hypothesis, controls were more likely than women in the intervention group to choose IUDs and implants.

CONCLUSIONS

Computerized counseling aids like Smart Choices are in an early stage of development. Future research is warranted to develop tools that lead to more productive and individualized clinic visits and, ultimately, to more effective contraceptive use and reduced levels of unintended pregnancy.

The prognostic value of pre-diagnosis health-related quality of life on survival: a prospective cohort study of older Americans with lung cancer

PURPOSE

Health-related quality of life (HRQOL) after cancer diagnosis is prognostic for overall survival (OS). However, no studies have assessed if HRQOL before diagnosis is predictive for OS. The objective of this study was to determine the association between pre-lung cancer diagnosis HRQOL and OS.

METHODS

Our prospective cohort study used surveillance, epidemiology, and end results linked to the Medicare Health Outcomes Survey. We included 6290 individuals 65 years or older diagnosed with incident lung cancer from 1998 to 2013. We assessed the prognostic value of (1) short-form 36 summary component and domain-specific scores, (2) activities of daily living (ADL), and (3) two global HRQOL questions. Cox-proportional hazards models were used to examine associations between HRQOL and OS, adjusting for demographics, comorbid conditions, and clinical characteristics.

RESULTS

Worse pre-diagnosis HRQOL was significantly associated with greater risk of death across HRQOL measures. An above average physical or mental component summary score was associated with 16 and 24% decreases in the hazard of death, respectively (p < 0.0001). Being unable to perform ADLs, such as bathing oneself, was associated with an 89% increased hazard of death (p < 0.0001). Reporting "poor" versus "excellent" health was associated with a 74% increase in the hazard of death (p < 0.0001).

CONCLUSION

This population-based study reinforces the importance of self-reported health status as a predictor for OS. Routine HRQOL screening may identify patients who could benefit from early interventions to improve HRQOL. Future studies should explore associations between changes in HRQOL before and after cancer diagnosis and OS.

Evaluation of quality of life using a tablet PC-based survey in cancer patients treated with radiotherapy: a multi-institutional prospective randomized crossover comparison of paper and tablet PC-based questionnaires (KROG 12-01)

PURPOSE

This study compared a tablet PC questionnaire with a paper method for reliability and patient preferences in the acquisition of patient-reported outcomes (PROs) for patients treated with radiotherapy. By comparing the two modes of PRO administration, we aimed to evaluate the adequacy of using tablet PC questionnaires in future clinical use.

METHODS

Patients were randomized in a crossover study design using two different methods for PRO entry. A group of 89 patients answered a paper questionnaire followed by the tablet PC version, whereas 89 patients in another group completed the tablet PC questionnaire followed by the paper version. Surveys were performed four times per patient throughout the course of the radiotherapy. The Korean versions of the M.D. Anderson Symptom Inventory (MDASI-K) and the Brief Fatigue Inventory (BFI-K) were used. The primary endpoint of our current study was an assessment of patient preference for the survey method. The proportions of patients preferring each mode of questionnaire were evaluated.

RESULTS

The proportion of patients who preferred the tablet PC version, paper form, or who had no preference was 52.2, 22.1, and 25.7 %, respectively. More than half of the patients preferred the tablet PC to the paper version in all four surveys. Age, gender, educational status, prior experience of using a tablet PC, and the order of paper to tablet PC administration did not impact patient preferences. Inter-class correlation coefficients (ICCs) between the modes were 0.92 for MDASI-K and 0.94 for BFI-K and ranged from 0.91 to 0.96 on both instruments during the four surveys.

CONCLUSIONS

A tablet PC-based PRO is an acceptable and reliable method compared with paper-based data collection for Korean patients receiving radiotherapy.

REDLetr: Workflow and tools to support the migration of legacy clinical data capture systems to REDCap

OBJECTIVE

A memory clinic at an academic medical center has relied on several ad hoc data capture systems including Microsoft Access and Excel for cognitive assessments over the last several years. However these solutions are challenging to maintain and limit the potential of hypothesis-driven or longitudinal research. REDCap, a secure web application based on PHP and MySQL, is a practical solution for improving data capture and organization. Here, we present a workflow and toolset to facilitate legacy data migration and real-time clinical research data collection into REDCap as well as challenges encountered.

MATERIALS AND METHODS

Legacy data consisted of neuropsychological tests stored in over 4000 Excel workbooks. Functions for data extraction, norm scoring, converting to REDCap-compatible formats, accessing the REDCap API, and clinical report generation were developed and executed in Python.

RESULTS

Over 400 unique data points for each workbook were migrated and integrated into our REDCap database. Moving forward, our REDCap-based system replaces the Excel-based data collection method as well as eases the integration into the standard clinical research workflow and Electronic Health Record.

CONCLUSION

In the age of growing data, efficient organization and storage of clinical and research data is critical for advancing research and providing efficient patient care. We believe that the workflow and tools described in this work to promote legacy data integration as well as real time data collection into REDCap ultimately facilitate these goals.

Test-Retest Reliability of an Interactive Voice Response (IVR) Version of the EORTC QLQ-C30

OBJECTIVE

The objective of this study was to assess the test-retest reliability of an interactive voice response (IVR) version of the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30.

METHODS

A convenience sample of outpatient cancer clinic patients (n = 127) was asked to complete the IVR version of the QLQ-C30 twice, 2 days apart. The QLQ-C30 is a 30-item, cancer-specific questionnaire composed of single-item and multi-item scales. The instrument has five functional scales (physical, role, cognitive, emotional, and social), three symptom scales (fatigue, pain, and nausea/vomiting), and a global quality-of-life scale. The remaining single items assess dyspnea, appetite loss, insomnia, constipation, diarrhea, and financial problems. The analyses focused on intraclass correlation coefficients (ICCs), comparing the ICC 95 % lower confidence interval (CI) value with a critical value of 0.70.

RESULTS

The ICCs for the nine multi-item scales were all above 0.69, ranging from 0.698 to 0.926 (ICC 95 % lower CI value range 0.588-0.895). All of the scales were significantly different from our threshold reliability of 0.70, with the exception of the cognitive functioning scale. The ICCs for the six single items ranged from 0.741 to 0.883 (ICC 95 % lower CI value range 0.646-0.835), and three of the six were statistically different from 0.70. The evidence supports the stability of 11 of the 15 scores obtained on the IVR version of the QLQ-C30 upon repeated measurement.

CONCLUSION

The measurement equivalence of the IVR and paper versions of the QLQ-C30 has been reported elsewhere. This analysis provides evidence demonstrating adequate test-retest reliability of the IVR version for 11 of the QLQ-C30's 15 scores.

Psychometric equivalence of electronic and telephone completion of the ICIQ modules

AIMS

To assess the equivalence of touch-screen (hand-held iPad) and telephone completion of patient-completed International Consultation on Incontinence Questionnaire (ICIQ) modules by comparison with corresponding data collected using conventional paper-and-pencil methods.

METHODS

Men and women, attending urology outpatients complaining of LUTS, were randomised to one of three groups which determined the order in which they completed three administrations of the same questionnaire: paper, iPad and telephone. Four ICIQ questionnaires were evaluated: ICIQ-MLUTS, ICIQ-LUTSqol, ICIQ-OABqol and ICIQ-UI SF.

RESULTS

From August 2012 to October 2014 a total of 448 out of 491 (91%) recruits completed the first two administrations and were included in the analysis. Three hundred forty-eight out of 491 (71%) completed the phone administration. The intra-class correlation coefficient (ICC) and Kappa statistic were calculated where appropriate between completed pairs of administrations. Mean ICC correlations were high (>0.8) between paper and iPad administrations. Paired paper and phone administrations were less well correlated, although still high (mean ICC>0.75). This may be partly due to the practical limitation that the phone interview was completed up to a week later than the initial two administrations. There was no evidence that potential moderator effects (gender, age and experience with computers or touch screen devices) significantly affected overall reliability of scores between administrations.

CONCLUSIONS

We can recommend the interchangeable use of ICIQ electronic or paper based questionnaires in a clinical or research setting. Self-report is preferable to telephone delivery where possible. Neurourol. Urodynam. 9999:XX-XX, 2016. © 2016 Wiley Periodicals, Inc.

The usefulness of a mobile device-based system for patient-reported outcomes in a spine outpatient clinic

BACKGROUND CONTEXT

Patient-reported outcomes (PROs) are typically collected using a paper form, but this format is cumbersome to incorporate into outpatient clinic visits as well as in research. Therefore, we developed a mobile device-based system (mobile system) for spinal PRO. We hypothesized that this system may improve the quality of care in an outpatient clinic.

PURPOSE

This study aimed to analyze the patient-reported efficacy of a mobile system through a survey of patients' responses compared with a paper system.

STUDY DESIGN/SETTING

A prospective observational study was carried out.

PATIENT SAMPLE

Surveys were conducted for 103 patients who had experience using both the paper and electronic systems in the outpatient clinic.

OUTCOME MEASURES

Patient-reported positive response score (PRS) was the outcome measure.

METHODS

The survey included the characteristics of the patients (sex, age, use of smartphone, familiarity with smartphone applications, proficiency of typing with mobile device, site of pain, and education level) and eight questions in four domains: (1) efficacy in the waiting room, (2) efficacy during the clinic visit, (3) overall satisfaction, and (4) opinion about the use of this system. The response to each question was scored from 1 to 5 (1, negative; 5, positive response). The patient-reported PRS was calculated by adding the scores of the 8 questions and converting the total range to 0-100 (60, neutral).

RESULTS

The mean PRS of the 8 questions was 79.8 (95% CI, 76.7-83.9). The mean PRS was 78.9 (75.6-82.2) at the waiting room and was 80.5 (77.1-83.9) during the clinic. The PRS for overall satisfaction and use of this system were 83.3 (79.6-87.0) and 77.1 (71.9-82.3), respectively. The use of smartphones and the proficiency of typing were independently significant predictors of PRS with an R(2) value of 0.325.

CONCLUSIONS

The mobile device-based system improved the patient-reported efficacy in spine outpatient clinics. However, various factors such as the use of smartphones need to be considered when developing and applying mobile systems.