Psychometric evaluation of patient-reported experience measures: is it valid?

Patient reported outcome and experience measures among patients with central venous access devices: a systematic review

PURPOSE

Patients receiving treatment for solid tumours and haematological malignancies, among other acute and chronic health conditions, are highly dependent upon central venous access devices (CVADs) for administering chemotherapy and other complex therapies; thus, CVADs can meaningfully impact their health outcomes and experiences. This systematic review aimed to identify and critique patient-reported outcome measure (PROM) and patient-reported experience measure (PREM) instruments related to CVADs.

METHODS

A systematic review was undertaken, commencing with an electronic search of health databases (April 2022). Studies were eligible if they used a self-reporting instrument (questionnaire) to quantitatively measure patient-reported outcomes and experiences related to CVADs (English only). Using a piloted data-extraction tool, two authors independently identified studies for full review, data extraction, and quality assessment. Data were synthesised narratively.

RESULTS

The search yielded 875 titles, of which 41 met the inclusion and no exclusion criteria. Of these, 31 reported results of purpose-built questionnaires; a further six reported results of generic measures used for CVADs; four included both purpose-built and generic measures. Overall study quality was low; only two studies evaluated both content validity and internal consistency. In total, 155 unique PROM items (across 27 studies) were extracted which encompassed five domains (e.g., 'Instrumental activities of daily living'; 'Pain and discomfort'). Similarly, 184 unique PREMs (from 31 studies) included 13 domains (e.g., 'Shared decision-making'; 'Education').

CONCLUSION

Increasingly, research and quality improvement studies about CVADs are incorporating PROM and PREM. These measures are largely purpose-built, however, and their validity and reliability have not been sufficiently established for use.

REVIEW REGISTRATION

Prospectively submitted to the International prospective register of systematic reviews (PROSPERO) 05 July 2020.

International survey of people living with chronic conditions (PaRIS survey): effects of general practitioner non-participation on the representativeness of the Norwegian patient data

BACKGROUND

The International Survey of People living with Chronic Conditions (OECD-PaRIS survey), aims to systematically gather patient-reported experiences (PREMs) and - outcomes (PROMs) and potential predictors for these outcomes for persons with chronic conditions as well as information from professionals about health care provided. In such patient surveys, the advantages of a multilevel (nested) approach in which patients are sampled 'within providers' need to be balanced against the potential for bias if patient populations from participating GPs significantly differ from those of non-participating GPs. The objective was to assess the effects of general practitioner (GP) non-participation on the representativeness of the Norwegian patient data of the International Survey of People living with Chronic Conditions (OECD-PaRIS survey).

METHODS

To test all aspects of the first main PaRIS survey, it was preceded by a field trial which this paper reports on the Norwegian part of. For the Norwegian part of the field trial in 2022, we randomly sampled and surveyed 75 GPs and 125 patients 45 years and older for each GP, regardless of whether their GP were also participating in the study. GPs were sampled from a national register that included all GPs. The surveys were primarily digital, but we sent postal questionnaires to non-digital patients and non-responding digital patients. We compared GP and patient characteristics as well as patient-reported experiences and outcomes according to GP participation status in bivariate analysis, supplemented with multiple linear regressions with PREMs/PROMs as dependent variables and participation status as independent adjusting for significant patient factors.

RESULTS

17 of 75 sampled GPs participated (22.7%), of which 993 of 2,015 patients responded (49.3%). 3,347 of 7,080 patients of non-responding GPs answered (47.3%). Persons with chronic conditions from participating GPs reported significantly better patient-centred coordinated care (p = 0.017), overall experiences with the GP office the last 12 months (p = 0.004), mental well-being (p = 0.039) and mental health (p = 0.013) than patients from non-participating GPs. The raw differences between participating and non-participating GPs on patient-reported experiences and - outcomes varied from 1.5 to 2.9 points on a 0-100 scale, and from 2.2 to 3.0 after adjustment for case-mix.

CONCLUSIONS

The Norwegian field trial indicates that estimates based on participants in the PaRIS survey may modestly overestimate patient-reported experiences and -outcomes at the aggregated level and the need for more research within and across countries to identify and address this potential bias.

Developing and exploring the validity of a patient reported experience measure for adult inpatient diabetes care

AIM

To develop and explore the validity of a Patient Reported Experience Measure (PREM) for adult inpatient diabetes care.

METHOD

27 in-depth interviews were conducted to inform the development of the 42-item PREM which was cognitively tested with 10 people. A refined 38-item PREM was piloted with 228 respondents completing a paper (n = 198) or online (n = 30) version. The performance of the PREM was evaluated by exploring (i) uptake/number of responses and (ii) survey validity by investigating whether the PREM data were of adequate quality and delivered useful information.

RESULTS

The PREM had low drop-out or missing data rates suggesting it was appropriately constructed. Analysis of item frequencies and variances, and problem score calculations concluded that questions provided sufficient score differentiation.

CONCLUSIONS

This new PREM allows for experiences of inpatient diabetes care to be measured, understood and reported on to help identify priority areas for improving care quality.

Patientenzentrierte Qualitätssicherung durch PREMs und PROMs: Befragungen von Patientinnen und Patienten mit einer perkutanen Koronarintervention (PCI) und Koronarangiografie

BACKGROUND

The Institute of Quality Assurance and Transparency in Health Care (IQTIG) was commissioned by the Federal Joint Committee (G-BA) to include the patient perspective in the quality assurance programme for patients undergoing percutaneous cardiac intervention (PCI) and coronary angiography ("QS PCI"). This article describes both the development methodology and the survey-based quality indicators.

METHODS

On the basis of a systematic literature research, focus groups with patients, interviews with doctors and an expert group meeting, quality criteria were defined which were relevant to patients. These criteria were translated into PREMs and PROMs. The questionnaires were subjected to a two-phase pretest. Quality indicators were composed by aggregating the items.

RESULTS

Altogether, 12 topic areas were identified which are relevant to patients receiving percutaneous cardiac intervention or coronary angiography. Here, communication and interaction were of high importance. In addition, information before, during and after the procedure was very relevant, and, likewise, the health care professionals' interaction with patients. Also, symptoms and treatment effects were important. In accordance with the defined topics, 19 quality indicators were developed which can display quality of health care from the patients' perspective.

CONCLUSION

The development of PREMs and PROMs expanded the quality assurance programme "QS PCI" to include significant dimensions relevant to patients, which can, therefore, provide valuable information to improve patient-centred care.

Use of Patient-Reported Experience Measures in Pediatric Care: A Systematic Review

Introduction: Patient-reported Experience Measures (PREMs) are validated questionnaires, that gather patients' and families' views of their experience receiving care and are commonly used to measure the quality of care, with the goal to make care more patient and family-centered. PREMs are increasingly being adopted in pediatric population, however knowledge gaps exist around understanding the use of PREMs in pediatrics. Objective: To identify and synthesize evidence on the use of PREMs in pediatric healthcare settings and their characteristics. Evidence Review: Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines governed the conduct and reporting of this review. An exhaustive search strategy was applied to MEDLINE, EMBASE, PsycINFO, Cochrane Library, and CINAHL databases to identify relevant peer-reviewed articles from high-income countries. Additionally, gray literature was searched to capture real-world implementation of PREMs. All the articles were screened independently by two reviewers in two steps. Data was extracted independently, synthesized, and tabulated. Findings from gray literature was synthesized and reported separately. Risk of bias for the studies identified through scientific databases was assessed independently by two reviewers using the National Institutes of Health Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies. Results: The initial search identified 15,457 articles. After removing duplicates, the title and abstracts of 11,543 articles were screened. Seven hundred ten articles were eligible for full-text review. Finally, 83 articles met the criteria and were included in the analyses. Of the 83 includes studies conducted in 14 countries, 48 were conducted in USA, 25 in European countries and 10 in other countries. These 83 studies reported on the use of 39 different PREMs in pediatric healthcare settings. The gray literature retrieved 10 additional PREMs. The number of items in these PREMs ranged from 7 to 89. Twenty-three PREMs were designed to be completed by proxy, 10 by either pediatric patients or family caregivers, and 6 by pediatric patients themselves. Conclusion and Relevance: This comprehensive review is the first to systematically search evidence around the use of PREMs in pediatrics. The findings of this review can guide health administrators and researchers to use appropriate PREMs to implement patient and family-centered care in pediatrics.

Patient experience of virtual consultations in Oral Medicine during the COVID-19 pandemic

Objective

The COVID‐19 pandemic resulted in a rapid shift to the use of virtual consultations in both primary and secondary care. The aim of this study was to assess patient experience of virtual consultations (telephone and video) in the Oral Medicine department during the first wave of the COVID‐19 pandemic.

Methods

A validated survey was developed with the Patient Experience Team in Guy's and St Thomas' NHS Foundation Trust. A combination of previously validated questions and newly validated psychometric questions were used to design the patient feedback questionnaire. The survey was administered to all patients following their virtual (telephone or video) consultation. Data were synthesised and electronically analysed. Qualitative data were thematically analysed.

Results

A total of 115 surveys were completed. Over 82% rated their experience as good or very good and 69% preferred a virtual consultation for their next consultation. Thematic analysis of individual comments identified positive themes including convenience and positive/helpful clinical experience. Areas for development identified from thematic analysis included accessibility and clinical limitations in not undertaking a physical examination.

Conclusion

Overall, the patient experience of virtual consultations in Oral Medicine was positive.