Scoring based on item response theory did not alter the measurement ability of EORTC QLQ-C30 scales

Item response theory assumptions were adequately met by the Oxford hip and knee scores

OBJECTIVES

To develop item response theory (IRT) models for the Oxford hip and knee scores which convert patient responses into continuous scores with quantifiable precision and provide these as web applications for efficient score conversion.

STUDY DESIGN AND SETTING

Data from the National Health Service patient-reported outcome measures program were used to test the assumptions of IRT (unidimensionality, monotonicity, local independence, and measurement invariance) before fitting models to preoperative response patterns obtained from patients undergoing primary elective hip or knee arthroplasty. The hip and knee datasets contained 321,147 and 355,249 patients, respectively.

RESULTS

Scree plots, Kaiser criterion analyses, and confirmatory factor analyses confirmed unidimensionality and Mokken analysis confirmed monotonicity of both scales. In each scale, all item pairs shared a residual correlation of ≤ 0.20. At the test level, both scales showed measurement invariance by age and gender. Both scales provide precise measurement in preoperative settings but demonstrate poorer precision and ceiling effects in postoperative settings.

CONCLUSION

We provide IRT parameters and web applications that can convert Oxford Hip Score or Oxford Knee Score response sets into continuous measurements and quantify individual measurement error. These can be used in sensitivity analyses or to administer truncated and individualized computerized adaptive tests.

An item response theory analysis of an item pool for the recovering quality of life (ReQoL) measure

Purpose

ReQoL-10 and ReQoL-20 have been developed for use as outcome measures with individuals aged 16 and over, experiencing mental health difficulties. This paper reports modelling results from the item response theory (IRT) analyses that were used for item reduction.

Methods

From several stages of preparatory work including focus groups and a previous psychometric survey, a pool of items was developed. After confirming that the ReQoL item pool was sufficiently unidimensional for scoring, IRT model parameters were estimated using Samejima’s Graded Response Model (GRM). All 39 mental health items were evaluated with respect to item fit and differential item function regarding age, gender, ethnicity, and diagnosis. Scales were evaluated regarding overall measurement precision and known-groups validity (by care setting type and self-rating of overall mental health).

Results

The study recruited 4266 participants with a wide range of mental health diagnoses from multiple settings. The IRT parameters demonstrated excellent coverage of the latent construct with the centres of item information functions ranging from − 0.98 to 0.21 and with discrimination slope parameters from 1.4 to 3.6. We identified only two poorly fitting items and no evidence of differential item functioning of concern. Scales showed excellent measurement precision and known-groups validity.

Conclusion

The results from the IRT analyses confirm the robust structure properties and internal construct validity of the ReQoL instruments. The strong psychometric evidence generated guided item selection for the final versions of the ReQoL measures.

Electronic supplementary material

The online version of this article (10.1007/s11136-020-02622-2) contains supplementary material, which is available to authorized users.

Development of an item bank for the EORTC Role Functioning Computer Adaptive Test (EORTC RF-CAT)

BACKGROUND

Role functioning (RF) as a core construct of health-related quality of life (HRQOL) comprises aspects of occupational and social roles relevant for patients in all treatment phases as well as for survivors. The objective of the current study was to improve its assessment by developing a computer-adaptive test (CAT) for RF. This was part of a larger project whose objective is to develop a CAT version of the EORTC QLQ-C30 which is one of the most widely used HRQOL instruments in oncology.

METHODS

In accordance with EORTC guidelines, the development of the RF-CAT comprised four phases. Phase I involved the conceptualization of RF. In Phase II, a provisional list of items was defined and revised by experts in the field. In phase III, feedback was obtained from cancer patients in various countries. Phase IV comprised field testing in an international sample, calibration of the item bank, and evaluation of the psychometric performance of the RF-CAT.

RESULTS

Phases I-III yielded a list of 12 items eligible for phase IV field-testing. The field-testing sample included 1,023 patients from Austria, Denmark, Italy, and the UK. Psychometric evaluation and item response theory analyses yielded 10 items with good psychometric properties. The resulting item bank exhibits excellent reliability (mean reliability = 0.85, median = 0.95). Using the RF-CAT may allow sample size savings from 11 % up to 50 % compared to using the QLQ-C30 RF scale.

CONCLUSIONS

The RF-CAT item bank improves the precision and efficiency with which RF can be assessed, promoting its integration into oncology research and clinical practice.

Psychometric evaluation of the Care Transition Measure in TRACE-CORE: do we need a better measure?

Background

The quality of transitional care is associated with important health outcomes such as rehospitalization and costs. The widely used Care Transitions Measure (CTM‐15) was developed with a classic test theory approach; its short version (CTM‐3) was included in the CAHPS Hospital Survey. We conducted a psychometric evaluation of both measures and explored whether item response theory (IRT) could produce a more precise measure.

Methods and Results

As part of the Transitions, Risks, and Actions in Coronary Events Center for Outcomes Research and Education, 1545 participants were interviewed during an acute coronary syndrome hospitalization, providing information on general health status (Short Form‐36), CTM‐15, health utilization, and care process questions at 1 month postdischarge. We used classic and IRT analyses and compared the measurement precision of CTM‐15–, CTM‐3–, and CTM‐IRT–based score using relative validity.

Participants were 79% non‐Hispanic white and 67% male, with an average age of 62 years. The CTM‐15 had good internal consistency (Cronbach's α=0.95) but demonstrated acquiescence bias (8.7% participants responded “Strongly agree” and 19% responded “Agree” to all items) and limited score variability. These problems were more pronounced for the CTM‐3. The CTM‐15 differentiated between patient groups defined by self‐reported health status, health care utilization, and care transition process indicators. Differences between groups were small (2 to 3 points). There was no gain in measurement precision from IRT scoring. The CTM‐3 was not significantly lower for patients reporting rehospitalization or emergency department visits.

Conclusion

We identified psychometric challenges of the CTM, which may limit its value in research and practice. These results are in line with emerging evidence of gaps in the validity of the measure.

The chemotherapy-induced peripheral neuropathy outcome measures standardization study: from consensus to the first validity and reliability findings

BACKGROUND

Chemotherapy-induced peripheral neuropathy (CIPN) is a debilitating and dose-limiting complication of cancer treatment. Thus far, the impact of CIPN has not been studied in a systematic clinimetric manner. The objective of the study was to select outcome measures for CIPN evaluation and to establish their validity and reproducibility in a cross-sectional multicenter study.

PATIENTS AND METHODS

After literature review and a consensus meeting among experts, face/content validity were obtained for the following selected scales: the National Cancer Institute-Common Toxicity Criteria (NCI-CTC), the Total Neuropathy Score clinical version (TNSc), the modified Inflammatory Neuropathy Cause and Treatment (INCAT) group sensory sumscore (mISS), the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30, and CIPN20 quality-of-life measures. A total of 281 patients with stable CIPN were examined. Validity (correlation) and reliability studies were carried out.

RESULTS

Good inter-/intra-observer scores were obtained for the TNSc, mISS, and NCI-CTC sensory/motor subscales. Test-retest values were also good for the EORTC QLQ-C30 and CIPN20. Acceptable validity scores were obtained through the correlation among the measures.

CONCLUSION

Good validity and reliability scores were demonstrated for the set of selected impairment and quality-of-life outcome measures in CIPN. Future studies are planned to investigate the responsiveness aspects of these measures.

[Modern testing theory and its application in the field of health measurement]

This paper briefly introduces item response theory (IRT) as a typical representation of modern testing theory (MTT), and systematically reviews the processes and contents of the application of IRT in the area of health measurement, including, for example, item bank development, scale revision and computerized adaptive testing. The author presents the potential benefits and the notable problems during health measuring by IRT. Then, the author asserts the need for thorough assessment of feasibility when using the IRT in patient-reported outcome research. Further research based on IRT and computerized adaptive testing in health measurement will be carried out in the field of medical care including traditional Chinese medicine and integrative medicine.

Quality criteria and user-friendliness in self-reported questionnaires on cancer-related fatigue: a review

OBJECTIVE

Cancer-related fatigue (CRF) is a distressing, persistent, subjective sense of tiredness or exhaustion that occurs in 70-100% of cancer patients. The purpose of this review was to provide an overview of the quality of research performed on existing CRF self-report questionnaires and compare their reported psychometric properties and user-friendliness.

METHODS

Database searches of CINAHL, Cochrane Library, EMBASE, MEDLINE, Scopus, PEDro, and PsycINFO were undertaken to find published scales. Standardized criteria were used to assess quality and user-friendliness.

RESULTS

Thirty-five articles were included that described 18 questionnaires-seven one-dimensional questionnaires and 11 multidimensional questionnaires. The mean item count was 20.8 (range: 3-83). The mean overall score of the one-dimensional questionnaires was 10.4 of a maximum of 18 points (range: 7.6-14.3). The mean overall score of the multidimensional questionnaires was 9.4 of a maximum of 18 points (range: 4.3-14.4).

CONCLUSION

Recommendations were made for the selection of a scale. We argue in favor of repeatedly reassessing psychometric properties of even established questionnaires to ensure they comply with evermore increasing stringent quality criteria.

Comparing translations of the EORTC QLQ-C30 using differential item functioning analyses

The European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30 is one of the most widely used quality of life instruments for cancer patients. The aim of this study was to assess whether there were linguistic differences in the way an international sample answered the EORTC QLQ-C30 questionnaire. Thirteen translations of the EORTC QLQ-C30, representing 22 countries, were investigated using a database of 27,891 respondents, incorporating 103 separate studies. Differential item functioning (DIF) analyses were conducted using logistic regression to identify items which, after controlling for subscale, were answered differently by language of administration. Both uniform and non-uniform DIF were assessed. Although most languages showed similar results to English, at least one instance of statistically significant DIF was identified for each translation, and a few of these differences were large. In some cases, the patterns were supported by the results of qualitative interviews with bilingual people. Although, overall, there appeared to be good linguistic equivalence for most of the EORTC QLQ-C30 items, several scales showed strongly discrepant results for some translations. Some of these effects are large enough to impact on the results of clinical trials. Based on our experience in this study, we suggest that validation of translations of health-related quality of life instruments should include exploration of DIF.

The development of the EORTC QLQ-C15-PAL: a shortened questionnaire for cancer patients in palliative care

This study aimed at developing a shortened version of the EORTC QLQ-C30, one of the most widely used health-related quality of life questionnaires in oncology, for palliative care research. The study included interviews with 41 patients and 66 health care professionals in palliative care to determine the appropriateness, relevance and importance of the various domains of the QLQ-C30. Item response theory methods were used to shorten scales. Patients and health care professionals rated pain, physical function, emotional function, fatigue, global health status/quality of life, nausea/vomiting, appetite, dyspnoea, constipation, and sleep as most important. Therefore, these scales/items were retained in the questionnaire. Four scales were shortened without reducing measurement precision. Important dimensions not covered by the questionnaire were identified. The resulting 15-item EORTC QLQ-C15-PAL is a 'core questionnaire' for palliative care. Depending on the research questions, it may be supplemented by additional items, modules or questionnaires.