Towards standardized patient reported physical function outcome reporting: linking ten commonly used questionnaires to a common metric

Differential item functioning between English, German, and Spanish PROMIS® physical function ceiling items

PURPOSE

We investigated the validity of the German and Spanish translations of 35 new high functioning items added to the Patient Reported Outcomes Measurement Information System (PROMIS®) Physical Function item bank 2.0. We assessed differential item functioning (DIF) between three general population samples from Argentina, Germany, and the United States.

METHODS

PROMIS Physical Function data was collected in online panels from 3601 individuals (mean age, 41.6 years old; range, 18-88 years; 53.7% female). Of these, 1001 participants completed the Spanish version, 1000 completed the German version, and 1600 completed the English version. DIF was assessed by a multiverse analysis that systematically varied analytic choices across the entire range of plausible options within the logistic ordinal regression framework.

RESULTS

Translated items generally met the assumptions of unidimensionality, monotonicity, and local independence. The 272 different analyses suggest consistent DIF between languages in four items. Test characteristic curves suggested that the magnitude and impact of DIF on the test scores were negligible for all items at the test level. After correcting for potential DIF, we observed greater scoring for physical functioning in Argentina compared to the US, Cohen's d = 0.25, [0.17, 0.33], and Argentina compared to Germany, Cohen's d = 0.23, [0.15, 0.32].

CONCLUSIONS

Our findings support the universal applicability of PROMIS Physical Function items across general populations in Argentina, Germany, and the U.S. The sensitivity analyses indicate that the identification of DIF items was robust for different data analytic decisions. Multiverse analysis is a promising approach to address lack of clear cutoffs in DIF identification.

The Case for Aggregated Rehabilitation-Relevant Data Across Health Care Systems and Settings

Health care value, quantified as outcome per unit cost, requires knowing which outcomes are influenced by which intervention at what cost. The value of rehabilitation is still largely unknown. Much of the reason for this limited evidence is historically poor standardization and collection of rehabilitation interventions, and objectively measured outcomes across care settings, care providers, and health care systems. The purposeful standardization and aggregation of rehabilitation-relevant data about interventions, cost, and outcomes from routine clinical practices offers potential to understand and improve the value of rehabilitation. This perspective details the critical need for rehabilitation-relevant data that are aggregated across settings, providers, and systems and proposes 3 options to meet this need, including (1) integrating rehabilitation-relevant data into existing research registry databases that are condition specific, (2) adding rehabilitation-relevant data to federally funded research networks, and (3) creating a novel rehabilitation registry database. There must be continued pursuit of discovering which rehabilitation interventions achieve which specific outcomes, in which settings, for which patients, and at what costs. Successfully aggregating rehabilitation-relevant data is critical for generating evidence that answers these key questions about the value of rehabilitation.

Developing a common metric using current scales for assessing functioning in patients with knee osteoarthritis

Objectives

Various scales exist to assess different domains of functioning in knee osteoarthritis (OA). This study aimed to explore whether it is possible to develop a common metric (CM) from the frequently used scales to assess functioning in knee OA.

Patients and methods

The methodological study evaluated 411 patients (81 males, 330 females; mean age: 61.8±10.5 years; range, 41 to 88 years) with knee OA. Data from the Health Assessment Questionnaire, Oxford Knee Score, Medical Outcomes Study Short Form 36, Knee Injury and Osteoarthritis Outcome Score, Western Ontario and McMaster Universities Arthritis Index, and the Nottingham Health Profile were used, and the items focusing on self-care, mobility, and domestic activity domains based on the activities and participation component of the International Classification of Functioning, Disability, and Health were included. Concurrent calibration was performed to combine the items of the scales. The CM parameters were estimated using the Rasch measurement model. Reliability was assessed using the person separation index. The CM was utilized to generate a transformation table to convert the scale scores to each other based on the reference metric score.

Results

Each scale fitted the Rasch model. Item invariance was achieved for the CM (p=0.775). The CM had a person separation index of 0.827. Age, sex, and disease duration did not cause difference in item functions. The CM satisfied the assumptions of unidimensionality and local independence.

Conclusion

A reliable CM was created from the commonly used scales to measure functioning in individuals with knee OA. Thus, clinicians and researchers can refer to the transformation table to directly compare scores of those scales and use them interchangeably.

Physical performance tasks were linked to the PROMIS physical function metric in patients undergoing hemodialysis

OBJECTIVES

To investigate whether a multi-item performance outcome measure, the physical performance test (PPT), can be calibrated to a common scale with patient-reported outcome measures, using the Patient-Reported Outcomes Measurement Information System (PROMIS) physical function (PF) metric.

STUDY DESIGN AND SETTING

We analyzed baseline data (N = 1,113) from the CONVINCE study, an international trial in end-stage kidney disease patients comparing high-dose hemodiafiltration with high-flux hemodialysis. Assumptions of item response theory (IRT) modelling were investigated for the combined set of the nine-item PPT and a four-item PROMIS PF short form (PROMIS-PF4a). We applied unidimensional IRT linking for calibrating the PPT to the PROMIS PF metric.

RESULTS

Although some evidence for multidimensionality was found, classical test statistics (Cronbach's Alpha = 0.93), Mokken (Loevinger's H = 0.50), and bifactor analysis (explained common variance = 0.65) indicated that PPT and PROMIS-PF4a items can be used to assess a common PF construct. On the group level, the agreement between PROMIS-PF4a and linked PPT scores was stable across several subsamples. On the individual level, scores differed considerably.

CONCLUSION

We found preliminary evidence that the PPT can be linked to the PROMIS PF metric in hemodialysis patients, enabling group comparisons across patient-reported outcome and performance outcome measures. Alternative linking methods should be applied in future studies using a more comprehensive PROMIS PF item set.

Psychometric properties of the Spanish version of the Activities Scale for Kids (ASK): reliability, validity and the Rasch model

AIMS

To evaluate the psychometric properties of the Spanish version of the Activities Scale for Kids capability (ASKc) and ASK performance (ASKp) questionnaires.

DESIGN

It includes an analysis of different types of reliability (internal consistency, test-retest, inter-rater, Rasch model) and validity (convergent and discriminant) values.

SETTINGS

The sample was recruited in schools, associations and one hospital in Spain.

PARTICIPANTS

The main sample comprised 448 children (114 with disabilities); and 96 parents of the group of 114 children with disabilities, along with 2 therapists.

METHODS AND PROCEDURE

Children with physical disabilities completed questionnaires at two different time points; while healthy children only once. Both ASK questionnaires were also administered to parents. Therapists observed 69 children in 15 of the 30 activities listed in the ASKc.

PRIMARY AND SECONDARY OUTCOME MEASURES

ASKc and ASKp were used to measure physical disability among children. The Childhood Health Assessment Questionnaire (CHAQ), The Screening For and Promotion of Health-Related Quality of Life in Childrenand Adolescents - a European Public Health perspective (KIDSCREEN), Gross Motor Function Classification System (GMFCS), Manual Ability Classification System (MACS) and Functional Mobility Scale (FMS) were used to analyse convergent validity.

RESULTS

Excellent values were obtained for the reliability of the scale. Internal consistency was >0.95 (Cronbach's α) for both questionnaires. The intraclass correlation coefficient test-retest reliability was 0.94 (ASKc) and 0.93 (ASKp). Correlations between parents' and children's scores were 0.91(ASKc) and 0.90 (ASKp); and the correlation between therapists' and children's scores was 0.78. The Rasch analysis indicated that the Spanish version had a unidimensional structure. Furthermore, the results revealed adequate validity indices. Both ASK questionnaires correlated significantly with the CHAQ, three dimensions of the KIDSCREEN and the GMFCS, MACS and FMS. Finally, children without disabilities had higher ASKc and ASKp scores than children with disabilities (p=0.0001).

CONCLUSIONS

Both the ASKc and the ASKp versions are reliable and valid instruments that can be used to measure the capabilities of Spanish-speaking children, whose responses also demonstrate their own reliability as informants of the impact of disability on the activities of daily living.

Common measures or common metrics? A plea to harmonize measurement results

OBJECTIVE

There is a great variety of measurement instruments to assess similar constructs in clinical research and practice. This complicates the interpretation of test results and hampers the implementation of measurement-based care.

METHOD

For reporting and discussing test results with patients, we suggest converting test results into universally applicable common metrics. Two well-established metrics are reviewed: T scores and percentile ranks. Their calculation is explained, their merits and drawbacks are discussed, and recommendations for the most convenient reference group are provided.

RESULTS

We propose to express test results as T scores with the general population as reference group. To elucidate test results to patients, T scores may be supplemented with percentile ranks, based on data from a clinical sample. The practical benefits are demonstrated using the published data of four frequently used instruments for measuring depression: the CES-D, PHQ-9, BDI-II and the PROMIS depression measure.

DISCUSSION

Recent initiatives have proposed to mandate a limited set of outcome measures to harmonize clinical measurement. However, the selected instruments are not without flaws and, potentially, this directive may hamper future instrument development. We recommend using common metrics as an alternative approach to harmonize test results in clinical practice, as this will facilitate the integration of measures in day-to-day practice.

The patient-reported outcomes measurement information systems (PROMIS®) physical function and its derivative measures in adults: a systematic review of content validity

PURPOSE

This study aims to systematically review and critically appraise the content validity of the adult versions of the Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS-PF) item bank and its derivative measures in any adult population.

METHODS

MEDLINE and EMBASE were searched in October 2021 for studies on measurement properties of PROMIS-PF measures in an adult population. Studies were included if the study described the development of a PROMIS-PF measure or investigated its relevance, comprehensiveness, or comprehensibility. Assessment of the methodological quality of eligible studies, rating of results, and summarizing evidence was performed following the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology for assessing content validity. A modified GRADE approach was used to determine the level of evidence.

RESULTS

Three development studies and eight studies on the content validity of one or more of the PROMIS-PF measures were identified. The methodological quality of most studies was rated doubtful. There was low to high level evidence for sufficient relevance, comprehensiveness, and comprehensibility of most PROMIS-PF measures for healthy seniors and various disease populations. We found low to moderate level evidence for insufficient relevance of PROMIS-PF measures for patients with conditions that affected only one body part, and insufficient comprehensibility of the PROMIS-PF measures for minority elderly.

CONCLUSION

Most PROMIS-PF measures demonstrate sufficient content validity in healthy seniors and various disease populations. However, the quality of this evidence is generally low to moderate, due to limitations in the methodological quality of the studies.

Health-Related Quality of Life Measurement in Public Health

Patient-reported outcomes are recognized as essential for the evaluation of medical and public health interventions. Over the last 50 years, health-related quality of life (HRQoL) research has grown exponentially from 0 to more than 17,000 papers published annually. We provide an overview of generic HRQoL measures used widely in epidemiological studies, health services research, population studies, and randomized clinical trials [e.g., Medical Outcomes Study SF-36 and the Patient-Reported Outcomes Measurement Information System (PROMIS^®)-29]. In addition, we review methods used for economic analysis and calculation of the quality-adjusted life year (QALY). These include the EQ-5D, the Health Utilities Index (HUI), the self-administered Quality of Well-being Scale (QWB-SA), and the Health and Activities Limitation Index (HALex). Furthermore, we consider hybrid measures such as the SF-6D and the PROMIS-Preference (PROPr). The plethora of HRQoL measures has impeded cumulative science because incomparable measures have been used in different studies. Linking among different measures and consensus on standard HRQoL measurement should now be prioritized. In addition, enabling widespread access to common measures is necessary to accelerate future progress. Expected final online publication date for the Annual Review of Public Health, Volume 43 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS

The psychometric process used to establish a relationship between the scores of two (or more) instruments is generically referred to as linking. When two instruments with the same content and statistical test specifications are linked, these instruments are said to be equated. Linking and equating procedures have long been used for practical benefit in educational testing. In recent years, health outcome researchers have increasingly applied linking techniques to patient-reported outcome (PRO) data. However, these applications have some noteworthy purposes and associated methodological questions. Purposes for linking health outcomes include the harmonization of data across studies or settings (enabling increased power in hypothesis testing), the aggregation of summed score data by means of score crosswalk tables, and score conversion in clinical settings where new instruments are introduced, but an interpretable connection to historical data is needed. When two PRO instruments are linked, assumptions for equating are typically not met and the extent to which those assumptions are violated becomes a decision point around how (and whether) to proceed with linking. We demonstrate multiple linking procedures-equipercentile, unidimensional IRT calibration, and calibrated projection-with the Patient-Reported Outcomes Measurement Information System Depression bank and the Patient Health Questionnaire-9. We validate this link across two samples and simulate different instrument correlation levels to provide guidance around which linking method is preferred. Finally, we discuss some remaining issues and directions for psychometric research in linking PRO instruments.

Common Functional Ability Score for Young People With Juvenile Idiopathic Arthritis

OBJECTIVE

As young people enter adulthood, the interchangeable use of child and adult outcome measures may inaccurately capture changes over time. This study aimed to use item response theory (IRT) to model a continuous score for functional ability that can be used no matter which questionnaire is completed.

METHODS

Adolescents (ages 11-17 years) in the UK Childhood Arthritis Prospective Study (CAPS) self-completed an adolescent Childhood Health Assessment Questionnaire (CHAQ) and a Health Assessment Questionnaire (HAQ). Their parents answered the proxy-completed CHAQ. Those children with at least 2 simultaneously completed questionnaires at initial presentation or 1 year were included. Psychometric properties of item responses within each questionnaire were tested using Mokken analyses to assess the applicability of IRT modeling. A previously developed IRT model from the Pharmachild-NL registry from The Netherlands was validated in CAPS participants. Agreement and correlations between IRT-scaled functional ability scores were tested using intraclass correlations and Wilcoxon's signed rank tests.

RESULTS

In 303 adolescents, the median age at diagnosis was 13 years, and 61% were female. CHAQ scores consistently exceeded HAQ scores. Mokken analyses demonstrated high scalability, monotonicity, and the fact that each questionnaire yielded reliable scores. There was little difference in item response characteristics between adolescents enrolled in CAPS and Pharmachild-NL (maximum item residual 0.08). Significant differences were no longer evident between IRT-scaled HAQ and CHAQ scores.

CONCLUSION

IRT modeling allows the direct comparison of function scores regardless of different questionnaires being completed by different people over time. IRT modeling facilitates the ongoing assessment of function as adolescents transfer from pediatric clinics to adult services.

Scale-Banking for Patient Reported Outcome Measures (PROMs) Measuring Functioning in Rheumatoid Arthritis: A Daily Activities Metric

OBJECTIVE

Functioning is an important outcome for rheumatoid arthritis (RA) management. Heterogeneity of respective patient-reported outcome measures (PROMs) challenges direct comparisons between their results. This study aimed to standardize reporting of such PROMs measuring functioning in RA to facilitate comparability.

METHODS

Common Item Non-Equivalent Groups Design (NEAT) with the Health Assessment Questionnaire (HAQ) as a common scale across data sets from various countries (incl. UK, Turkey and Germany) to establish a common metric. Other PROMs included are the Physical Function items of the Multidimensional Health Assessment Questionnaire (MDHAQ), Disabilities of Arm, Shoulder and Hand (DASH), Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), World Health Organization Disability Assessment Schedule Version 2.0 (WHODAS 2.0), and four short forms (20, 10, 6, and 4 physical function items) from the Patient-Reported Outcomes Measurement Information System (PROMIS). As the HAQ includes mobility, self-care and domestic life items, this study focuses on these three domains. PROMs were described using Standard Error of Measurement (SEM) and Smallest Detectable Difference (SDD). Rasch Measurement model was used to create the common metric.

RESULTS

Range of SEM is 0.2 (MDHAQ) to 7.4 (SF36-PF). SDD revealed a range from 9.7 % (WOMAC-RAT) to 33.5 % (WHODAS-PF). PROMs co-calibration revealed fit to the Rasch measurement model. A transformation table was developed to allow exchange between PROMs scores.

DISCUSSION

Scores between the Daily Activity PROMs commonly used in RA can now be compared. Factors such as SEM and SDD help determine choice of PROM in clinical practice and research.

Using subdomain-specific item sets affected PROMIS physical function scores differently in cardiology and rheumatology patients

OBJECTIVES

The Patient-Reported Outcomes Measurement Information System (PROMIS) physical function (PF) item bank has been developed to standardize patient-reported PF across medical fields. However, evidence of scoring equivalence across cardiology and rheumatology patients is still missing. Therefore, this study aims to investigate both (1) the extent of disease-related differential item functioning (DIF) and (2) the impact of the disease group on using subdomain-specific item sets for generating PROMIS PF scores in cardiology and rheumatology patients.

STUDY DESIGN AND SETTING

Ordinal regression was used to evaluate DIF between cardiology (n = 201) and rheumatology (n = 200) inpatients. To explore the disease-specific impact of PF subdomains on scoring, we compared scores derived from the full item bank with scores derived from subdomain-specific item sets for each disease group.

RESULTS

DIF was detected in 18 items, predominately from the upper extremity subdomain. When upper extremity items were used, cardiology patients reached systematically higher scores than using the full item bank. Rheumatology patients scored substantially higher when mobility items were used.

CONCLUSION

Applying the PROMIS PF metric to disease-specific item sets including items from differing subdomains may lead to biased comparisons of PF levels across disease groups. Disease-specific item parameters should be provided for items showing DIF, and subdomain-related content balancing is recommended for scoring the generic PROMIS PF construct.

International Consortium for Health Outcome Measurement Set of Outcomes That Matter to People Living With Inflammatory Arthritis: Consensus From an International Working Group

Objective

The implementation of value‐based health care in inflammatory arthritis requires a standardized set of modifiable outcomes and risk‐adjustment variables that is feasible to implement worldwide.

Methods

The International Consortium for Health Outcomes Measurement (ICHOM) assembled a multidisciplinary working group that consisted of 24 experts from 6 continents, including 6 patient representatives, to develop a standard set of outcomes for inflammatory arthritis. The process followed a structured approach, using a modified Delphi process to reach consensus on the following decision areas: conditions covered by the set, outcome domains, outcome measures, and risk‐adjustment variables. Consensus in areas 2 to 4 were supported by systematic literature reviews and consultation of experts.

Results

The ICHOM Inflammatory Arthritis Standard Set covers patients with rheumatoid arthritis (RA), axial spondyloarthritis, psoriatic arthritis, and juvenile idiopathic arthritis (JIA). We recommend that outcomes regarding pain, fatigue, activity limitations, overall physical and mental health impact, work/school/housework ability and productivity, disease activity, and serious adverse events be collected at least annually. Validated measures for patient‐reported outcomes were endorsed and linked to common reporting metrics. Age, sex at birth, education level, smoking status, comorbidities, time since diagnosis, and rheumatoid factor and anti‐citrullinated protein antibody lab testing for RA and JIA should be collected as risk‐adjustment variables.

Conclusion

We present the ICHOM inflammatory arthritis Standard Set of outcomes, which enables health care providers to implement the value‐based health care framework and compare outcomes that are important to patients with inflammatory arthritis.

Taking the patient and the patient's perspective into account to improve outcomes of care of patients with musculoskeletal diseases

Patient-reported outcome measures are commonly used in the assessment of patients with musculoskeletal diseases. The present review provides an overview of historic and recent developments, including core set recommendations for assessing patient-reported outcomes in patients with fibromyalgia, osteoarthritis, rheumatoid arthritis, ankylosing spondylitis, and psoriatic arthritis. The evidence supporting commonly used patient-reported outcomes measures is reviewed. Furthermore, various methodological approaches that can be utilized to evaluate validity and measurement precision of patient reported outcomes are introduced. Commonly used methods based on the classical test theory as well as modern approaches based on item response theory will be discussed. The review finally describes the increasing use of item response theory-based approaches used in patient-reported outcomes assessment in the musculoskeletal diseases.