International application of PROMIS computerized adaptive tests: US versus country-specific item parameters can be consequential for individual patient scores

Psychometric properties and post-hoc CAT analysis of the pediatric PROMIS® item banks anxiety and depressive symptoms in a combined Swedish Child and Adolescent Psychiatry and School sample

PURPOSE

The objective of this study is to assess the psychometric properties and reliability of the Swedish Patient-Reported Outcomes Measurement Information System (PROMIS) item banks for anxiety and depressive symptoms with item response theory analysis and post-hoc computerized adaptive testing in a combined Swedish Child and Adolescent Psychiatry (CAP) and school sample.

METHODS

Participants (n = 928, age 12-20) were recruited from junior and high schools and Child and Adolescent Psychiatry Clinics in the region of Västerbotten. Unidimensionality, local independence, and monotonicity was tested. We fitted a graded response model to the data and tested differential item functioning (DIF) for sex, age group, sample type, and language (Swedish vs. U.S.). Moreover, a post-hoc computer adaptive testing (CAT) simulation was performed. All analysis were made in R.

RESULTS

Unidimensionality, local independence, and monotonicity were acceptable. The graded response model yielded acceptable item fit, discriminative, and threshold values for all items in both item banks. DIF for language (Swedish vs. U.S.) was found for two items from the anxiety and one item from the depressive symptoms item banks. A Stocking-lord transformation was used for the items displaying language DIF, and post-hoc CAT simulations were performed. The post-hoc CAT simulation showed reliability around 0.9 for both Swedish and official U.S. item parameters T-scores calibration from within normal limits to severe anxiety and depressive symptoms.

CONCLUSION

The Swedish pediatric PROMIS item banks of anxiety and depressive symptoms are appropriate to assess mild to severe symptoms of anxiety and depressive symptoms in Swedish school- and CAP samples.

Self-Efficacy in Nepali Adults With Musculoskeletal Pain: Measurement Properties of Hard-Copy and Online Versions of the Pain Self-Efficacy Questionnaire

The Pain Self-Efficacy Questionnaire (PSEQ) is commonly used in pain self-efficacy research. Yet its Nepali translation is unavailable, limiting the ability to conduct cross-cultural research on the role of self-efficacy in musculoskeletal pain and its management. This study aimed to 1) translate and culturally adapt the 10-item (PSEQ-10) and 2-item (PSEQ-2) versions of the PSEQ into Nepali, 2) evaluate their measurement properties in Nepali adults with musculoskeletal pain, and 3) evaluate whether the type of administration (ie, hard-copy vs online) affected their measurement properties. The measurement properties of different administrations of the Nepali PSEQ-10 and PSEQ-2 were evaluated in 180 Nepali adults (120 hard-copy and 60 online administrations) with musculoskeletal pain. We conducted confirmatory factor analyses and estimated the measures' internal consistencies, test-retest reliabilities, and smallest detectable changes using standard error of measurement. We planned to conclude that the measures were valid if ≥ 75% of the validity hypotheses were supported. The results supported unidimensionality for the Nepali PSEQ-10. The Nepali PSEQ-2 and PSEQ-10 evidenced excellent internal consistencies (Cronbach alphas = .90-.95) and good to excellent test-retest reliabilities (intraclass correlation coefficient = .61-.85) for both administrations. Construct validity (r's ≥ .20) and concurrent validity (r's ≥ .83) were supported, as hypothesized. Both hard-copy and online administrations of the Nepali PSEQ-2 and PSEQ-10 are similar, reliable, and valid ways to assess self-efficacy in Nepali adults with musculoskeletal pain. The findings should facilitate telehealth and cross-cultural research on pain self-efficacy in Nepal. PERSPECTIVE: This is the first Nepali adaptation of a self-efficacy scale with testing of measurement properties for hard-copy and online administrations. It will facilitate the assessment of pain self-efficacy in clinical practice and research and facilitate a deeper cross-cultural understanding of the role of self-efficacy in musculoskeletal pain.

Monitoring Gait and Physical Activity of Elderly Frail Individuals in Free-Living Environment: A Feasibility Study

INTRODUCTION

Frailty is conventionally diagnosed using clinical tests and self-reported assessments. However, digital health technologies (DHTs), such as wearable accelerometers, can capture physical activity and gait during daily life, enabling more objective assessments. In this study, we assess the feasibility of deploying DHTs in community-dwelling older individuals, and investigate the relationship between digital measurements of physical activity and gait in naturalistic environments and participants' frailty status, as measured by conventional assessments.

METHODS

Fried Frailty Score (FFS) was used to classify fifty healthy individuals as non-frail (FFS = 0, n/female = 21/11, mean ± SD age: 71.10 ± 3.59 years), pre-frail (FFS = 1-2, n/female = 23/9, age: 73.74 ± 5.52 years), or frail (FFS = 3+, n/female = 6/6, age: 70.70 ± 6.53 years). Participants wore wrist-worn and lumbar-worn GENEActiv accelerometers (Activinsights Ltd., Kimbolton, UK) during three in-laboratory visits, and at-home for 2 weeks, to measure physical activity and gait. After this period, they completed a comfort and usability questionnaire. Compliant days at-home were defined as follows: those with ≥18 h of wear time, for the wrist-worn accelerometer, and those with ≥1 detected walking bout, for the lumbar-worn accelerometer. For each at-home measurement, a group analysis was performed using a linear regression model followed by ANOVA, to investigate the effect of frailty on physical activity and gait. Correlation between at-home digital measurements and conventional in-laboratory assessments was also investigated.

RESULTS

Participants were highly compliant in wearing the accelerometers, as 94% indicated willingness to wear the wrist device, and 66% the lumbar device, for at least 1 week. Time spent in sedentary activity and time spent in moderate activity as measured from the wrist device, as well as average gait speed and its 95th percentile from the lumbar device were significantly different between frailty groups. Moderate correlations between digital measurements and self-reported physical activity were found.

CONCLUSIONS

This work highlights the feasibility of deploying DHTs in studies involving older individuals. The potential of digital measurements in distinguishing frailty phenotypes, while unobtrusively collecting unbiased data, thus minimizing participants' travels to sites, will be further assessed in a follow-up study.

Common measures or common metrics? the value of IRT-based common metrics

There is a clear need to harmonize outcome measurement. Some authors propose to express scores as T scores to facilitate interpretation of PROM results in clinical practice. While this is a step in the right direction, there are important limitations to the acceptance of the T score metric as a common metric when T scores are based on raw sum scores of ordinal items: Such T scores of different instruments are not exactly comparable because they are not interval scaled; T scores of different measures are only on the same scale if exactly the same reference group is used; and the T sore metric cannot be maintained because it is reference population-dependent and needs to be updated regularly. These limitations can be overcome by using an item response theory (IRT)-based metric. Items from different measures can be placed on the same IRT metric to make scores comparable on an interval scale. The PROMIS initiative used IRT to develop item banks for measuring various health outcomes. Other PROMs have been linked to the PROMIS metric. Although PROMIS uses a T-score metric for practical reasons, the underlying PROMIS metric is actually an IRT metric. An IRT approach also enables further development of an item bank while preserving the underlying metric. Therefore, IRT-based metrics should be considered as common metrics for the future.

Psychometric properties and reference values of the Patient-Reported Outcomes Measurement Information System (PROMIS) sleep item banks in the Dutch general population

Psychometric properties of the v1.0 Patient-Reported Outcomes Measurement Information System (PROMIS®) sleep disturbance (27 items) and sleep-related impairment (SRI; 16 items) item banks, short forms derived from the item bank, and simulated computerised adaptive test (CAT), were assessed in a representative sample of 1,006 adults from the Dutch general population. For sleep disturbance all items fitted the item response theory model. Four items showed differential item functioning (i.e., lack of measurement invariance) for age and two for language but the impact on scores (expressed as T-scores) was small. Reliable scores (r > 0.90) were found for 92.2%-96.3% of respondents with the full bank, short forms with six and eight items, and CAT, but for only 25.6% with the four-item short form. For SRI two items did not fit the item response theory model. Four items showed differential item functioning for language but the impact on T-scores was small. Reliable scores were found for 82.1% with the full bank, for 47.8%-69.5% with short forms and CAT. T-scores of 49.7 and 49.3 represent the average score of the Dutch general population for sleep disturbance and SRI, respectively. In conclusion, sufficient structural validity, reliability, and cross-cultural validity was found for the full banks but short forms of four items are not reliable enough for clinical practice. For SRI we recommend the full item bank if this is the primary outcome.

Hungarian PROMIS-29+2: psychometric properties and population reference values

OBJECTIVES

This study aims to assess psychometric properties of the Hungarian PROMIS-29+2 profile measure and provide general population reference values for Hungary.

METHODS

An adult general population sample (n = 1700) completed PROMIS-29+2 v2.1 in an online survey. The following psychometric properties were assessed: floor and ceiling effect, convergent validity with SF-36v1 domains, internal consistency (McDonald's omega), unidimensionality, local independence, monotonicity, graded response model (GRM) fit and differential item functioning (DIF). Age- and gender-specific reference values were established using the US item calibrations.

RESULTS

Depending on scale orientation, high floor or ceiling effects were observed for all domains (25.2-60.7%) except for sleep disturbance. McDonald's omega for domains ranged from 0.87-0.97. Unidimensionality, local independence and monotonicity were supported and the GRM adequately fitted for all but one domains. The sleep disturbance domain demonstrated item misfit, response level disordering and low discrimination ability, particularly for item Sleep116 ('refreshing sleep'). Strong correlations were observed between PROMIS-29+2 and corresponding SF-36 domains (r_s=│0.60│ to │0.78│). No DIF was detected for most sociodemographic characteristics. Problems with physical function, pain interference and social roles tended to increase, whereas problems with anxiety, depression, fatigue and cognitive function declined with age (p < 0.01). In all domains except for cognitive function, more health problems occurred in females than in males (p < 0.001).

CONCLUSION

The Hungarian PROMIS-29+2 shows satisfactory psychometric properties; however, the sleep disturbance domain substantially underperforms that requires further attention. Population reference values were generated that facilitate the interpretation of health outcomes in various patient populations.

Common measures or common metrics? A plea to harmonize measurement results

OBJECTIVE

There is a great variety of measurement instruments to assess similar constructs in clinical research and practice. This complicates the interpretation of test results and hampers the implementation of measurement-based care.

METHOD

For reporting and discussing test results with patients, we suggest converting test results into universally applicable common metrics. Two well-established metrics are reviewed: T scores and percentile ranks. Their calculation is explained, their merits and drawbacks are discussed, and recommendations for the most convenient reference group are provided.

RESULTS

We propose to express test results as T scores with the general population as reference group. To elucidate test results to patients, T scores may be supplemented with percentile ranks, based on data from a clinical sample. The practical benefits are demonstrated using the published data of four frequently used instruments for measuring depression: the CES-D, PHQ-9, BDI-II and the PROMIS depression measure.

DISCUSSION

Recent initiatives have proposed to mandate a limited set of outcome measures to harmonize clinical measurement. However, the selected instruments are not without flaws and, potentially, this directive may hamper future instrument development. We recommend using common metrics as an alternative approach to harmonize test results in clinical practice, as this will facilitate the integration of measures in day-to-day practice.

Validity and reliability of Patient-Reported Outcomes Measurement Information System (PROMIS®) using Computerized Adaptive Testing (CAT) in patients with advanced chronic kidney disease

BACKGROUND

The Patient-Reported Outcomes Measurement Information System (PROMIS®) has been recommended for computerized adaptive testing (CAT) of health-related quality of life (HRQOL). This study compared the content, validity and reliability of seven PROMIS CATs to the 12-item Short-Form Health Survey (SF-12) in patients with advanced chronic kidney disease (CKD).

METHODS

Adult CKD patients with an eGFR < 30 ml/min.1.73m2 not receiving dialysis treatment completed seven PROMIS CATs (assessing physical function, pain interference, fatigue, sleep disturbance, anxiety, depression, and ability to participate in social roles and activities), the SF-12 and, additionally, the PROMIS Pain Intensity single item and Dialysis Symptom Index (DSI) at inclusion and 2-weeks. A content comparison was performed between PROMIS CATs and SF-12. Construct validity of PROMIS CATs was assessed using Pearson's correlations. Test-retest reliability of all patient-reported outcome measures (PROMs) was assessed by calculating the intra-class correlation coefficient (ICC) and minimal detectable change (MDC).

RESULTS

In total, 207 patients participated in the study. A median of 45 items (10 minutes) was completed for PROMIS CATs. All PROMIS CATs showed evidence for sufficient construct validity. PROMIS CATs, most SF-12 domains and summary scores, and DSI showed sufficient test-retest reliability (ICC ≥ 0.70). PROMIS CATs had a lower MDC compared to the SF-12 (5.7-7.4 compared to 11.2-21.7 across domains, respectively).

CONCLUSION

PROMIS CATs showed sufficient construct validity and test-retest reliability in patients with advanced CKD. PROMIS CATs required more items but showed better reliability than the SF-12. Future research is needed to investigate the feasibility of PROMIS CATs for routine nephrology care.

Assessing anger and irritability in children: psychometric evaluation and normative data for the German version of the PROMIS® Parent Proxy Anger Scale

PURPOSE

Anger and irritability are common and impairing symptoms in children. The PROMIS Anger scales assess self- and parent-reported irritable and angry mood over the past 7 days. The aim of this study was to evaluate the psychometric properties of the German version of the PROMIS Parent Proxy Short Form v1.0-Anger and to provide normative data.

METHODS

To evaluate the psychometric properties, data from the study ADOPT Epidemiology were used. In this study, the PROMIS Anger Scale was administered to a population-based sample of n = 8746 parents of children aged 8-12 years. Psychometric analyses were carried out including the investigation of distribution characteristics, factor structure, model fit, internal consistency, and congruent validity. Normative data were calculated as percentile ranks and T-scores.

RESULTS

The PROMIS Anger Scale demonstrated good psychometric properties, including satisfactory distribution characteristics, unidimensionality, good internal consistency as well as congruent validity. German normative data for the PROMIS Anger Scale are presented.

CONCLUSION

Based on first psychometric analyses, the German version of the PROMIS Anger Scale can be recommended for use in research and practice; however, further investigations using clinical data are needed. The normative data will allow researchers and clinicians an interpretation of the test scores in future applications.

Generic PROMIS item banks in adults with hemophilia for patient-reported outcome assessment: Feasibility, measurement properties, and relevance

BACKGROUND

Legacy hemophilia-specific questionnaires are considered too long, show floor-/ceiling effects, and/or include irrelevant questions. Patient Reported Outcomes Measurement Information System (PROMIS) item banks, including Computer Adaptive Tests (CATs) and short forms, were designed for more efficient outcome assessment.

OBJECTIVES

Evaluate the feasibility, measurement properties, and relevance of seven PROMIS CATs and two short forms in patients with hemophilia.

PATIENTS/METHODS

In this cross-sectional study, Dutch adults with hemophilia completed nine PROMIS item banks electronically. Feasibility was assessed by number of items and floor/ceiling effects. Reliability was determined as the proportion of reliable scores (standard error ≤3.2). Construct validity was assessed by comparison with legacy instruments and expected differences between subgroups. Relevance of item banks was determined by proportions of limited scores.

RESULTS

Overall, 142 of 373 invited patients (mean age, 47 [range, 18-79], 49% severe hemophilia, 46% receiving prophylaxis) responded. Per CAT item bank, mean number of items answered varied from 5 (range, 3-12) to 9 (range, 5-12), with floor effects in pain interference (26% lowest scores) and depression (18% lowest scores). Construct validity and reliability were good for physical function, pain interference, satisfaction with social roles and activities, and fatigue. The CAT physical function showed the most limited scores (38%). The self-efficacy short forms showed ceiling effects (22%-28%) and no relation with the legacy instruments.

CONCLUSIONS

The PROMIS CATs physical function, pain interference, satisfaction with social roles and activities, and fatigue are feasible, reliable, and valid alternatives to legacy instruments for patients with hemophilia, with few items and low floor-/ceiling effects.