Scaling the sickness impact profile using item response theory: an exploration of linearity, adaptive use, and patient driven item weights

Examination of the measurement equivalence of the Functional Assessment in Acute Care MCAT (FAMCAT) mobility item bank using differential item functioning analyses

OBJECTIVE

To assess differential item functioning (DIF) in an item pool measuring the mobility of hospitalized patients across educational, age, and gender groups.

DESIGN

Measurement evaluation cohort study. Content experts generated DIF hypotheses to guide the interpretation. The graded response item response theory (IRT) model was used. Primary DIF tests were Wald statistics; sensitivity analyses were conducted using the IRT ordinal logistic regression procedure. Magnitude and impact were evaluated by examining group differences in expected item and scale score functions.

SETTING

Hospital-based rehabilitation PARTICIPANTS: 2216 hospitalized patients MAIN OUTCOME MEASURES: 111 self-reported mobility items RESULTS: Two linking items among those used to set the metric across forms evidenced DIF for gender and age: 'difficulty climbing stairs step-over-step without a handrail (alternating feet)' and 'difficulty climbing 3 to 5 steps without a handrail'. Conditional on the mobility state, the items were more difficult for women and older people (aged 65 and over). An additional 18 items were identified with DIF. Items with both high DIF magnitude and hypotheses related to age were difficulty: 'crossing road at a 4-lane traffic light with curbs'; 'jumping/landing on one leg'; 'strenuous activities'; 'descending 3-5 steps with no handrail'. Although DIF of higher magnitude was observed for several items, the scale-level impact was relatively small and the exposure rate for the most problematic items was low (0.35, 0.27 and 0.20).

CONCLUSIONS

This was the first study to evaluate measurement equivalence of the hospital-based rehabilitation mobility item bank. Although 20 items evidenced high magnitude DIF, five related to stairs, the scale-level impact was minimal; however, it is recommended that such items be avoided in the development of short-form measures. No items with salient DIF were removed from calibrations, supporting the use of the item bank across groups differing in education, age, and gender. The bank may thus be useful to assist clinical assessment and decision-making regarding risk for specific mobility restrictions at discharge as well as identifying mobility-related functions targeted for post- discharge interventions. Additionally, with the goal of avoiding long and burdensome assessments for patients and clinical staff; these results could be informative for those using the item bank to construct short forms.

Measurement invariance across chronic conditions: a systematic review and an empirical investigation of the Health Education Impact Questionnaire (heiQ™)

BACKGROUND

To examine whether lack of measurement invariance (MI) influences mean comparisons among different disease groups, this paper provides (1) a systematic review of MI in generic constructs across chronic conditions and (2) an empirical analysis of MI in the Health Education Impact Questionnaire (heiQ™).

METHODS

(1) We searched for studies of MI among different chronic conditions in online databases. (2) Multigroup confirmatory factor analyses were used to study MI among five chronic conditions (orthopedic condition, rheumatism, asthma, COPD, cancer) in the heiQ™ with N = 1404 rehabilitation inpatients. Impact on latent and composite mean differences was examined.

RESULTS

(1) A total of 30 relevant studies suggested that about one in three items lacked MI. However, only four studies examined impact on latent mean differences. Scale means were only affected in one of these three studies. (2) Across the eight heiQ™ scales, seven scales had items with lack of MI in at least one disease group. However, in only two heiQ™ scales were some latent or composite mean differences affected.

CONCLUSIONS

Lack of MI among disease groups is common and may have a relevant influence on mean comparisons when using generic instruments. Therefore, when comparing disease groups, tests of MI should be implemented. More studies of MI and according impact on mean differences in generic questionnaires are needed.

Measurement invariance across chronic conditions: a systematic review and an empirical investigation of the Health Education Impact Questionnaire (heiQ™)

BACKGROUND

To examine whether lack of measurement invariance (MI) influences mean comparisons among different disease groups, this paper provides (1) a systematic review of MI in generic constructs across chronic conditions and (2) an empirical analysis of MI in the Health Education Impact Questionnaire (heiQ™).

METHODS

(1) We searched for studies of MI among different chronic conditions in online databases. (2) Multigroup confirmatory factor analyses were used to study MI among five chronic conditions (orthopedic condition, rheumatism, asthma, COPD, cancer) in the heiQ™ with N = 1404 rehabilitation inpatients. Impact on latent and composite mean differences was examined.

RESULTS

(1) A total of 30 relevant studies suggested that about one in three items lacked MI. However, only four studies examined impact on latent mean differences. Scale means were only affected in one of these three studies. (2) Across the eight heiQ™ scales, seven scales had items with lack of MI in at least one disease group. However, in only two heiQ™ scales were some latent or composite mean differences affected.

CONCLUSIONS

Lack of MI among disease groups is common and may have a relevant influence on mean comparisons when using generic instruments. Therefore, when comparing disease groups, tests of MI should be implemented. More studies of MI and according impact on mean differences in generic questionnaires are needed.

Measuring the ICF components of impairment, activity limitation and participation restriction: an item analysis using classical test theory and item response theory

BACKGROUND

The International Classification of Functioning, Disability and Health (ICF) proposes three main health outcomes, Impairment (I), Activity Limitation (A) and Participation Restriction (P), but good measures of these constructs are needed The aim of this study was to use both Classical Test Theory (CTT) and Item Response Theory (IRT) methods to carry out an item analysis to improve measurement of these three components in patients having joint replacement surgery mainly for osteoarthritis (OA).

METHODS

A geographical cohort of patients about to undergo lower limb joint replacement was invited to participate. Five hundred and twenty four patients completed ICF items that had been previously identified as measuring only a single ICF construct in patients with osteoarthritis. There were 13 I, 26 A and 20 P items. The SF-36 was used to explore the construct validity of the resultant I, A and P measures. The CTT and IRT analyses were run separately to identify items for inclusion or exclusion in the measurement of each construct. The results from both analyses were compared and contrasted.

RESULTS

Overall, the item analysis resulted in the removal of 4 I items, 9 A items and 11 P items. CTT and IRT identified the same 14 items for removal, with CTT additionally excluding 3 items, and IRT a further 7 items. In a preliminary exploration of reliability and validity, the new measures appeared acceptable.

CONCLUSION

New measures were developed that reflect the ICF components of Impairment, Activity Limitation and Participation Restriction for patients with advanced arthritis. The resulting Aberdeen IAP measures (Ab-IAP) comprising I (Ab-I, 9 items), A (Ab-A, 17 items), and P (Ab-P, 9 items) met the criteria of conventional psychometric (CTT) analyses and the additional criteria (information and discrimination) of IRT. The use of both methods was more informative than the use of only one of these methods. Thus combining CTT and IRT appears to be a valuable tool in the development of measures.

Condensed version of the Quality of Life Scale for schizophrenia for use in outcome studies

The Quality of Life Scale (QLS(21)) is widely used in clinical trials involving schizophrenia patients. This study aimed to identify a core subset of QLS(21) items that maintains the validity and psychometric properties of the complete version. A parsimonious subset of items from the QLS(21) that can accurately predict the total scale score was sought and evaluated in 133 schizophrenia patients, using the heuristic algorithm for a regression model. Two additional data sets were used for model validation: a subset of 124 patients who participated in the model construction and who completed the QLS(21) 1 year later as well as a new sample of 40 inpatients. Patients were examined with the Positive and Negative Syndrome Scale (PANSS), the Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q), the Talbieh Brief Distress Inventory (TBDI), the Extrapyramidal Symptom Rating Scale (ESRS) and the Global Assessment of Functioning Scale (GAF). Using only five QLS items (social initiatives, adequacy, acquaintances, motivation, and time utilization; QLS(5)) as predictors, the correlation was 0.9805 between the predicted and true QLS totals. Two validation samples confirmed this finding. Additional analyses indicate that the QLS(5) exhibited similar performance to the QLS(21) regarding construct validity, test-retest reliability and responsiveness to changes over time. Thus, the five-item condensed Quality of Life Scale for schizophrenia maintains the validity of the full QLS, and has the advantage of shorter administration time. Utilization of the revised QLS(5) in routine care and clinical trials may potentially facilitate evaluation of treatment outcomes in schizophrenia.