More relevant, precise, and efficient items for assessment of physical function and disability: moving beyond the classic instruments

Constructing country-specific quality-of-life item banks for adults with amblyopia and strabismus in Australia and India

CLINICAL RELEVANCE

Understanding the quality-of-life (QoL) impacts of amblyopia and strabismus from the perspectives of patients using validated tools would enable eye care practitioners to provide better clinical management.

BACKGROUND

Item banks are advanced patient reported outcome measures with several advantages over traditional QoL assessment. This paper describes the development of amblyopia and strabismus-specific QoL item banks for two distinct country settings: Australia (high-income) and India (low-middle income) and examines the unique QoL issues.

METHODS

Using a bottom-up systematic approach, the content for the item banks was identified from three sources: existing self-report amblyopia and strabismus questionnaires (n = 22), qualitative literature (n = 5) and prospective qualitative studies in Australia (n = 49) and India (n = 30). The initial item pool underwent item evaluation, construction, and pre-testing to form optimal sets of representative items. The Indian item pools were first developed in English and translated into Hindi and Tamil using a rigorous translation protocol. The differences in QoL experiences that emanated from the qualitative studies and the number of common and unique items in the final item pools were compared.

RESULTS

The final Australian and Indian item pools comprised 312 and 277 items, respectively, covering 11 QoL domains. Two hundred and sixty items (79%) were common to both countries, and 21% were unique. Of the 11 domains, except for activity limitation (64.5%), visual symptoms (73.3%) and emotional impact (75.5%) domains, all other domains had over 80% of items common to both countries. The unique items can be attributed to differences in QoL experiences, individual perspectives, culture, lifestyle, country setting and health systems.

CONCLUSION

Amblyopia has a multifaceted impact on QoL irrespective of the country settings. Despite a huge overlap in QoL impacts between Australia and India, both countries had unique issues, especially activity limitations. The study developed comprehensive, country-specific item pools for Australia and India.

Satisfaction in Social Roles and Physical Function in Immune-mediated Inflammatory Diseases: A Cross-Sectional Study

BACKGROUND

Although mood disorders have been well characterized in Immune-mediated inflammatory diseases, physical function and satisfaction in social roles have not yet been defined as independent domains.

OBJECTIVE

The study aims to assess satisfaction in social roles and physical function alterations in an Immune-mediated inflammatory diseases population and identify associated characteristics.

METHODS

Physical function and social roles satisfaction were evaluated through the Patient-Reported Outcomes Measurement System. Besides comparisons between groups, univariate and multivariable logistic regression were performed to identify independent predictors.

RESULTS

Two hundred sixty-five Immune-mediated Inflammatory Diseases patients and 206 controls were recruited. Compared with controls, Inflammatory Bowel Diseases patients had impaired physical function (p<0.001), while Inflammatory Arthritis patients reported impairment in both domains (p<0.001, each). In the univariate logistic regression, gender, high school educational level, physical activity and occupation were positively associated with physical function and social role satisfaction (p<0.001; p=0.001; p<0.001; p=0.001 and p<0.001; p=0.012; p=0.008; p=0.004, respectively). Active disease and steroids were inversely associated with physical function and social roles satisfaction (p=0.033; p=0.022 and p=0.002; p=0.038, respectively). Further associations were found between age and physical function (p=0.002); biological treatment and ESR with social roles satisfaction (p<0.001; p=0.043; respectively). In the multivariable regression, gender remained associated with physical function (p<0.001) and social roles satisfaction (p=0.003). Negatively associated factors were biological treatment for satisfaction in social roles (p<0.001) and steroids for physical function (p=0.021) and social roles satisfaction (p=0.018).

CONCLUSION

Immune-mediated Inflammatory diseases determine alterations in physical function and social life satisfaction. Gender and treatment are independent associated factors. Patient-Reported Outcomes should be considered in clinical management to define patients' real needs.

Construct validity of the enfranchisement scale of the community participation indicators

OBJECTIVE

This study examined the construct validity of the Enfranchisement scale of the Community Participation Indicators.

DESIGN

We conducted a secondary analysis of data collected in a cross-sectional study of rehabilitation outcomes.

SUBJECTS

The parent study included 604 community-dwelling adults with chronic traumatic brain injury, stroke, or spinal cord injury. The sample had a mean age of 64.1 years, was two-thirds male, and included a high proportion of racial minorities (n = 250, 41.4%).

MAIN MEASURES

The Enfranchisement scale contains two subscales: the Control subscale and the Importance subscale. We examined correlations between each Enfranchisement subscale and measures of participation, environment, and impairments. The current analyses included cases with at least 80% of items completed on each subscale (Control subscale: n = 391; Importance subscale: n = 219). Missing values were imputed using multiple imputation.

RESULTS

The sample demonstrated high scores, indicating poor enfranchisement (Control subscale: M = 51.7; Importance subscale: M = 43.0). Both subscales were most strongly associated with measures of participation (Control subscale: r = 0.56; Importance subscale: r = 0.52), and least strongly associated with measures of cognition (Control subscale: r = 0.03; Importance subscale: r = 0.03). The Importance subscale was closely associated with depression (r = 0.54), and systems, services, and policies (r = 0.50). Both subscales were associated with social attitudes (Control subscale: r = 0.44; Importance subscale: r = 0.44) and social support (Control subscale: r = 0.49; Importance subscale: r = 0.41).

CONCLUSIONS

We found evidence of convergent validity between the Enfranchisement scale and measures of participation, and discriminant validity between the Enfranchisement scale and measures of disability-related impairments. The analyses also revealed the importance of the environment to enfranchisement outcomes.

Measurement properties of the Dutch-Flemish patient-reported outcomes measurement information system (PROMIS) physical function item bank and instruments: a systematic review

BACKGROUND

Limitations in physical functioning are a big concern especially for patients with chronic or musculoskeletal diseases. Therefore, physical functioning is often used as a core outcome of treatments. The generic patient-reported outcomes information system (PROMIS) physical function (PF) item bank has shown potential to measure PF with better precision, interpretability and lower respondent burden compared with traditional patient-reported outcome measures. This study provides an overview of the current evidence on the quality of the measurement properties of the translated Dutch-Flemish PROMIS-PF item bank and its subdomains, and their derived short forms and computer adaptive tests (CATs).

METHODS

PubMed was searched up to June 17th 2020 for validation studies of Dutch-Flemish PROMIS-PF in Dutch and Flemish adults. Quality assessment of the included studies was conducted using the COSMIN Risk of bias checklist. The COSMIN criteria for good measurement properties were used to judge the results of the studies, which were adjusted and added to where needed for this review, in the context of IRT instruments and item banks. The quality of evidence was summarized for each measurement property based on the Grading of Recommendation Assessment, Development, and Evaluation (GRADE) approach.

RESULTS

Eleven studies were included, evaluating the PROMIS-PF item bank, the Upper Extremity (UE) subdomain, and/or their derived short forms and CATs in different clinical populations. There is evidence for sufficient structural validity, measurement precision, construct validity, and cross-cultural validity of the Dutch-Flemish PROMIS-PF item bank. The upper extremity subdomain item bank shows high quality evidence for structural validity and measurement precision. Content validity of these item banks has not been thoroughly demonstrated in a Dutch-Flemish population. Furthermore, the derived instruments have far less robust evidence: there are fewer validation studies available and none examined their performance as stand-alone administered instruments.

CONCLUSIONS

The first studies into the Dutch-Flemish PROMIS-PF item bank and the UE subdomain show promising results, with especially high quality evidence for sufficient structural validity and measurement precision. However, more studies, and with higher methodological quality, are needed to study the instruments derived from these item banks. These studies should also evaluate content validity, reliability and responsiveness.

Differential item functioning of the PROMIS physical function, pain interference, and pain behavior item banks across patients with different musculoskeletal disorders and persons from the general population

PURPOSE

To investigate the validity of comparisons across patients with different musculoskeletal disorders and persons from the general population by evaluating differential item functioning (DIF) for the PROMIS physical function (PROMIS-PF), pain interference (PROMIS-PI), and pain behavior (PROMIS-PB) item banks.

METHODS

Patients with chronic pain, rheumatoid arthritis (RA), or osteoarthritis (OA); patients receiving physiotherapy (PT); and persons from the Dutch general population completed the full Dutch-Flemish PROMIS-PF (121-items), PROMIS-PI (40-items), or PROMIS-PB (39-items) banks. DIF was assessed with ordinal logistic regression models and McFadden's pseudo R²-change of ≥ 2% as critical value. The impact of DIF on item scores and the T-scores per bank was examined by inspecting item characteristic curves (ICCs) and test characteristic curves (TCCs).

RESULTS

2762 patients with chronic pain, 2029 with RA, 1247 with OA, 805 receiving PT, and 1310 healthy persons participated. For the PROMIS-PF, 25 out of 121 items were flagged for DIF, of which 10 items were flagged in multiple comparisons. For the PROMIS-PI, only 2 out of 40 items were flagged for DIF and for the PROMIS-PB, only 3 out of 39 items. Most DIF items had R² values just above the critical value of 2% and all showed uniform DIF. The ICCs and TCCs showed that the magnitude and impact of DIF on the item and T-scores were negligible.

CONCLUSIONS

This study supports the universal applicability of PROMIS across (patient) populations. Comparisons across patients with different musculoskeletal disorders and persons from the general population are valid, when applying the PROMIS-PF, PROMIS-PI, and PROMIS-PB banks.

Development and Evaluation of the Boston University Osteoarthritis Functional Pain Short Form (BU-OA-FPS)

BACKGROUND

Pragmatic studies have gained popularity, thus emphasizing the need for patient-reported outcomes (PRO) to be integrated into electronic health records.

OBJECTIVE

This study describes the development of a customized short form from the Boston University Osteoarthritis Functional Assessment PRO (BU-OA-PRO) for a specific pragmatic clinical trial.

METHODS

A Functional Pain Short Form was created from an existing item bank of deidentified data in the BU-OA-PRO. Item response theory (IRT) methods were used to select items. Reliability was measured with the Cronbach alpha, then with IRT simulation methods. To examine validity, ceiling and floor effects, correlations between the short-form scores and scores from the BU-OA-PRO and the Western Ontario McMasters University Osteoarthritis Index (WOMAC) Pain and Difficulty subscales, and the area under the curve (AUC) were calculated. A minimum detectable change at 90% confidence (MDC90) was calculated based on a calibration sample.

RESULTS

The BU-OA-PRO was reduced from 126 items to 10 items to create the BU-OA Functional Pain Short Form (BU-OA-FPS). The Cronbach alpha indicated high internal consistency (0.91), and reliability distribution estimates were 0.96 (uniform) and 0.92 (normal). Low ceiling effects (4.57%) and floor effects (0%) were found. Moderate-to-high correlations between the BU-OA-PRO and BU-OA-FPS were found with WOMAC Pain (BU-OA-FPS = 0.67; BU-OA-PRO = 0.64) and Difficulty (BU-OA-FPS = 0.73; BU-OA-PRO = 0.69) subscales. The correlation between the BU-OA-PRO and BU-OA-FPS was 0.94. The AUC ranged from 0.80 to 0.88. The MDC90 was approximately 6 standardized points.

CONCLUSIONS

The BU-OA-FPS provides reliable and valid measurement of functional pain. Pragmatic studies may consider the BU-OA-FPS for use in electronic health records to capture outcomes.

Psychometric properties of the PROMIS Physical Function item bank in patients receiving physical therapy

OBJECTIVES

The Patient-Reported Outcomes Measurement Information System (PROMIS) is a universally applicable set of instruments, including item banks, short forms and computer adaptive tests (CATs), measuring patient-reported health across different patient populations. PROMIS CATs are highly efficient and the use in practice is considered feasible with little administration time, offering standardized and routine patient monitoring. Before an item bank can be used as CAT, the psychometric properties of the item bank have to be examined. Therefore, the objective was to assess the psychometric properties of the Dutch-Flemish PROMIS Physical Function item bank (DF-PROMIS-PF) in Dutch patients receiving physical therapy.

DESIGN

Cross-sectional study.

SETTING AND PARTICIPANTS

805 patients >18 years, who received any kind of physical therapy in primary care in the past year, completed the full DF-PROMIS-PF (121 items).

METHODS

Unidimensionality was examined by Confirmatory Factor Analysis and local dependence and monotonicity were evaluated. A Graded Response Model was fitted. Construct validity was examined with correlations between DF-PROMIS-PF T-scores and scores on two legacy instruments (SF-36 Health Survey Physical Functioning scale [SF36-PF10] and the Health Assessment Questionnaire Disability-Index [HAQ-DI]). Reliability (standard errors of theta) was assessed.

RESULTS

The results for unidimensionality were mixed (scaled CFI = 0.924, TLI = 0.923, RMSEA = 0.045, 1th factor explained 61.5% of variance). Some local dependence was found (8.2% of item pairs). The item bank showed a broad coverage of the physical function construct (threshold-parameters range: -4.28-2.33) and good construct validity (correlation with SF36-PF10 = 0.84 and HAQ-DI = -0.85). Furthermore, the DF-PROMIS-PF showed greater reliability over a broader score-range than the SF36-PF10 and HAQ-DI.

CONCLUSIONS

The psychometric properties of the DF-PROMIS-PF item bank are sufficient. The DF-PROMIS-PF can now be used as short forms or CAT to measure the level of physical function of physiotherapy patients.

Detecting functional change in response to exercise in knee osteoarthritis: a comparison of two computerized adaptive tests

Background

The intent of this study was to examine and compare the ability to detect change of two patient reported outcome (PRO) instruments that use a computerized adaptive test (CAT) approach to measurement. The Patient Reported Outcomes Measurement Information System (PROMIS®) Physical Function scale is a generic PRO, while the Osteoarthritis Computerized Adaptive Test (OA-CAT) is an osteoarthritis-specific PRO.

Methods

This descriptive, longitudinal study was conducted in a community setting, involving individuals from the greater Boston area. Inclusion criteria: age > 50, self-reported doctor-diagnosed knee osteoarthritis (OA) and knee pain. The PROMIS® Physical Function CAT and OA-CAT Functional Difficulty scale were administered at baseline and at the conclusion of a 6-week exercise program. Effect sizes (ES) were calculated for both measures, and bootstrap methods were used to construct confidence intervals and to test for significant ES differences between the measures.

Results

The OA-CAT Functional Difficulty scale achieved an ES of 0.62 (0.43, 0.87) compared to the PROMIS® Physical Function CAT ES of 0.42 (0.24, 0.63). ES estimates for the two CAT measures were not statistically different.

Conclusions

The condition-specific OA-CAT and generic PROMIS® Physical Function CAT both demonstrated the ability to detect change in function. While the OA-CAT scale showed larger effect size, no statistically significant difference was found in the effect size estimates for the generic and condition-specific CATs. Both CATs have potential for use in arthritis research.

Trial registration

This trial is registered with ClinicalTrials.gov on 6/21/11 (Identifier NCT01394874)

Mobile Phone-Based Measures of Activity, Step Count, and Gait Speed: Results From a Study of Older Ambulatory Adults in a Naturalistic Setting

Background

Cellular mobile telephone technology shows much promise for delivering and evaluating healthcare interventions in cost-effective manners with minimal barriers to access. There is little data demonstrating that these devices can accurately measure clinically important aspects of individual functional status in naturalistic environments outside of the laboratory.

Objective

The objective of this study was to demonstrate that data derived from ubiquitous mobile phone technology, using algorithms developed and previously validated by our lab in a controlled setting, can be employed to continuously and noninvasively measure aspects of participant (subject) health status including step counts, gait speed, and activity level, in a naturalistic community setting. A second objective was to compare our mobile phone-based data against current standard survey-based gait instruments and clinical physical performance measures in order to determine whether they measured similar or independent constructs.

Methods

A total of 43 ambulatory, independently dwelling older adults were recruited from Nebraska Medicine, including 25 (58%, 25/43) healthy control individuals from our Engage Wellness Center and 18 (42%, 18/43) functionally impaired, cognitively intact individuals (who met at least 3 of 5 criteria for frailty) from our ambulatory Geriatrics Clinic. The following previously-validated surveys were obtained on study day 1: (1) Late Life Function and Disability Instrument (LLFDI); (2) Survey of Activities and Fear of Falling in the Elderly (SAFFE); (3) Patient Reported Outcomes Measurement Information System (PROMIS), short form version 1.0 Physical Function 10a (PROMIS-PF); and (4) PROMIS Global Health, short form version 1.1 (PROMIS-GH). In addition, clinical physical performance measurements of frailty (10 foot Get up and Go, 4 Meter walk, and Figure-of-8 Walk [F8W]) were also obtained. These metrics were compared to our mobile phone-based metrics collected from the participants in the community over a 24-hour period occurring within 1 week of the initial assessment.

Results

We identified statistically significant differences between functionally intact and frail participants in mobile phone-derived measures of percent activity (P=.002, t test), active versus inactive status (P=.02, t test), average step counts (P<.001, repeated measures analysis of variance [ANOVA]) and gait speed (P<.001, t test). In functionally intact individuals, the above mobile phone metrics assessed aspects of functional status independent (Bland-Altman and correlation analysis) of both survey- and/or performance battery-based functional measures. In contrast, in frail individuals, the above mobile phone metrics correlated with submeasures of both SAFFE and PROMIS-GH.

Conclusions

Continuous mobile phone-based measures of participant community activity and mobility strongly differentiate between persons with intact functional status and persons with a frailty phenotype. These measures assess dimensions of functional status independent of those measured using current validated questionnaires and physical performance assessments to identify functional compromise. Mobile phone-based gait measures may provide a more readily accessible and less-time consuming measure of gait, while further providing clinicians with longitudinal gait measures that are currently difficult to obtain.

Validity of PROMIS physical function measured in diverse clinical samples

OBJECTIVES

To evaluate the validity of the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function measures using longitudinal data collected in six chronic health conditions.

STUDY DESIGN AND SETTING

Individuals with rheumatoid arthritis (RA), major depressive disorder (MDD), back pain, chronic obstructive pulmonary disease (COPD), chronic heart failure (CHF), and cancer completed the PROMIS Physical Function computerized adaptive test or fixed-length short form at baseline and at the end of clinically relevant follow-up intervals. Anchor items were also administered to assess change in physical function and general health. Linear mixed-effects models and standardized response means were estimated at baseline and follow-up.

RESULTS

A total of 1,415 individuals participated (COPD n = 121; CHF n = 57; back pain n = 218; MDD n = 196; RA n = 521; cancer n = 302). The PROMIS Physical Function scores improved significantly for treatment of CHF and back pain patients but not for patients with MDD or COPD. Most of the patient subsamples that reported improvement or worsening on the anchors showed a corresponding positive or negative change in PROMIS Physical Function.

CONCLUSION

This study provides evidence that the PROMIS Physical Function measures are sensitive to change in intervention studies where physical function is expected to change and able to distinguish among different clinical samples. The results inform the estimation of meaningful change, enabling comparative effectiveness research.

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument

BACKGROUND

COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) is an initiative of an international multidisciplinary team of researchers who aim to improve the selection of outcome measurement instruments both in research and in clinical practice by developing tools for selecting the most appropriate available instrument.

METHOD

In this paper these tools are described, i.e. the COSMIN taxonomy and definition of measurement properties; the COSMIN checklist to evaluate the methodological quality of studies on measurement properties; a search filter for finding studies on measurement properties; a protocol for systematic reviews of outcome measurement instruments; a database of systematic reviews of outcome measurement instruments; and a guideline for selecting outcome measurement instruments for Core Outcome Sets in clinical trials. Currently, we are updating the COSMIN checklist, particularly the standards for content validity studies. Also new standards for studies using Item Response Theory methods will be developed. Additionally, in the future we want to develop standards for studies on the quality of non-patient reported outcome measures, such as clinician-reported outcomes and performance-based outcomes.

CONCLUSIONS

In summary, we plea for more standardization in the use of outcome measurement instruments, for conducting high quality systematic reviews on measurement instruments in which the best available outcome measurement instrument is recommended, and for stopping the use of poor outcome measurement instruments.

Impact of Verb Tense on Response to the Four-Dimensional Symptom Questionnaire (4DSQ)

Abstract. The verb tense of a questionnaire hypothetically might influence the way people respond to its items. We examined the effects of the verb tense on the response to the Four-Dimensional Symptom Questionnaire (4DSQ) in a population-based sample (N = 4,959; present tense N = 605; past tense N = 4,354). We determined whether the verb tense impacted the overall response rate, the scale structure, differential item functioning, reliability, proportions of respondents scoring above a range of cut-offs, and mean scale scores. The verb tense did not influence response rate, scale structure, item functioning, and reliability. The present tense increased the number of respondents scoring above very low cut-offs for distress and somatization. The effect on mean scale scores was limited and of little clinical importance.

Experiences of gout-related disability from the patients' perspective: a mixed methods study

Disability is a common problem in patients with gout. Recently, the Health Assessment Questionnaire Disability Index (HAQ-DI) was recommended for assessing patient reported activity limitations in gout. However, few studies have explored experiences and issues of disability from the perspective of gout patients themselves. The objectives of this study were to explore disability issues in patients with gout and to examine the content validity of the HAQ-DI in this patient group. Two studies were performed: a semi-structured interview study with seven male gout patients and a cross-sectional survey study among 34 other patients with gout. In both studies, specific examples of activity limitations were elicited using open-ended methods. The survey study additionally aimed to quantify the relevance of the activities listed in the HAQ-DI. Most patients experienced several gout attacks in the previous year. Limitations were reported to occur during a flare, but patients were generally not limited between attacks. During an attack, patients mainly experienced limitations related to mobility, especially walking and climbing stairs. Patients also mentioned limitations in activities related to domestic life, such as gardening and doing housework. Limitations related to self-care or activities requiring the use of the upper extremities were rarely mentioned. Corresponding HAQ-DI items were skewed towards very low disability scores over the past week. Assessments of gout-related disability should particularly focus on mobility and lower extremity functioning and should consider the intermittent nature of the disease. The HAQ-DI may not adequately meet these requirements, suggesting the need to explore other measures of gout-related disability

Identifying content for the glaucoma-specific item bank to measure quality-of-life parameters

PURPOSE

Patient-reported outcomes (PROs) have become essential clinical trial end points. However, a comprehensive, multidimensional, patient-relevant, and precise glaucoma-specific PRO instrument is not available. Therefore, the purpose of this study was to identify content for a new, glaucoma-specific, quality-of-life (QOL) item bank.

METHODS

Content identification was undertaken in 5 phases: (1) identification of extant items in glaucoma-specific instruments and the qualitative literature; (2) focus groups and interviews with glaucoma patients; (3) item classification and selection; (4) expert review and revision of items; and (5) cognitive interviews with patients.

RESULTS

A total of 737 unique items (extant items from PRO instruments, 247; qualitative articles, 14 items; focus groups and semistructured interviews, 476 items) were identified. These items were classified into 10 QOL domains. Four criteria (item redundancy, item inconsistent with domain definition, item content too narrow to have wider applicability, and item clarity) were used to remove and refine the items. After the cognitive interviews, the final minimally representative item set had a total of 342 unique items belonging to 10 domains: activity limitation (88), mobility (20), visual symptoms (19), ocular surface symptoms (22), general symptoms (15), convenience (39), health concerns (45), emotional well-being (49), social issues (23), and economic issues (22).

CONCLUSIONS

The systematic content identification process identified 10 QOL domains, which were important to patients with glaucoma. The majority of the items were identified from the patient-specific focus groups and semistructured interviews suggesting that the existing PRO instruments do not adequately address QOL issues relevant to individuals with glaucoma.

Response rates of three modes of survey administration and survey preferences of rheumatoid arthritis patients

OBJECTIVE

To compare survey response rates and preferences for 3 modes of survey administration in rheumatoid arthritis (RA) patients.

METHODS

Adult RA patients were identified from Kaiser Permanente Southern California's electronic medical records. One hundred patients each were randomly assigned to telephone, mail with followup letter, and mail with followup telephone call as modes of survey administration. Respondents completed a 7-page survey (in English or Spanish) that included the Health Assessment Questionnaire disability index and pain scale, EuroQol 5-dimensions, and generic questions on preferences and sociodemographics. Response rates were based on the number of completed surveys received from the total number of patients contacted (n = 295; 5 were ineligible).

RESULTS

The mean ± SD age of the cohort was 61 ± 12.3 years, with the majority being women (141 [85%] of 166). Of the 3 modes of survey administration, telephone (63 [64%] of 99) had the highest response rate, followed by mail with followup telephone call (55 [56%] of 98) and mail with followup letter (48 [49%] of 98). When asked about preference over administration mode, 57% (95 of 166) preferred to complete the survey by mail, followed by telephone (27 [16%] of 166) and internet (17 [10%] of 166).

CONCLUSION

When asked about their choice of survey administration mode for reporting their health status, the majority of RA patients in this study had a strong preference for a mail survey as compared to a telephone interview. However, the response rate in the telephone administration mode of survey was the highest between the 3 modes of administration that were compared. Future studies in RA can achieve better response using telephone over mail-in surveys.

Development and calibration of an item bank for the assessment of activities of daily living in cardiovascular patients using Rasch analysis

Background

To develop and calibrate the activities of daily living item bank (ADLib-cardio) as a prerequisite for a Computer-adaptive test (CAT) for the assessment of ADL in patients with cardiovascular diseases (CVD).

Methods

After pre-testing for relevance and comprehension a pool of 181 ADL items were answered on a five-point Likert scale by 720 CVD patients, who were recruited in fourteen German cardiac rehabilitation centers. To verify that the relationship between the items is due to one factor, a confirmatory factor analysis (CFA) was conducted. A Mokken analysis was computed to examine the double monotonicity (i.e. every item generates an equivalent order of person traits, and every person generates an equivalent order of item difficulties). Finally, a Rasch analysis based on the partial credit model was conducted to test for unidimensionality and to calibrate the item bank.

Results

Results of CFA and Mokken analysis confirmed a one factor structure and double monotonicity. In Rasch analysis, merging response categories and removing items with misfit, differential item functioning or local response dependency reduced the ADLib-cardio to 33 items. The ADLib-cardio fitted to the Rasch model with a nonsignificant item-trait interaction (chi-square=105.42, df=99; p=0.31). Person-separation reliability was 0.81 and unidimensionality could be verified.

Conclusions

The ADLib-cardio is the first calibrated, unidimensional item bank that allows for the assessment of ADL in rehabilitation patients with CVD. As such, it provides the basis for the development of a CAT for the assessment of ADL in patients with cardiovascular diseases. Calibrating the ADLib-cardio in other than rehabilitation cardiovascular patient settings would further increase its generalizability.

Development of an instrument to measure behavioral health function for work disability: item pool construction and factor analysis

OBJECTIVES

To develop a broad set of claimant-reported items to assess behavioral health functioning relevant to the Social Security disability determination processes, and to evaluate the underlying structure of behavioral health functioning for use in development of a new functional assessment instrument.

DESIGN

Cross-sectional.

SETTING

Community.

PARTICIPANTS

Item pools of behavioral health functioning were developed, refined, and field tested in a sample of persons applying for Social Security disability benefits (N=1015) who reported difficulties working because of mental or both mental and physical conditions.

INTERVENTIONS

None.

MAIN OUTCOME MEASURE

Social Security Administration Behavioral Health (SSA-BH) measurement instrument.

RESULTS

Confirmatory factor analysis (CFA) specified that a 4-factor model (self-efficacy, mood and emotions, behavioral control, social interactions) had the optimal fit with the data and was also consistent with our hypothesized conceptual framework for characterizing behavioral health functioning. When the items within each of the 4 scales were tested in CFA, the fit statistics indicated adequate support for characterizing behavioral health as a unidimensional construct along these 4 distinct scales of function.

CONCLUSIONS

This work represents a significant advance both conceptually and psychometrically in assessment methodologies for work-related behavioral health. The measurement of behavioral health functioning relevant to the context of work requires the assessment of multiple dimensions of behavioral health functioning. Specifically, we identified a 4-factor model solution that represented key domains of work-related behavioral health functioning. These results guided the development and scale formation of a new SSA-BH instrument.

Validation of a mobility item bank for older patients in primary care

BACKGROUND

To develop and validate an item bank to measure mobility in older people in primary care and to analyse differential item functioning (DIF) and differential bundle functioning (DBF) by sex.

METHODS

A pool of 48 mobility items was administered by interview to 593 older people attending primary health care practices. The pool contained four domains based on the International Classification of Functioning: changing and maintaining body position, carrying, lifting and pushing, walking and going up and down stairs.

RESULTS

The Late Life Mobility item bank consisted of 35 items, and measured with a reliability of 0.90 or more across the full spectrum of mobility, except at the higher end of better functioning. No evidence was found of non-uniform DIF but uniform DIF was observed, mainly for items in the changing and maintaining body position and carrying, lifting and pushing domains. The walking domain did not display DBF, but the other three domains did, principally the carrying, lifting and pushing items.

CONCLUSIONS

During the design and validation of an item bank to measure mobility in older people, we found that strength (carrying, lifting and pushing) items formed a secondary dimension that produced DBF. More research is needed to determine how best to include strength items in a mobility measure, or whether it would be more appropriate to design separate measures for each construct.

Modern psychometrics applied in rheumatology--a systematic review

BACKGROUND

Although item response theory (IRT) appears to be increasingly used within health care research in general, a comprehensive overview of the frequency and characteristics of IRT analyses within the rheumatic field is lacking. An overview of the use and application of IRT in rheumatology to date may give insight into future research directions and highlight new possibilities for the improvement of outcome assessment in rheumatic conditions. Therefore, this study systematically reviewed the application of IRT to patient-reported and clinical outcome measures in rheumatology.

METHODS

Literature searches in PubMed, Scopus and Web of Science resulted in 99 original English-language articles which used some form of IRT-based analysis of patient-reported or clinical outcome data in patients with a rheumatic condition. Both general study information and IRT-specific information were assessed.

RESULTS

Most studies used Rasch modeling for developing or evaluating new or existing patient-reported outcomes in rheumatoid arthritis or osteoarthritis patients. Outcomes of principle interest were physical functioning and quality of life. Since the last decade, IRT has also been applied to clinical measures more frequently. IRT was mostly used for evaluating model fit, unidimensionality and differential item functioning, the distribution of items and persons along the underlying scale, and reliability. Less frequently used IRT applications were the evaluation of local independence, the threshold ordering of items, and the measurement precision along the scale.

CONCLUSION

IRT applications have markedly increased within rheumatology over the past decades. To date, IRT has primarily been applied to patient-reported outcomes, however, applications to clinical measures are gaining interest. Useful IRT applications not yet widely used within rheumatology include the cross-calibration of instrument scores and the development of computerized adaptive tests which may reduce the measurement burden for both the patient and the clinician. Also, the measurement precision of outcome measures along the scale was only evaluated occasionally. Performed IRT analyses should be adequately explained, justified, and reported. A global consensus about uniform guidelines should be reached concerning the minimum number of assumptions which should be met and best ways of testing these assumptions, in order to stimulate the quality appraisal of performed IRT analyses.

Calibrating ADL-IADL scales to improve measurement accuracy and to extend the disability construct into the preclinical range: a systematic review

BACKGROUND

Interest in measuring functional status among nondisabled older adults has increased in recent years. This is, in part, due to the notion that adults identified as 'high risk' for functional decline portray a state that is potentially easier to reverse than overt disability. Assessing relatively healthy older adults with traditional self-report measures (activities of daily living) has proven difficult because these instruments were initially developed for institutionalised older adults. Perhaps less evident, are problems associated with change scores and the potential for 'construct under-representation', which reflects the exclusion of important features of the construct (e.g., disability). Furthermore, establishing a formal hierarchy of functional status tells more than the typical simple summation of functional loss, and may have predictive value to the clinician monitoring older adults: if the sequence task difficulty is accelerated or out of order it may indicate the need for interventions.

METHODS

This review identified studies that employed item response theory (IRT) to examine or revise functional status scales. IRT can be used to transform the ordinal nature of functional status scales to interval level data, which serves to increase diagnostic precision and sensitivity to clinical change. Furthermore, IRT can be used to rank items unequivocally along a hierarchy based on difficulty. It should be noted that this review is not concerned with contrasting IRT with more traditional classical test theory methodology.

RESULTS

A systematic search of four databases (PubMed, Embase, CINAHL, and PsychInfo) resulted in the review of 2,192 manuscripts. Of these manuscripts, twelve met our inclusion/exclusion requirements and thus were targeted for further inspection.

CONCLUSIONS

Manuscripts presented in this review appear to summarise gerontology's best efforts to improve construct validity and content validity (i.e., ceiling effects) for scales measuring the early stages of activity restriction in community-dwelling older adults. Several scales in this review were exceptional at reducing ceiling effects, reducing gaps in coverage along the construct, as well as establishing a formal hierarchy of functional decline. These instrument modifications make it plausible to detect minor changes in difficulty for IADL items positioned at the edge of the disability continuum, which can be used to signal the onset of progressive type disability in older adults.

Domains of health-related quality of life important and relevant to multiethnic English-speaking Asian systemic lupus erythematosus patients: a focus group study

OBJECTIVE

To identify health-related quality of life (HRQOL) domains of importance to multiethnic Asian systemic lupus erythematosus (SLE) patients, to identify content gaps in existing SLE-specific HRQOL measures, and to determine whether the Patient-Reported Outcomes Measurement Information System (PROMIS) item banks could serve as a core set of questions for HRQOL assessment among SLE patients.

METHODS

English-speaking patients with physician-diagnosed SLE from a specialist clinic in a tertiary care hospital in Singapore and a patient support group were recruited. Thematic analysis was performed to distill themes from transcripts through open coding by 2 independent coders and axial coding for refinement of categories. Items from 3 existing SLE-specific measures and PROMIS Version 1.0 Item Banks were compared with identified subthemes.

RESULTS

Twenty-seven female and 2 male participants (21 Chinese, 4 Malay, 3 Indian, 1 other) ages 23-62 years participated in 6 focus groups and 2 individual interviews, respectively. Twenty-one domains and 92 subthemes were identified. Domains of family, relationships, stigma and discrimination, and freedom were unaddressed by existing SLE-specific measures. Forty subthemes from 14 domains were addressed by the PROMIS Version 1.0 Item Banks (Physical Function, Pain, Fatigue, Sleep Disturbance, Sleep-Related Impairment, Anger, Anxiety, and Depression banks).

CONCLUSION

Family and stigma and discrimination (identified as content gaps) may be accentuated in the Asian sociocultural context. PROMIS item banks have tremendous potential to serve as a core set of items for HRQOL assessment in SLE patients. Additional items may be written to fill the gaps in existing PROMIS item banks.

Using video animation to assess mobility in older adults

BACKGROUND

Although the importance of the context of task performance in the assessment of mobility in older adults is generally understood, there is little empirical evidence that demonstrates how sensitive older adults are to subtle changes in task demands. Thus, we developed a novel approach to examine this issue.

METHODS

We collected item response data to 81 animated video clips, where various mobility-related tasks were modified in a systematic fashion to manipulate task difficulty.

RESULTS

The participants (N = 234), 166 women and 68 men, had an average age of 81.9 years and a variety of comorbidities. Histograms of item responses revealed dramatic and systematic effects on older adults' self-reported ability when varying walking speed, use of a handrail during ascent and descent of stairs, walking at different speeds outdoors over uneven terrain, and carrying an object. For example, there was almost a threefold increase in reporting the inability to walk at the fast speed compared with the slow speed for a minute or less, and twice as many participants reported the inability to walk at the fast speed outdoors over uneven terrain compared with indoors.

CONCLUSIONS

The data provide clear evidence that varying the contextual features and demands of a simple task such as stair climbing has a significant impact on older adults' self-reporting of ability related to mobility. More work is needed on the psychometric properties of such assessments and to determine if this methodology has conceptual and clinical relevance in studying mobility disability.

The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008

OBJECTIVES

Patient-reported outcomes (PROs) are essential when evaluating many new treatments in health care; yet, current measures have been limited by a lack of precision, standardization, and comparability of scores across studies and diseases. The Patient-Reported Outcomes Measurement Information System (PROMIS) provides item banks that offer the potential for efficient (minimizes item number without compromising reliability), flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate) measurement of commonly studied PROs. We report results from the first large-scale testing of PROMIS items.

STUDY DESIGN AND SETTING

Fourteen item pools were tested in the U.S. general population and clinical groups using an online panel and clinic recruitment. A scale-setting subsample was created reflecting demographics proportional to the 2000 U.S. census.

RESULTS

Using item-response theory (graded response model), 11 item banks were calibrated on a sample of 21,133, measuring components of self-reported physical, mental, and social health, along with a 10-item Global Health Scale. Short forms from each bank were developed and compared with the overall bank and with other well-validated and widely accepted ("legacy") measures. All item banks demonstrated good reliability across most of the score distributions. Construct validity was supported by moderate to strong correlations with legacy measures.

CONCLUSION

PROMIS item banks and their short forms provide evidence that they are reliable and precise measures of generic symptoms and functional reports comparable to legacy instruments. Further testing will continue to validate and test PROMIS items and banks in diverse clinical populations.

Self-reported functional status as predictor of observed functional capacity in subjects with early osteoarthritis of the hip and knee: a diagnostic study in the CHECK cohort

OBJECTIVES

Patients with hip or knee osteoarthritis (OA) may experience functional limitations in work settings. In the Cohort Hip and Cohort Knee study (CHECK) physical function was both self-reported and measured performance-based, using Functional Capacity Evaluation (FCE). Relations between self-reported scores on SF-36 and WOMAC (Western Ontario and McMaster Arthritis Index, function scales) and FCE performance were studied, and their diagnostic value for clinicians in predicting observed physical work limitations was assessed.

METHODS

Ninety-two subjects scored physical function on SF-36 (scale 0-100, 100 indicating the best health level) and WOMAC (scale 0-68, 68 indicates maximum restriction) and performed the FCE. Correlations were calculated between all scores. Cross-tables were constructed using both questionnaires as diagnostic tests to identify work limitations. Subjects lifting <22.5 kg on the FCE-test 'lifting-low' were labeled as having physical work limitations. Diagnostic aspects at different cut-off scores for both questionnaires were analysed.

RESULTS

Statistically significant correlations (Spearman's rho 0.34-0.49) were found between questionnaire scores and lifting and carrying tests. Results of a diagnostic cross-table with cut-off point <60 on SF-36 'physical functioning' were: sensitivity 0.34, specificity 0.97 and positive predictive value (PV+) 0.95. Cut-off point > or =21 on WOMAC 'function' resulted in sensitivity 0.51, specificity 0.88 and PV+ 0.88.

CONCLUSION

Low self-reported function scores on SF-36 and WOMAC diagnosed subjects with limitations on the FCE. However, high scores did not guarantee performance without physical work limitations. These results are specific to the tested persons with early OA, in populations with a different prevalence of limitations, different diagnostic values will be found. FCE may be indicated to help clinicians to assess actual work capacity.

Using item banks to construct measures of patient reported outcomes in clinical trials: investigator perceptions

BACKGROUND

Item response theory (IRT) promises more sensitive and efficient measurement of patient-reported outcomes (PROs) than traditional approaches; however, the selection and use of PRO measures from IRT-based item banks differ from current methods of using PRO measures.

PURPOSE

To anticipate barriers to the adoption of IRT item banks into clinical trials.

METHODS

We conducted semistructured telephone or in-person interviews with 42 clinical researchers who published results from clinical trials in the Journal of the American Medical Association, the New England Journal of Medicine, or other leading clinical journals from July 2005 through May 2006. Interviews included a brief tutorial on IRT item banks.

RESULTS

After the tutorial, 39 of 42 participants understood the novel products available from an IRT item bank, namely customized short forms and computerized adaptive testing. Most participants (38/42) thought that item banks could be useful in their clinical trials, but they mentioned several potential barriers to adoption, including economic and logistical constraints, concerns about whether item banks are better than current PRO measures, concerns about how to convince study personnel or statisticians to use item banks, concerns about FDA or sponsor acceptance, and the lack of availability of item banks validated in specific disease populations.

LIMITATIONS

Selection bias might have led to more positive responses to the concept of item banks in clinical trials.

CONCLUSIONS

Clinical investigators are open to a new method of PRO measurement offered in IRT item banks, but bank developers must address investigator and stakeholder concerns before widespread adoption can be expected.