Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes

Applying multidimensional computerized adaptive testing to the MSQOL-54: a simulation study

BACKGROUND

The Multiple Sclerosis Quality of Life-54 (MSQOL-54) is one of the most commonly-used MS-specific health-related quality of life (HRQOL) measures. It is a multidimensional, MS-specific HRQOL inventory, which includes the generic SF-36 core items, supplemented with 18 MS-targeted items. Availability of an adaptive short version providing immediate item scoring may improve instrument usability and validity. However, multidimensional computerized adaptive testing (MCAT) has not been previously applied to MSQOL-54 items. We thus aimed to apply MCAT to the MSQOL-54 and assess its performance.

METHODS

Responses from a large international sample of 3669 MS patients were assessed. We calibrated 52 (of the 54) items using bifactor graded response model (10 group factors and one general HRQOL factor). Then, eight simulations were run with different termination criteria: standard errors (SE) for the general factor and group factors set to different values, and change in factor estimates from one item to the next set at < 0.01 for both the general and the group factors. Performance of the MCAT was assessed by the number of administered items, root mean square difference (RMSD), and correlation.

RESULTS

Eight items were removed due to local dependency. The simulation with SE set to 0.32 (general factor), and no SE thresholds (group factors) provided satisfactory performance: the median number of administered items was 24, RMSD was 0.32, and correlation was 0.94.

CONCLUSIONS

Compared to the full-length MSQOL-54, the simulated MCAT required fewer items without losing precision for the general HRQOL factor. Further work is needed to add/integrate/revise MSQOL-54 items in order to make the calibration and MCAT performance efficient also on group factors, so that the MCAT version may be used in clinical practice and research.

Feasibility of PROMIS using computerized adaptive testing during inpatient rehabilitation

BACKGROUND

There has been an increased significance on patient-reported outcomes in clinical settings. We aimed to evaluate the feasibility of administering patient-reported outcome measures by computerized adaptive testing (CAT) using a tablet computer with rehabilitation inpatients, assess workload demands on staff, and estimate the extent to which rehabilitation inpatients have elevated T-scores on six Patient Reported Outcomes Measurement Information System® (PROMIS®) measures.

METHODS

Patients (N = 108) with stroke, spinal cord injury, traumatic brain injury, and other neurological disorders participated in this study. PROMIS computerized adaptive tests (CAT) were administered via a web-based platform. Summary scores were calculated for six measures: Pain Interference, Sleep Disruption, Anxiety, Depression, Illness Impact Positive, and Illness Impact Negative. We calculated the percent of patients with T-scores equivalent to 2 standard deviations or greater above the mean.

RESULTS

During the first phase, we collected data from 19 of 49 patients; of the remainder, 61% were not available or had cognitive or expressive language impairments. In the second phase of the study, 40 of 59 patients participated to complete the assessment. The mean PROMIS T-scores were in the low 50 s, indicating an average symptom level, but 19-31% of patients had elevated T-scores where the patients needed clinical action.

CONCLUSIONS

The study demonstrated that PROMIS assessment using a CAT administration during an inpatient rehabilitation setting is feasible with the presence of a research staff member to complete PROMIS assessment.

Technological developments enable measuring and using patient-reported outcomes data in orthopaedic clinical practice

Patient-reported outcomes measures form the backbone of outcomes evaluation in orthopaedics, with most of the literature now relying on these scoring tools to measure change in patient health status. This patient-reported information is increasingly collected routinely by orthopaedic providers but use of the data is typically restricted to academic research. Developments in electronic data capture and the outcome tools themselves now allow use of this data as part of the clinical consultation. This review evaluates the role of patient reported outcomes data as a tool to enhance daily orthopaedic clinical practice, and documents how develop-ments in electronic outcome measures, computer-adaptive questionnaire design and instant graphical display of questionnaire can facilitate enhanced patient-clinician shared decision making.

Sensitivity of the SCI-FI/AT in Individuals With Traumatic Spinal Cord Injury

OBJECTIVE

To examine the ability of the Spinal Cord Injury-Functional Index/Assistive Technology (SCI-FI/AT) measure to detect change in persons with spinal cord injury (SCI).

DESIGN

Multisite longitudinal (12-mo follow-up) study.

SETTING

Nine SCI Model Systems programs.

PARTICIPANTS

Adults (N=165) with SCI enrolled in the SCI Model Systems database.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

SCI-FI/AT computerized adaptive test (CAT) (Basic Mobility, Self-Care, Fine Motor Function, Wheelchair Mobility, and/or Ambulation domains) completed at discharge from rehabilitation and 12 months after SCI. For each domain, effect size estimates and 95% confidence intervals were calculated for subgroups with paraplegia and tetraplegia.

RESULTS

The demographic characteristics of the sample were as follows: 46% (n=76) individuals with paraplegia, 76% (n=125) male participants, 57% (n=94) used a manual wheelchair, 38% (n=63) used a power wheelchair, 30% (n=50) were ambulatory. For individuals with paraplegia, the Basic Mobility, Self-Care, and Ambulation domains of the SCI-FI/AT detected a significantly large amount of change; in contrast, the Fine Motor Function and Wheelchair Mobility domains detected only a small amount of change. For those with tetraplegia, the Basic Mobility, Fine Motor Function, and Self-Care domains detected a small amount of change whereas the Ambulation item domain detected a medium amount of change. The Wheelchair Mobility domain for people with tetraplegia was the only SCI-FI/AT domain that did not detect significant change.

CONCLUSIONS

SCI-FI/AT CAT item banks detected an increase in function from discharge to 12 months after SCI. The effect size estimates for the SCI-FI/AT CAT vary by domain and level of lesion. Findings support the use of the SCI-FI/AT CAT in the population with SCI and highlight the importance of multidimensional functional measures.

Redefining Health: Implication for Value-Based Healthcare Reform

Health definition consists of three domains namely, physical, mental, and social health that should be prioritized in delivering healthcare. The emergence of chronic diseases in aging populations has been a barrier to the realization of a healthier society. The value-based healthcare concept seems in line with the true health objective: increasing value. Value is created from health outcomes which matter to patients relative to the cost of achieving those outcomes. The health outcomes should include all domains of health in a full cycle of care. To implement value-based healthcare, transformations need to be done by both health providers and patients: establishing true health outcomes, strengthening primary care, building integrated health systems, implementing appropriate health payment schemes that promote value and reduce moral hazards, enabling health information technology, and creating a policy that fits well with a community.

Redefining Health: Implication for Value-Based Healthcare Reform

Health definition consists three domains, namely physicial, mental and social health, that should be prioritized in delivering healthcare. The emergence of chronic diseases in the aging populations has been a barrier to the realization of healthier society. The value-based healthcare concept seems in line with the true health objectives: increasing value. Value is created from health outcomes which matter to patient relative to cost of achieving those outcomes. The health outcomes should include all domain of health in a full cycle of care. To implement value-based healthcare, transformations need to be done by both health providers and patients: establishing true health outcomes, strenghtening primary care, buidling integrated health system, implementing appropriate health payment scheme that promote value and reduce moral hazards, enabling health information technology, and policy that fits well with community.

Evaluation of Computerized Adaptive Tests (CATs) for longitudinal monitoring of depression, anxiety, and stress reactions

BACKGROUND

Computerized adaptive testing (CAT) based on Item Response Theory, (IRT) offers an efficient way for accurate measurement of patient reported outcomes. The efficiency lies within a minimal response burden and a high measurement precision over a broad measurement range. The objective of the study was to evaluate and compare the responsiveness of CATs measuring anxiety, depression, and stress reaction to standard static self-assessment tools.

METHODS

Longitudinal data of n=595 psychosomatic inpatients were analyzed for evaluating retest-reliability and sensitivity to change of the CATs compared to static measures (GAD-7, PHQ-9, and PSQ) using correlational and ANOVA statistics. The study hypothesized that CATs are at least as retest-reliable and as sensitive to change as static tools.

RESULTS

The three CATs show a low burden for patients, administering on average 5-7 (±2-6SD) items with similar retest-reliability compared to the static tools applied (A-CAT: r=.78 vs. GAD-7: r=.75, D-CAT: r=.71 vs. PHQ-9: r=.75, S-CAT: r=.80 vs. PSQworries scale: r=.80). The CATs were overall as sensitive to change as the static tools (Cohen׳s d ranged between .19 and .69).

LIMITATIONS

This is a monocenter, observational, longitudinal study without external clinical criteria; thus generalization to other settings may be limited.

CONCLUSIONS

The tested CATs belong to the first generation of CATs being used in daily routine for more than a decade. They are as retest reliable and sensitive to change as static tools. Newer CATs may provide further practical advantages.

Feasibility of computerized adaptive testing for collection of patient-reported outcomes after inpatient rehabilitation

OBJECTIVE

To evaluate the feasibility of computer adaptive testing (CAT) using an Internet or telephone interface to collect patient-reported outcomes after inpatient rehabilitation and to examine patient characteristics associated with completion of the CAT-administered measure and mode of administration.

DESIGN

Prospective cohort study of patients contacted approximately 4 weeks after discharge from inpatient rehabilitation. Patients selected an Internet or telephone interface.

SETTING

Rehabilitation hospital.

PARTICIPANTS

Patients (N=674) with diagnoses of neurologic, orthopedic, or medically complex conditions.

INTERVENTIONS

None.

MAIN OUTCOME MEASURE

CAT version of the Community Participation Indicators (CAT-CPI).

RESULTS

From an eligible pool of 3221 patients, 674 (21%) agreed to complete the CAT-CPI. Patients who agreed to complete the CAT-CPI were younger and reported slightly higher satisfaction with overall care than those who did not participate. Among these patients, 231 (34%) actually completed the CAT-CPI; 141 (61%) selected telephone administration, and 90 (39%) selected Internet administration. Decreased odds of completing the CAT-CPI were associated with black and other race; stroke, brain injury, or orthopedic and other impairments; and being a Medicaid beneficiary, whereas increased odds of completing the CAT-CPI were associated with longer length of stay and higher discharge FIM cognition measure. Decreased odds of choosing Internet administration were associated with younger age, retirement status, and being a woman, whereas increased odds of choosing Internet administration were associated with higher discharge FIM motor measure.

CONCLUSIONS

CAT administration by Internet and telephone has limited feasibility for collecting postrehabilitation outcomes for most rehabilitation patients, but it is feasible for a subset of patients. Providing alternative ways of answering questions helps assure that a larger proportion of patients will respond.

Development and Psychometric Evaluation of a German Version of the PROMIS® Item Banks for Satisfaction With Participation

The Patient Reported Outcomes Measurement Information System (PROMIS) initiative aims to provide reliable and precise item banks measuring patient-reported outcomes in different health domains. The aim of the present work was to provide a German translation of the PROMIS item banks for satisfaction with participation and to psychometrically test these German versions. Cognitive interviews followed a forward-backward translation. Distribution characteristics, unidimensionality, Rasch model fit, reliability, construct validity, and internal responsiveness were tested in 262 patients with chronic low back pain undergoing rehabilitation. Results for the final 13- and 10-item German static scales (Satisfaction with Participation in Social Roles-German version [PSR-G] and Satisfaction for Participation in Discretionary Social Activities-German version [PSA-G]) regarding unidimensionality were satisfactory. The scales are reliable and show good Rasch model fit and distribution characteristics. Both scales are sensitive to small to moderate clinical changes, and we observed initial proof of construct validity. These German versions of the Satisfaction with Participation scales can be recommended to assess participation in a clinical context. The scales' applicability in other contexts should be examined.

Development of a computerized adaptive test for assessing activities of daily living in outpatients with stroke

BACKGROUND

An efficient, reliable, and valid measure for assessing activities of daily living (ADL) function is useful to improve the efficiency of patient management and outcome measurement.

OBJECTIVE

The purpose of this study was to construct a computerized adaptive testing (CAT) system for measuring ADL function in outpatients with stroke.

DESIGN

Two cohort studies were conducted at 6 hospitals in Taiwan.

METHODS

A candidate item bank (44 items) was developed, and 643 outpatients were interviewed. An item response theory model was fitted to the data and estimated the item parameters (eg, difficulty and discrimination) for developing the ADL CAT. Another sample of 51 outpatients was interviewed to examine the concurrent validity and efficiency of the CAT. The ADL CAT, as the outcome measure, and the Barthel index (BI) and Frenchay Activities index (FAI) were administered on the second group of participants.

RESULTS

Ten items did not satisfy the model's expectations and were deleted. Thirty-four items were included in the final item bank. Two stopping rules (ie, reliability coefficient >.9 and maximum test length of 7 items) were set for the CAT. The participants' ADL scores had an average reliability of .93. The CAT scores were highly associated with those of the full 34 items (Pearson r=.98). The scores of the CAT were closely correlated with those of the combined BI and FAI (r=.82). The time required to complete the CAT was about one fifth of the time used to administer both the BI and FAI.

LIMITATIONS

The participants were outpatients living in the community. Further studies are needed to cross-validate the results.

CONCLUSIONS

The results demonstrated that the ADL CAT is quick to administer, reliable, and valid in outpatients with stroke.

Development of a computer-adaptive version of the forgotten joint score

Patient-reported outcomes (PROs) are an important endpoint in orthopedics providing comprehensive information about patients' perspectives on treatment outcome. Computer-adaptive test (CAT) measures are an advanced method for assessing PROs using item sets that are tailored to the individual patient. This provides increased measurement precision and reduces the number of items. We developed a CAT version of the Forgotten Joint Score (FJS), a measure of joint awareness in everyday life. CAT development was based on FJS data from 580 patients after THA or TKA (808 assessments). The CAT version reduced the number of items by half at comparable measurement precision. In a feasibility study we administered the newly developed CAT measure on tablet PCs and found that patients actually preferred electronic questionnaires over paper-pencil questionnaires.

Responsiveness of two participation measures in an outpatient rehabilitation setting

OBJECTIVE

The responsiveness of a new participation measure, the Utrecht Scale for Evaluation of Rehabilitation-Participation (USER-Participation) was compared with that of the Impact on Particpation and Autonomy (IPA). Furthermore, the concurrent validity of these two measures was assessed.

METHOD

Participants were patients with brain injury or neuromuscular disease, who received occupational therapy as part of a multidisciplinary outpatient rehabilitation programme. They completed the IPA and the USER-Participation at the start and end of the programme, and at three-month follow-up. Responsiveness was analysed using the effect size (ES) and the standardized response mean (SRM).

RESULTS

Responsiveness figures were small to moderate (-0.4 to 0.5) and similar for the IPA and the USER-Participation. The USER-Participation showed a consistent result, with Frequency scores declining and Restriction and Satisfaction scores inclining over time, and the IPA showed mixed results, with the different domain scores both inclining and declining over time. Correlations between IPA and USER-Participation scales were all significant, but by far strongest for the USER-Participation Satisfaction scale (-0.64 to -0.81).

CONCLUSION

The USER-Participation was at least as responsive as the IPA. Further, satisfaction with participation as measured with the USER-Participation is highly similar to the concept of autonomy in participation as measured with the IPA.

Computerized adaptive testing--ready for ambulatory monitoring?

BACKGROUND

Computerized adaptive tests (CATs) have abundant theoretical advantages over established static instruments, which could improve ambulatory monitoring of patient-reported outcomes (PROs). However, an empirical demonstration of their practical benefits is warranted.

METHODS

We reviewed the literature and evaluated existing data to discuss the potential of CATs for use in ambulatory monitoring outside clinical facilities.

RESULTS

Computerized adaptive tests are not being used for ambulatory monitoring, but initial results from their use in health care research allow for discussion of some issues relevant to ambulatory care. Evidence shows that CATs can capture the most relevant health outcomes as well as established static tools, with substantially decreased respondent burden. They can be more precise than static tools of similar length and can reduce floor and ceiling effects. Computerized adaptive tests can reliably measure a construct over time with different items, which yields the potential of introducing item exposure control in ambulatory monitoring. Studies have shown that CATs can be at least as valid as well-designed static tools in group comparisons, but further investigation is needed to determine whether psychometric advantages lead to increased responsiveness of CATs.

CONCLUSIONS

Ambulatory monitoring of PROs demands short, yet very precise measurements, which can be repeated up to many times a day. Computerized adaptive tests may address several present shortcomings in ambulatory monitoring of PROs efficiently. However, most CAT developments have primarily focused on psychometric improvements. To use the full potential of CATs for ambulatory monitoring purposes, content must also be carefully considered.

Long-term disabilities associated with combat casualties: measuring disability and reintegration in combat veterans

Many physical and mental health problems associated with combat casualties affect the reintegration of service members into home and community life. Quantifying and measuring reintegration is important to answer questions about clinical, research, economic, and policy issues that directly affect combat veterans. Although the construct of participation presented in the International Statistical Classification of Diseases and Related Health Problems and in the International Classification of Functioning, Disability and Health provides a theoretical framework with which to understand and measure community reintegration in general, a measure was needed that specifically addressed the reintegration of combat veterans. To address this need, the Community Reintegration for Service Members global outcomes measure was developed. It consists of three scales, which measure extent of participation, perceived limitations, and satisfaction. The measure was validated in a general sample of veterans and in a sample of severely wounded service members. The computer-adapted test version shows good precision, reliability, construct validity, and predictive validity.

Response burden and questionnaire length: is shorter better? A review and meta-analysis

BACKGROUND

Response burden is often defined as the effort required by the patient to answer a questionnaire. A factor that has been proposed to affect the response burden is questionnaire length, and this burden is manifested in, for example, response rate. Even though response burden is frequently mentioned as a reason for abridging questionnaires, evidence to support the notion that shorter instruments are preferable is limited.

OBJECTIVES

This study aimed to accumulate, analyze, and discuss evidence regarding the association between response burden, as measured by response rate, and questionnaire length.

METHODS

A systematic literature review and meta-analysis of studies reporting response rates in relation to questionnaire length was performed. A Cochran-Mantel-Haenszel test stratified by study using the Breslow-Day test was undertaken to investigate homogeneity of the odds ratios.

RESULTS

Thirty-two reports were identified, of which 20 were eligible for inclusion in the meta-analysis. Three studies used patient input as main outcome when evaluating response burden. In the meta-analysis, a general association between response rate and questionnaire length was found (P ≤ 0.0001). Response rates were lower for longer questionnaires, but because the P value for test of homogeneity was P = 0.03, this association should be interpreted with caution because it is impossible to separate the impact of content from length of the questionnaires.

CONCLUSION

Given the inherently problematic nature of comparing questionnaires of various lengths, it is preferable to base decisions on use of instruments on the content rather than the length per se.

Participação e necessidade de assistência na realização de tarefas escolares em crianças com paralisia cerebral

INTRODUÇÃO: A educação é um bem-estar relacionado à saúde. Acredita-se que no ambiente escolar as crianças com necessidades especiais podem alcançar o mais completo progresso educacional e de integração social. OBJETIVO: O objetivo deste estudo foi avaliar a participação e a necessidade de assistência na realização de tarefas escolares em crianças com paralisia cerebral (PC). MATERIAIS E MÉTODOS: Trinta crianças com PC foram avaliadas por meio do School Function Assessment (SFA). A análise estatística foi realizada por meio do teste Mann Whitney, do coeficiente de correlação de Spearman e da análise de regressão múltipla stepwise backward. RESULTADOS: Foram encontradas diferenças significativas entre os grupos em todas as partes do SFA. Houve correlação negativa entre o nível de comprometimento motor e as partes I e II do SFA e correlação positiva entre as partes I e II do SFA. CONCLUSÃO: A inserção das crianças com PC em um meio coletivo e de participação é dificultada, muitas vezes, pelos múltiplos comprometimentos da criança. O grau de comprometimento motor e a necessidade de assistência na realização de tarefas físicas e cognitivas/comportamentais juntos explicaram 89% da variabilidade da participação escolar em crianças com PC. A necessidade de assistência na realização de tarefas físicas foi o fator que representou maior influência à restrição na participação escolar.

A systematic review of instruments assessing participation: challenges in defining participation

OBJECTIVES

To evaluate: (1) whether instruments which intend to measure participation actually do and (2) how frequently specific aspects and domains of participation are addressed.

DATA SOURCES

A systematic search was performed in PubMed.

STUDY SELECTION

Included were patient-reported instruments that primarily aim to measure participation.

DATA EXTRACTION

The full-text instruments were extracted from the articles or obtained from the authors. Two reviewers independently rated each item of the included instruments as measuring participation (yes, no, or undetermined). For each item, the specific aspect and domain of participation were categorized.

DATA SYNTHESIS

Included were 103 instruments (2445 items). Of the included items, 619 items concerned participation and 217 concerned undetermined items. In total, 68 instruments contained at least 1 (sub)scale with 50% or more participation or undetermined items. The participation items referred to the participation aspects: participation problems (53%), participation accomplishment (31%), and satisfaction with participation (9%). The domains of the participation items concerned: work/study (27%), social life (27%), general participation (19%), and home (11%). The undetermined items mainly referred to domains about leisure (43%), transport (26%), and shopping (12%).

CONCLUSIONS

According to our working definition of participation, most instruments that aim to measure participation do so only to a limited extent. These instruments mainly assess aspects of participation problems and participation accomplishment. The domains of participation covered by these instruments primarily include work/study, social life, general participation, home, leisure, transport, and shopping.

Assessing the factor structure of a role functioning item bank

PURPOSE

Role functioning (RF) is an important part of health-related quality of life, but is hard to measure due to the wide definition of roles and fluctuations in role participation. This study aims to explore the dimensionality of a newly developed item bank assessing the impact of health on RF.

METHODS

A battery of measures with skip patterns including the new RF bank was completed by 2,500 participants answering only questions on social roles relevant to them. Confirmatory factor analyses were conducted for the participants answering items from all conceptual domains (N = 1193). Conceptually based dimensionality and method effects reflecting positively and negatively worded items were explored in a series of models.

RESULTS

A bi-factor model (CFI = .93, RMSEA = .08) with one general and four conceptual factors (social, family, occupation, generic) was retained. Positively worded items were excluded from the final solution due to misfit. While a single factor model with methods factors had a poor fit (CFI = .88, RMSEA = .13), high loadings on the general factor in the bi-factor model suggest that the RF bank is sufficiently unidimensional for IRT analysis.

CONCLUSIONS

The bank demonstrated sufficient unidimensionality for IRT-based calibration of all the items on a common metric and development of a computerized adaptive test.

Validation of a computer-adaptive test to evaluate generic health-related quality of life

Background

Health Related Quality of Life (HRQoL) is a relevant variable in the evaluation of health outcomes. Questionnaires based on Classical Test Theory typically require a large number of items to evaluate HRQoL. Computer Adaptive Testing (CAT) can be used to reduce tests length while maintaining and, in some cases, improving accuracy. This study aimed at validating a CAT based on Item Response Theory (IRT) for evaluation of generic HRQoL: the CAT-Health instrument.

Methods

Cross-sectional study of subjects aged over 18 attending Primary Care Centres for any reason. CAT-Health was administered along with the SF-12 Health Survey. Age, gender and a checklist of chronic conditions were also collected. CAT-Health was evaluated considering: 1) feasibility: completion time and test length; 2) content range coverage, Item Exposure Rate (IER) and test precision; and 3) construct validity: differences in the CAT-Health scores according to clinical variables and correlations between both questionnaires.

Results

396 subjects answered CAT-Health and SF-12, 67.2% females, mean age (SD) 48.6 (17.7) years. 36.9% did not report any chronic condition. Median completion time for CAT-Health was 81 seconds (IQ range = 59-118) and it increased with age (p < 0.001). The median number of items administered was 8 (IQ range = 6-10). Neither ceiling nor floor effects were found for the score. None of the items in the pool had an IER of 100% and it was over 5% for 27.1% of the items. Test Information Function (TIF) peaked between levels -1 and 0 of HRQoL. Statistically significant differences were observed in the CAT-Health scores according to the number and type of conditions.

Conclusions

Although domain-specific CATs exist for various areas of HRQoL, CAT-Health is one of the first IRT-based CATs designed to evaluate generic HRQoL and it has proven feasible, valid and efficient, when administered to a broad sample of individuals attending primary care settings.

A New Stopping Rule for Computerized Adaptive Testing

The goal of the current study was to introduce a new stopping rule for computerized adaptive testing. The predicted standard error reduction stopping rule (PSER) uses the predictive posterior variance to determine the reduction in standard error that would result from the administration of additional items. The performance of the PSER was compared to that of the minimum standard error stopping rule and a modified version of the minimum information stopping rule in a series of simulated adaptive tests, drawn from a number of item pools. Results indicate that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant.

Development of a computerized adaptive test for assessing balance function in patients with stroke

BACKGROUND

An efficient and precise measure of balance is needed to improve administration efficiency and to reduce the assessment burden for patients.

OBJECTIVE

The purpose of this study was to develop a computerized adaptive testing (CAT) system for assessing balance function in an efficient, reliable, and valid fashion in patients with stroke.

DESIGN

Two cross-sectional prospective studies were conducted.

SETTING

This study was conducted in the departments of physical medicine and rehabilitation in 6 hospitals.

PATIENTS

The participants were inpatients and outpatients who were receiving rehabilitation.

MEASUREMENTS

A balance item pool (41 items) was developed on the basis of predefined balance concepts, expert opinions, and field testing. The items were administered by 5 raters to 764 patients. An item response theory model was fit to the data, and the item parameters were estimated. A simulation study was used to determine the performance (eg, reliability, efficiency) of the Balance CAT. The Balance CAT and the Berg Balance Scale (BBS) then were tested on another independent sample of 56 patients to determine the concurrent validity and time needed for administration.

RESULTS

Seven items did not meet the model's expectations and were excluded from further analysis. The remaining 34 items formed the item bank of the Balance CAT. Two stopping rules (ie, reliability coefficient > 0.9 or < or = 6 items) were set for the CAT. The simulation study showed that the patients' balance scores estimated by the CAT had an average reliability value of .94. The scores obtained from the CAT were closely associated with those of the full item set (Pearson r=.98). The scores of the Balance CAT were highly correlated with those of the BBS (Pearson r=.88). The average time needed to administer the Balance CAT (83 seconds) was only 18% of that of the BBS.

LIMITATIONS

The convenience sampling of both samples may limit the generalization of the results. Further psychometric investigation of the Balance CAT is needed.

CONCLUSION

The results provide strong evidence that the Balance CAT is efficient and has reliability and validity for patients with stroke.

Performance of common clinician- and self-report measures in assessing the function of community-dwelling people with metastatic breast cancer

Cheville AL, Basford JR, Troxel AB, Kornblith AB. Performance of common clinician- and self-report measures in assessing the function of community-dwelling people with metastatic breast cancer.

OBJECTIVE

To characterize the performance of common clinician- and self-report measures of function in assessing community-dwelling people with metastatic breast cancer.

DESIGN

Cross-sectional study.

SETTING

A tertiary medical center outpatient cancer clinic.

PARTICIPANTS

A consecutive sample of community-dwelling patients (N=163) with stage IV breast cancer.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Medical Outcomes Study 36-Item Short-Form Physical Functioning (PF-10) and Role Physical subscales; the Older Americans Resource Study (OARS) activities of daily living (ADL) and instrumental ADL subscales; Karnofsky Performance Scale (KPS); and the FIM Total and FIM Mobility scores.

RESULTS

With the exception of the PF-10 and Role Physical subscales, which demonstrated floor effects, ceiling effects were detected in all the measures and were particularly persistent in the OARS ADL subscale. Instrument and item score distributions varied markedly across KPS-defined subgroups with FIM Mobility, FIM Total, and OARS subscale score distributions deviating least from the normal in the lowest performing (KPS 40-50) participants. Correlations between self-reported (Role Physical subscales, PF-10, OARS ADL subscales) and the clinician-rated (KPS and FIM scales) scales were moderate to high (r=.55-.82); however, clinician-reported scores were more consistently associated with the presence of physical impairments.

CONCLUSIONS

In this population with stage IV breast cancer, ceiling effects limit the discriminatory capacity of the common functional scales assessed in this study. Instruments and items, particularly when ADL based, tend to perform better at lower levels of function (KPS 40-50) and less well at higher levels. Clinician-rated outcomes may have greater capacity to discriminate the presence of physical impairments.

Health and role functioning: the use of focus groups in the development of an item bank

BACKGROUND

Role functioning is an important part of health-related quality of life. However, assessment of role functioning is complicated by the wide definition of roles and by fluctuations in role participation across the life-span. The aim of this study is to explore variations in role functioning across the lifespan using qualitative approaches, to inform the development of a role functioning item bank and to pilot test sample items from the bank.

METHODS

Eight focus groups were conducted with a convenience sample of 38 English-speaking adults recruited in Rhode Island. Participants were stratified by gender and four age groups. Focus groups were taped, transcribed, and analyzed for thematic content.

RESULTS

Participants of all ages identified family roles as the most important. There was age variation in the importance of social life roles, with younger and older adults rating them as more important. Occupational roles were identified as important by younger and middle-aged participants. The potential of health problems to affect role participation was recognized. Participants found the sample items easy to understand, response options identical in meaning and preferred five response choices.

CONCLUSIONS

Participants identified key aspects of role functioning and provided insights on their perception of the impact of health on their role participation. These results will inform item bank generation.

Comparing the content of participation instruments using the international classification of functioning, disability and health

BACKGROUND

The concept of participation is recognized as an important rehabilitation outcome and instruments have been developed to measure participation using the International Classification of Functioning, Disability and Health (ICF). To date, few studies have examined the content of these instruments to determine how participation has been operationalized. The purpose of this study was to compare the content of participation instruments using the ICF classification.

METHODS

A systematic literature search was conducted to identify instruments that assess participation according to the ICF. Instruments were considered to assess participation and were included if the domains contain content from a minimum of three ICF chapters ranging from Chapter 3 Communication to Chapter 9 Community, social and civic life in the activities and participation component. The instrument content was examined by first identifying the meaningful concepts in each question and then linking these concepts to ICF categories. The content analysis included reporting the 1) ICF chapters (domains) covered in the activities and participation component, 2) relevance of the meaningful concepts to the activities and participation component and 3) context in which the activities and participation component categories are evaluated.

RESULTS

Eight instruments were included: Impact on Participation and Autonomy, Keele Assessment of Participation, Participation Survey/Mobility, Participation Measure-Post Acute Care, Participation Objective Participation Subjective, Participation Scale (P-Scale), Rating of Perceived Participation and World Health Organization Disability Assessment Schedule II (WHODAS II). 1351 meaningful concepts were identified in the eight instruments. There are differences among the instruments regarding how participation is operationalized. All the instruments cover six to eight of the nine chapters in the activities and participation component. The P-Scale and WHODAS II have questions which do not contain any meaningful concepts related to the activities and participation component. Differences were also observed in how other ICF components (body functions, environmental factors) and health are operationalized in the instruments.

CONCLUSION

Linking the meaningful concepts in the participation instruments to the ICF classification provided an objective and comprehensive method for analyzing the content. The content analysis revealed differences in how the concept of participation is operationalized and these differences should be considered when selecting an instrument.

Creating a computer adaptive test version of the late-life function and disability instrument

BACKGROUND

This study applied item response theory (IRT) and computer adaptive testing (CAT) methodologies to develop a prototype function and disability assessment instrument for use in aging research. Herein, we report on the development of the CAT version of the Late-Life Function and Disability Instrument (Late-Life FDI) and evaluate its psychometric properties.

METHODS

We used confirmatory factor analysis, IRT methods, validation, and computer simulation analyses of data collected from 671 older adults residing in residential care facilities. We compared accuracy, precision, and sensitivity to change of scores from CAT versions of two Late-Life FDI scales with scores from the fixed-form instrument. Score estimates from the prototype CAT versus the original instrument were compared in a sample of 40 older adults.

RESULTS

Distinct function and disability domains were identified within the Late-Life FDI item bank and used to construct two prototype CAT scales. Using retrospective data, scores from computer simulations of the prototype CAT scales were highly correlated with scores from the original instrument. The results of computer simulation, accuracy, precision, and sensitivity to change of the CATs closely approximated those of the fixed-form scales, especially for the 10- or 15-item CAT versions. In the prospective study, each CAT was administered in <3 minutes and CAT scores were highly correlated with scores generated from the original instrument.

CONCLUSIONS

CAT scores of the Late-Life FDI were highly comparable to those obtained from the full-length instrument with a small loss in accuracy, precision, and sensitivity to change.