Measuring change over time: assessing the usefulness of evaluative instruments

Family reported outcomes, an unmet need in the management of a patient's disease: appraisal of the literature

BACKGROUND

A person's chronic health condition or disability can have a huge impact on the quality of life (QoL) of the whole family, but this important impact is often ignored. This literature review aims to understand the impact of patients' disease on family members across all medical specialities, and appraise existing generic and disease-specific family quality of life (QoL) measures.

METHODS

The databases Medline, EMBASE, CINHAL, ASSIA, PsycINFO and Scopus were searched for original articles in English measuring the impact of health conditions on patients' family members/partner using a valid instrument.

RESULTS

Of 114 articles screened, 86 met the inclusion criteria. They explored the impact of a relative's disease on 14,661 family members, mostly 'parents' or 'mothers', using 50 different instruments across 18 specialities including neurology, oncology and dermatology, in 33 countries including the USA, China and Australia. These studies revealed a huge impact of patients' illness on family members. An appraisal of family QoL instruments identified 48 instruments, 42 disease/speciality specific and six generic measures. Five of the six generics are aimed at carers of children, people with disability or restricted to chronic disease. The only generic instrument that measures the impact of any condition on family members across all specialities is the Family Reported Outcome Measure (FROM-16). Although most instruments demonstrated good reliability and validity, only 11 reported responsiveness and only one reported the minimal clinically important difference.

CONCLUSIONS

Family members' QoL is greatly impacted by a relative's condition. To support family members, there is a need for a generic tool that offers flexibility and brevity for use in clinical settings across all areas of medicine. FROM-16 could be the tool of choice, provided its robustness is demonstrated with further validation of its psychometric properties.

Clinically Significant Changes in the 17- and 6-Item Hamilton Rating Scales for Depression: A STAR*D Report

OBJECTIVE

To develop clinically meaningful improvement thresholds in both the 17-item and the 6-item Hamilton Rating Scale for Depression (HRSD) total scores in depressed outpatients.

METHODS

The post-hoc analysis included all adult outpatients with non-psychotic major depressive disorder in the STAR*D trial who entered and exited the first treatment step (up to 14 weeks of citalopram) with a complete set of study measures at baseline and exit and at least one post-baseline measure. Within-patient change and linear regression anchor-based analyses were conducted to define meaningful and substantial changes in the HRSD17 and HRSD6 using three patient-reported outcomes [Work and Social Adjustment Scale (WSAS), Quality of Life Enjoyment and Satisfaction-Short Form (Q-LES-Q-SF); Mini-Q-LES-Q] obtained at baseline and exit from the first treatment step in STAR*D.

RESULTS

Linear regression analyses identified a meaningful change threshold for the HRSD17 as 3.9 [3.7-4.1] [lower, upper 95% CI] and a substantial change as 7.8 [7.4-8.3] with the WSAS. Analogous thresholds based on the Q-LES-Q-SF were 5.8 [5.5-6.1] and 11.6 [11.0-12.2], respectively, and 4.9 [4.7-5.2] and 9.9 [9.3-10.4] for the Mini-QLES-Q, respectively. For the HRSD6, linear regression analyses with the WSAS identified a meaningful change as 2.2 [2.1-2.4], while a substantial change was 4.5 [4.2-4.7]. Analogous figures based on the Q-LES-Q-SF were 3.2 [3.0-3.4] and 6.4 [6.1-6.8]. Similarly, based on the Mini-QLESQ, results were 2.8 [2.6-2.9] and 5.6 [5.3-5.9]. For both the HRSD17 and the HRSD6, within-patient analyses produced less precise estimates of the same change thresholds with substantial overlap between groups. Based on the WSAS, a clinically meaningful change in the HRSD17 total score was 9.6 (SD = 6.5), while a substantial change was 15.0 (SD = 6.7). Analogous change thresholds based on the Q-LESQ-SF were 12.9 (SD = 6.2) and 16.8 (SD = 6.4), respectively. For the Mini-Q-LES-Q, thresholds were 10.9 (SD = 6.5) and 16.1 (SD = 6.2).

CONCLUSION

A 4-6 point change in the HRSD17 is clinically meaningful; a 7-12 point change is clinically substantial. For the HRSD6, analogous estimates were 2-3 and 4-7 point changes, respectively.

The international trauma questionnaire (ITQ) measures reliable and clinically significant treatment-related change in PTSD and complex PTSD

Background: The International Trauma Questionnaire (ITQ) is a validated measure that assesses ICD-11 posttraumatic stress disorder (PTSD) and complex PTSD (CPTSD). An important task is to determine whether the ITQ is an appropriate evaluative measure for clinical trials. Objective: To assess the psychometric properties of the ITQ in the context of treatment and determine if the ITQ measures reliable and clinically significant change over the course of a psychosocial intervention. Method: Analyses were based on data from an online skills training programme delivered to 254 U.S. Veterans. Reliability and validity of the ITQ scores were assessed at baseline. Changes in symptom scores and probable diagnostic rates were compared at pre-, mid- and post-treatment. A reliable change index (RCI) score was computed to classify participants as improved, unchanged, or worsened. The PCL-5 was used as a comparison measure. Results: Baseline concurrent and factorial validity was similar to previous studies. Internal consistency at each assessment was excellent and comparable to the PCL-5. Decline in symptoms from pre-to-post-treatment was significant for PTSD and CPTSD symptom profiles. Rate of probable disorder (PTSD or CPTSD) declined significantly from pre-treatment to post-treatment. Pre-to-post treatment declines exceeded the critical RCI values for the ITQ. Clinically significant changes were observed where most participants improved, some stayed the same, and few worsened. The performance of the ITQ was consistent with the PCL-5 regarding sensitivity to change. Conclusion: This study provides the first demonstration that the ITQ measures reliable and clinically significant treatment-related change of ICD-11 PTSD and CPTSD symptoms.

Agitation in Alzheimer's disease: Novel outcome measures reflecting the International Psychogeriatric Association (IPA) agitation criteria

Introduction

The 2017 European Union‐North American Clinical Trials in Alzheimer's Disease Task Force recommended development of clinician‐rated primary outcome measures for Alzheimer's disease (AD) agitation trials, incorporating International Psychogeriatric Association (IPA) criteria.

Methods

In a modified Delphi process, Cohen‐Mansfield Agitation Inventory (CMAI) and Neuropsychiatric Inventory‐Clinician (NPI‐C) items were mapped to IPA agitation domains generating novel instruments, CMAI‐IPA and NPI‐C‐IPA. Validation in the Agitation and Aggression AD Cohort (A3C) assessed minimal clinically important differences (MCIDs), change sensitivity, and predictive validity.

Results

MCID was –17 (odds ratio [OR] = 14.9, 95% confidence interval [CI] = 6.8–32.6) for CMAI; –5 (OR = 9.3, 95% CI = 4.0–21.2) for CMAI‐IPA; –3 (OR = 11.9, 95% CI = 4.1–34.8) for NPI‐C‐A+A; and –5 (OR = 7.8, 95% CI = 3.4–17.9) for NPI‐C‐IPA at 3 months. Areas under the curve suggested no scale better predicted global clinician ratings. Sensitivity to change for all measures was high.

Conclusion

Internal consistency and reliability analyses demonstrated better accuracy for the NPI‐C‐IPA than for the CMAI‐IPA and can be used for agitation clinical trial inclusion, and for response to intervention.

Real-life cost-effectiveness of benralizumab in patients with severe asthma

BACKGROUND

Availability of clinically effective and cost-effective treatments for severe asthma would be beneficial to patients and national healthcare systems. The aim of this study was to evaluate clinical outcomes and healthcare expenditure after incorporating benralizumab into the standard treatment of refractory eosinophilic asthma.

METHODS

This was a cross-sectional multicentre study of consecutive patients with refractory eosinophilic asthma who received treatment with benralizumab during at least 12 months. Patient follow-up was performed in specialised severe asthma units. The main effectiveness parameters measured were: the avoidance of one asthma exacerbation, a 3-point increase in the asthma control test (ACT) score, and the difference in utility scores (health-related quality of life) between a 1-year baseline treatment and 1-year benralizumab treatment. The health economic evaluation included direct costs and incremental cost-effectiveness ratios (ICERs).

RESULTS

After 1 year of treatment with benralizumab, patients with refractory eosinophilic asthma showed an improvement in all the effectiveness parameters analysed: improvement of asthma control and lung function, and decrease in the number of exacerbations, oral corticosteroid (both as corticosteroid courses and maintenance therapy), and inhaled corticosteroid use. The total annual cost per patient for the baseline and benralizumab treatment periods were €11,544 and €14,043, respectively, reflecting an increase in costs due to the price of the biological agent but a decrease in costs for the remaining parameters. The ICER was €602 per avoided exacerbation and €983.86 for every 3-point increase in the ACT score.

CONCLUSIONS

All the pharmacoeconomic parameters analysed show that treatment with benralizumab is a cost-effective option as an add-on therapy in patients with refractory eosinophilic asthma.

Mind the methods of determining minimal important differences: three critical issues to consider

OBJECTIVE

Clinical trialists, meta-analysts and clinical guideline developers are increasingly using minimal important differences (MIDs) to enhance the interpretability of patient-reported outcome measures (PROMs). Here, we elucidate three critical issues of which MID users should be aware. Improved understanding of MID concepts and awareness of common pitfalls in methodology and reporting will better inform the application of MIDs in clinical research and decision-making.

METHODS

We conducted a systematic review to inform the development of an inventory of anchor-based MID estimates for PROMs. We searched four electronic databases to identify primary studies empirically calculating an anchor-based MID estimate for any PROM in adolescent or adult populations across all clinical areas. Our findings are based on information from 338 studies reporting 3389 MIDs for 358 PROMs published between 1989 and 2015.

RESULTS

We identified three key issues in the MID literature that demand attention. (1) The profusion of terms representing the MID concept adds unnecessary complexity to users' task in identifying relevant MIDs, requiring meticulous inspection of methodology to ensure estimates offered truly reflect the MID. (2) A multitude of diverse methods for MID estimation that will yield different estimates exist, and whether there are superior options remains unresolved. (3) There are serious issues of incomplete presentation and reporting of key aspects of the design, methodology and results of studies providing anchor-based MIDs, which threatens the optimal use of these estimates for interpretation of intervention effects on PROMs.

CONCLUSIONS

Although the MID represents a powerful tool for enhancing the interpretability of PROMs, realising its full value will require improved understanding and reporting of its measurement fundamentals.

TX-001HR is associated with a clinically meaningful effect on severity of moderate to severe vasomotor symptoms in the REPLENISH trial

Objective:

The aim of the study was to evaluate the clinically meaningful effect of oral TX-001HR (17β-estradiol [E2]/progesterone [P4]) capsules on hot flushes severity (vasomotor symptoms [VMS] severity scale) using the patient-reported Clinical Global Impression (CGI).

Methods:

REPLENISH (NCT01942668) was a phase 3, randomized, double-blind, placebo-controlled, multicenter trial that evaluated TX-001HR in postmenopausal women (40-65 y) with a uterus. Those with frequent moderate to severe hot flushes (≥7/d or ≥50/wk) were randomized in a VMS substudy to daily E2/P4 (1/100, 0.5/100, 0.5/50, or 0.25/50 mg/mg), or placebo. Patients rated VMS severity from 1 (mild) to 3 (severe) and symptom improvements with the CGI. CGI results were an anchor in a nonparametric discriminant analysis to define clinically important differences (CIDs) and minimal CID in VMS severity at weeks 4 and 12.

Results:

In the VMS substudy (n = 726), determined CID and minimal CID severity thresholds were reductions of 0.525 and 0.350 points at week 4, respectively, and 0.775 and 0.225 points at week 12. Significantly more women taking the two highest E2/P4 doses (1/100 and 0.5/100) versus placebo met CID severity thresholds at weeks 4 (40% and 44% vs 17%; P < 0.05) and 12 (56% and 48% vs 29%; P < 0.05).

Conclusion:

REPLENISH trial data demonstrated that E2/P4 1/100 and 0.5/100 provided clinically meaningful improvements in hot flushes severity in postmenopausal women. In conjunction with previously demonstrated clinically meaningful VMS frequency improvements, these data support oral E2/P4 1/100 and 0.5/100 for postmenopausal women with a uterus seeking treatment for moderate to severe VMS.

Health-Related Quality of Life With Trastuzumab Monotherapy Versus Trastuzumab Plus Standard Chemotherapy as Adjuvant Therapy in Older Patients With HER2-Positive Breast Cancer

PURPOSE

We report findings on quality of life (QoL) in the RESPECT trial, which compared adjuvant trastuzumab monotherapy with trastuzumab plus chemotherapy in older patients with human epidermal growth factor receptor 2 (HER2)-positive breast cancer (BC).

PATIENTS AND METHODS

Patients age 70-80 years with human epidermal growth factor receptor 2-positive surgically treated breast cancer were randomly assigned to receive trastuzumab (T) or trastuzumab plus chemotherapy (T + C). QoL was assessed using the Functional Assessment of Cancer Therapy-General (FACT-G), Philadelphia Geriatric Center Morale Scale, Hospital Anxiety and Depression Scale, Patient Neurotoxicity Questionnaire, and Tokyo Metropolitan Institute of Gerontology Index of Competence at baseline and after 2, 12, and 36 months. Comparisons were based on individual changes from baseline and were performed by Fisher's test or mixed-model repeated-measures.

RESULTS

Among 275 patients in the parent study, 231 (84%) (average age: 74 years) were included in the analysis. At 2, 12, and 36 months, 198, 177, and 178 patients completed surveys, and the mean FACT-G scores at each survey point were 78.9, 80.4, 82.7, and 79.1 in group T and 79.5, 74.5, 78.4, and 78.5 in group T + C. Compared with group T + C, the proportion of patients showing QoL deterioration (≥ 5 points decrease from baseline in FACT-G) was significantly lower at 2 months (31% v 48%; P = .016) and 12 months (19% v 38%; P = .009). In group T, the Hospital Anxiety and Depression Scale score (P = .003) and the proportion of severe sensory peripheral neuropathy (P = .001) were significantly lower at 2 months, and Philadelphia Geriatric Center Morale Scale and Tokyo Metropolitan Institute of Gerontology Index of Competence scores were significantly higher (P = .024, .042) at 12 months. At 36 months, there were no significant differences in any QoL items.

CONCLUSION

Detrimental effects of adjuvant chemotherapy on global QoL, morale, and activity capacity lasted for at least 12 months but were not observed at 36 months.

Comparison of reliability, construct validity and responsiveness of the IPAQ-SF and PASE in adults with osteoarthritis

BACKGROUND

This study assessed the measurement properties of two commonly used self-report physical activity (PA) measures: the International Physical Activity Questionnaire-Short Form (IPAQ-SF) and the Physical Activity Scale for the elderly (PASE) in adults with osteoarthritis.

METHODS

Secondary analysis of the MOSAICS cluster randomised controlled trial baseline and 3-month follow-up questionnaires, total scores and subdomains of the IPAQ-SF and PASE were compared. Intra-class correlations (ICC) were used to assess test-retest reliability, measurement error was assessed using standard error of measurement (SEM), smallest detectable change (SDC) and 95% limits of agreement (LoA). Responsiveness was assessed using effect size (ES), standard responsive measurement (SRM) and response ratio (RR).

RESULTS

There was moderate correlation (r = 0.56) between the total IPAQ-SF scores (score ranges 0-16,398) and the total PASE scores (score ranges 0-400). Subdomain correlations were also moderate (ranges 0.39-0.57). The PASE showed greater reliability compared to the IPAQ-SF (ICC = 0.68; 0.61-0.74 95% CI and ICC = 0.64; 0.55-0.72, respectively). Measurement errors in both measures were large: PASE SEM = 46.7, SDC = 129.6 and 95% LoA ranges = -117 to 136, the IPAQ-SF SEM = 3532.2 METS^{-1 min-1 week} , SDC = 9790.8 and 95% LoA ranges = -5222 to 5597. Responsiveness was poor: ES -0.14 and -0.16, SRM -0.21 and -0.21, and RR 0.12 and 0.09 for the IPAQ-SF and PASE, respectively.

DISCUSSION

The IPAQ-SF and PASE appear limited in reliability, measurement error and responsiveness. Researchers and clinicians should be aware of these limitations, particularly when comparing different levels of PA and monitoring PA levels changes over time in those with osteoarthritis.

Identifying the minimal important difference in patient-reported outcome measures in the field of people with severe mental illness: a pre-post-analysis of the Illness Management and Recovery Programme

Purpose

Complementary interventions for persons with severe mental illness (SMI) focus on both personal recovery and illness self-management. This paper aimed to identify the patient-reported outcome measures (PROMs) associated with the most relevant and meaningful change in persons with SMI who attended the Illness Management and Recovery Programme (IMR).

Methods

The effect of the IMR was measured with PROMs concerning recovery, illness self-management, burden of symptoms and quality of life (QoL). From the QoL measures, an anchor was chosen based on the most statistically significant correlations with the PROMs. Then, we estimated the minimal important difference (MID) for all PROMs using an anchor-based method supported by distribution-based methods. The PROM with the highest outcome for effect score divided by MID (the effect/MID index) was considered to be a measure of the most relevant and meaningful change.

Results

All PROMs showed significant pre–post-effects. The QoL measure ‘General Health Perception (Rand-GHP)’ was identified as the anchor. Based on the anchor method, the Mental Health Recovery Measure (MHRM) showed the highest effect/MID index, which was supported by the distribution-based methods. Because of the modifying gender covariate, we stratified the MID calculations. In most MIDs, the MHRM showed the highest effect/MID indexes.

Conclusion

Taking into account the low sample size and the gender covariate, we conclude that the MHRM was capable of showing the most relevant and meaningful change as a result of the IMR in persons with SMI.

Evaluation of Clinically Meaningful Changes in Measures of Frailty

Background

To determine the clinically meaningful changes and responsiveness of widely used frailty measures.

Methods

We analyzed data from a prospective cohort study of 1,135 community-dwelling older adults who underwent assessments of frailty and health-related quality of life using the EuroQol-5D at baseline and 1 year later. Frailty measures included deficit-accumulation frailty index (FI); frailty phenotype; Fatigue, Resistance, Ambulation, Illness, and Loss of Weight scale; and the Study of Osteoporotic Fracture (SOF) index. We determined the clinically meaningful changes by the distribution-based method and the anchor-based method using the EuroQol-5D score and responsiveness indices.

Results

Frailty measures were available in 925 participants at 1 year (81.5%). Based on the distribution-based method, small and large clinically meaningful changes were 0.019 and 0.057 for FI, 0.249 and 0.623 for frailty phenotype, 0.235 and 0.587 for FRAIL scale, and 0.116 and 0.289 for SOF index, respectively. The anchor-based estimates of small and large changes were 0.028 and 0.076 for FI, 0.097 and 0.607 for frailty phenotype, 0.269 and 0.368 for FRAIL scale, and 0.023 and 0.287 for SOF index, respectively. Based on the responsiveness index, per-group sample sizes to achieve 80% power in clinical trials, ranged from 51 (FI) to 7,272 (SOF index) for a small change and 9 (FI) to 133 (FRAIL scale) for a large change.

Conclusions

The estimates of clinically meaningful change of frailty measures can inform the choice of frailty measures to track longitudinal changes of frailty in clinical trials and clinical care of community-dwelling older adults.

How Replicates Can Inform Potential Users of a Measurement Procedure about Measurement Error: Basic Concepts and Methods

Measurement procedures are not error-free. Potential users of a measurement procedure need to know the expected magnitude of the measurement error in order to justify its use, in particular in health care settings. Gold standard procedures providing exact measurements for comparisons are often lacking. Consequently, scientific investigations of the measurement error are often based on using replicates. However, a standardized terminology (and partially also methodology) for such investigations is lacking. In this paper, we explain the basic conceptual approach of such investigations with minimal reference to existing terminology and describe the link to the existing general statistical methodology. This way, some of the key measures used in such investigations can be explained in a simple manner and some light can be shed on existing terminology. We encourage clearly conceptually distinguishing between investigations of the measurement error of a single measurement procedure and the comparison between different measurement procedures or observers. We also identify an unused potential for more advanced statistical analyses in scientific investigations of the measurement error.

Assessing responsiveness of the EQ-5D-3L, the Oxford Hip Score, and the Oxford Knee Score in the NHS patient-reported outcome measures

BACKGROUND

The degree to which a validated instrument is able to detect clinically significant change over time is an important issue for the better management of hip or knee replacement surgery. This study examines the internal responsiveness of the EQ-5D-3L, the Oxford Hip Score (OHS), and the Oxford Knee Score (OKS) by various methods. Data from NHS patient-reported outcome measures (PROMs) linked to the Hospital Episodes Statistics (HES) dataset (2009-2015) was analysed for patients who underwent primary hip surgery (N = 181,424) and primary knee surgery (N = 191,379).

METHODS

Paired data-specific univariate responsiveness was investigated using the standardized response mean (SRM), the standardized effect size (SES), and the responsiveness index (RI). Multivariate responsiveness was furthermore examined using the defined capacity of benefit score (i.e. paired data-specific MCID), adjusting baseline covariates such as age, gender, and comorbidities in the Box-Cox regression models. The observed and predicted percentages of patient improvement were examined both as a whole and by the patients' self-assessed transition level.

RESULTS

The results showed that both the OHS and the OKS demonstrated great univariate and multivariate responsiveness. The percentages of the observed (predicted) total improvement were high: 51 (54)% in the OHS and 73 (58)% in OKS. The OHS and the OKS showed distinctive differences in improvement by the 3-level transition, i.e. a little better vs. about the same vs. a little worse. The univariate responsiveness of the EQ-5D-3L showed moderate effects in total by Cohen's thresholds. The percentages of improvement in the EQ-5D-3L were moderate: 44 (48)% in the hip and 42 (44)% for the knee replacement population.

CONCLUSIONS

Distinctive percentage differences in patients' perception of improvement were observed when the paired data-specific capacity of benefit score was applied to examine responsiveness. This is useful in clinical practice as rationale for access to surgery at the individual-patient level. This study shows the importance of analytic methods and instruments for investigation of the health status in hip and/or knee replacement surgery. The study finding also supports the idea of using a generic measure along with the disease-specific instruments in terms of cross-validation.

Test-retest reliability on the Cambridge Neuropsychological Test Automated Battery: Comment on Karlsen et al. (2020)

Test-retest reliability is essential to the development and validation of psychometric tools. Here we respond to the article by Karlsen et al. (Applied Neuropsychology: Adult, 2020), reporting test-retest reliability on the Cambridge Neuropsychological Test Automated Battery (CANTAB), with results that are in keeping with prior research on CANTAB and the broader cognitive assessment literature. However, after adopting a high threshold for adequate test-retest reliability, the authors report inadequate reliability for many measures. In this commentary we provide examples of stable, trait-like constructs which we would expect to remain highly consistent across longer time periods, and contrast these with measures which show acute within-subject change in response to contextual or psychological factors. Measures characterized by greater true within-subject variability typically have lower test-retest reliability, requiring adequate powering in research examining group differences and longitudinal change. However, these measures remain sensitive to important clinical and functional outcomes. Setting arbitrarily elevated test-retest reliability thresholds for test adoption in cognitive research limits the pool of available tools and precludes the adoption of many well-established tests showing consistent contextual, diagnostic, and treatment sensitivity. Overall, test-retest reliability must be balanced with other theoretical and practical considerations in study design, including test relevance and sensitivity.

Optimisation of Scores Generated by an Online Feline Health-Related Quality of Life (HRQL) Instrument to Assist the Veterinary User Interpret Its Results

Using methodology previously described for the dog health-related quality of life (HRQL) tool (VetMetrica™), the aim was to optimize the scores profile of a comparable feline online HRQL instrument for monitoring HRQL in cats, to assist in its interpretation. Measuring HRQL helps quantify the impact of disease and its treatment on well-being, aids clinical decision making and provides information in clinical trials. In Study 1, using data collected from previous studies, scores generated for three domains of HRQL (Vitality, Comfort, Emotional Well-being) in healthy cats were normalized using standard statistical techniques of logit transformation and T-scores, such that the average healthy cat has a score of 50 in all three HRQL domains. Using normalized scores from healthy and sick cats, a threshold score of 44.8 was determined, above which 70% of healthy cats should score. Study 2 determined the Minimal Important Difference (MID) in normalized score that constituted a clinically significant improvement in each domain. Three methods were tested in order to determine the MID, with the final choice made based on statistical and clinical considerations. Thresholds of 5, 7.5, and 5 were chosen for the three HRQL domains representing Vitality, Comfort and Emotional Well-being, respectively. This study makes available a means of displaying HRQL scores from an online application in an easily interpretable manner and quantifies a clinically meaningful improvement in score. To illustrate the practical application of these developments, three case examples are presented. Example 1 illustrates the raw and normalized scores for a group of overweight cats enrolled in a Feline Weight Management Programme. Example 2 shows three groups of osteoarthritic cats, each with different severity of disease. The third is an elderly, un-well cat whose HRQL was recorded over time, specifically to facilitate end of life discussion between owner and veterinary clinician.

Assessing the experience of the quality of care of patients living with multiple sclerosis and their caregivers: The MusiCare questionnaire

BACKGROUND AND PURPOSE

Patients with a chronic illness, such as multiple sclerosis (MS), and their natural caregivers have a specific experience of healthcare and health services. These experiences need to be assessed to evaluate the quality of care. Our objective was to develop a French-language questionnaire to evaluate the quality of care as experienced by MS patients and their natural caregivers.

METHODS

Eligible patients had been diagnosed with MS according to the McDonald criteria. Eligible caregivers were individuals designated by the patients. The MusiCare questionnaire was developed in two standard phases: (i) item generation, based on interviews with patients and caregivers; and (ii) validation, consisting of validity, reliability, external validity, reproducibility, and responsiveness measures.

RESULTS

In total, 1088 patients (n = 660) and caregivers (n = 488) were recruited. The initial 64-item version of MusiCare was administered to a random subsample (n = 748). The validation process generated a 35-item questionnaire. Internal consistency and scalability were satisfactory. Testing of the external validity revealed expected associations between MusiCare scores and sociodemographic and clinical data. The questionnaire showed good reproducibility and responsiveness.

CONCLUSIONS

The availability of a reliable and validated French-language self-report questionnaire probing the experience of the quality of care for MS will allow the feedback of patients and caregivers to be incorporated into a continuous healthcare quality-improvement strategy.

Between-group minimally important change versus individual treatment responders

PURPOSE

Estimates of the minimally important change (MIC) can be used to evaluate whether group-level differences are large enough to be important. But responders to treatment have been based upon group-level MIC thresholds, resulting in inaccurate classification of change over time. This article reviews options and provides suggestions about individual-level statistics to assess whether individuals have improved, stayed the same, or declined.

METHODS

Review of MIC estimation and an example of misapplication of MIC group-level estimates to assess individual change. Secondary data analysis to show how perceptions about meaningful change can be used along with significance of individual change.

RESULTS

MIC thresholds yield over-optimistic conclusions about responders to treatment because they classify those who have not changed as responders.

CONCLUSIONS

Future studies need to evaluate the significance of individual change using appropriate individual-level statistics such as the reliable change index or the equivalent coefficient of repeatability. Supplementing individual statistical significance with retrospective assessments of change is desirable.

Patient-reported Outcomes (PROs) in clinical trials in paediatric dentistry

Patient-reported outcomes (PROs) are reports directly from patients without interpretation by clinicians or others and captured using validated patient-reported outcome measures (PROMs). These measures are increasingly employed in clinical practice and can be incorporated into clinical trials. Benefits of using PROs include reducing observer bias; eliciting unique views on aspects important to patients and increasing public accountability. Despite inclusion in clinical trials PRO data is often under-reported and the results may not be adopted into clinical practice due to concerns about the data generated. This review discusses what PROs are and how to measure them; the benefits of using PROs; how to choose an appropriate PROM to answer the research question; considerations for using PROs in paediatric dentistry and reporting guidelines. Finally, some examples of how PROs have been included in paediatric dentistry trials are given along with discussion of the development of core outcome sets and how these may improve reporting of PROs in the future.

Identifying Minimal and Meaningful Change in PROMIS® for Rheumatoid Arthritis: Use of Multiple Methods and Perspectives

BACKGROUND

Rheumatoid arthritis (RA) is chronic, painful, disabling condition resulting in significant impairments in physical, emotional, and social health. We used different methods and perspectives to evaluate the responsiveness of PROMIS® short forms (SFs) and identify minimal and meaningful score changes.

METHODS

Adults with RA enrolled in a multi-site prospective observational cohort completed PROMIS Physical Function, Pain Interference, Fatigue, Participation in Social Roles/Activities SFs, PROMIS-29, and pain, patient global, and rated change in specific symptoms and RA (a little vs. lot better or worse) at the second visit. Physicians recorded joint counts, MD Global Assessment, and change in RA at visit 2. We compared mean score differences for minimal and meaningful improvement/worsening using patient and MD change ratings and distribution-based methods, and visually inspected empirical cumulative distribution function curves by change categories.

RESULTS

The 348 adults were mostly (81%) female with longstanding RA. Using patient ratings, generally 1-3 point differences were observed for minimal change and 3-7 points for meaningful change. Larger differences were observed with patient vs. physician ratings and for symptom-specific vs. RA change. Mean differences were similar among SF versions. Prespecified hypotheses about change in PROMIS Physical Function, Pain Interference, Fatigue and Participation and legacy scales were supported.

CONCLUSIONS

PROMIS SFs and the PROMIS-29 Profile are responsive to change and generally distinguish between minimal and meaningful improvement and worsening in key RA domains. These data add to a growing body of evidence demonstrating robust psychometric properties of PROMIS and supporting use in RA care, research, and decision-making.

Responsiveness, minimal important difference, minimal relevant difference, and optimal number of patients for a study

Several terms are used to describe changes in PROM scores in relation to treatments. Whether the change is small, large, or relevant is defined in different ways, yet these change scores are used to recommend or oppose treatments. They are also used to calculate the necessary number of patients for a study. This article offers a theoretical explanation behind the terms responsiveness, minimal important difference (MID), minimal important change (MIC), minimal relevant difference (MIREDIF), and threshold of clinical importance. It also gives instructions on how these and the optimal number of patients for a study are calculated. Responses to two domains of the Knee Injury and Osteoarthritis Outcome Score (KOOS), before and 1 year after reconstruction of the anterior cruciate ligament of 164 patients, are used to illustrate the calculations. This paper presents the most common methods used to calculate and interpret MID. Results vary substantially across domains, patient location on the scale, and health conditions. The optimal number of patients depends on the minimal relevant difference (MIREDIF), the standard error of the measure (SEM), the desired statistical power for the measurement, and the responsiveness of the measurement instrument (the PROM). There is often uncertainty surrounding the calculation and interpretation of responsiveness, MID, and MIREDIF, as these concepts are complex. When MID is used to evaluate research results, authors should specify how the MID was calculated, and its relevance for the study population. These measures should only be used after thorough consideration to justify healthcare decisions.

Measuring Change in the Communication Skills of Children With Austim Spectrum Disorder Using the Communication Complexity Scale

Changes in minimal verbal communication by children with autism spectrum disorder (ASD) were measured with the Communication Complexity Scale (CCS) and other communication assessments. The CCS measures complexity of preverbal and beginning verbal communication used to communicate behavior regulation and joint attention. The purpose was to investigate if the CCS was responsive to changes associated with a behavioral intervention aimed at improving communication skills. Changes were detected with CCS scores, rates of initiating joint attention, and the Mullen Scales of Early Learning (MSEL) Expressive Language subscale. Significant changes in CCS scores were also detected for a subgroup of participants who did not show significant changes on the MSEL Expressive Language subscale, demonstrating that CCS scores are sensitive to changes associated with a behavioral intervention.

Validity and reliability of the Korean version of the Quality of Recovery-15 questionnaire

Background

The quality of recovery-40 questionnaire (QoR-40) has been widely used to assess quality of recovery after surgery, but it is too lengthy for clinical use. The short form of QoR-40, QoR-15, has been validated in many languages; however, an official Korean version of the QoR-15 (QoR-15K) has not yet been established. This study aimed to develop and validate QoR-15K.

Methods

Based on the previously-validated Korean QoR-40, we selected 15 items; the QoR-15K was patterned on the original QoR-15. We analyzed 210 subjects who had been scheduled for elective surgery under general anesthesia. The patients completed the questionnaire before surgery and on postoperative days one and two. The validity, reliability, and responsiveness of the QoR-15K were evaluated.

Results

We obtained excellent convergent validity on visual analog scale for recovery (ρ = 0.882, P < 0.001). The duration of anesthesia, post-anesthesia care unit, and overall hospital stay with the QoR-15K showed a significant negative correlation (ρ = −0.183, −0.151, and −0.185, respectively). Cronbach’s α was 0.909. Cohen’s effect size and standardized response mean were 0.819 and 0.721. The recruitment and completion rate were 92.9% and 100%, respectively. We based the above calculations on the results obtained on the first day following surgery.

Conclusions

The validity and reliability of the QoR-15K are comparable to those of the English version. The QoR-15K would be a good instrument to assess the quality of recovery in Korean patients after surgery.

Minimum Clinically Important Differences of the Hospital for Special Surgery Dysphagia and Dysphonia Inventory and Other Dysphagia Measurements in Patients Undergoing ACDF

BACKGROUND

Postoperative dysphagia is a common complication after anterior cervical surgery, and it can be measured using patient-reported outcome measures (PROMs). The Hospital for Special Surgery Dysphagia and Dysphonia Inventory (HSS-DDI) is a condition-specific PROM to evaluate dysphagia and dysphonia after anterior cervical discectomy and fusion (ACDF). The minimum clinically important difference (MCID) of the HSS-DDI has not, to our knowledge, been established. Other PROMs have been used to assess dysphagia (SWAL-QOL and MD Anderson Dysphagia Inventory [MDADI]) in ACDF. Currently, few studies have addressed the MCIDs of these PROMs.

QUESTIONS/PURPOSES

To determine (1) the minimum detectable changes (MDC) of the HSS-DDI, SWAL-QOL, and MDADI using a distribution-based approach, and (2) the MCID of the HSS-DDI, SWAL-QOL, and MDADI, using an anchor-based approach.

METHODS

We used a longitudinally maintained database that was originally established for the HSS-DDI development and validation study. In all, 323 patients who underwent elective ACDF were assessed for enrollment eligibility; 83% (268 of 323) met the inclusion criteria and completed the HSS-DDI Week 4 survey. We set six outcomes: distribution-based MDCs for the (1) HSS-DDI, (2) SWAL-QOL, (3) MDADI, in addition to anchor-based MCIDs for the (4) HSS-DDI, (5) SWAL-QOL, and (6) MDADI. The HSS-DDI consists of 31 questions and ranges 0 (worst) to 100 (normal). We used the focused SWAL-QOL, which consists of 14 selected items from the original SWAL-QOL and ranges from 0 (worst) to 100 (normal). The MDADI is a 20-item survey and ranges from 20 (worst) to 100 (normal). A distribution-based approach is used to calculate values defined as the smallest difference above the measurement error. An anchor-based approach is used to determine the MCIDs based on an external scale, called an anchor, which indicates the minimal symptom change that is considered clinically important. All 268 patients were used for the distribution-based (0.5 SD) HSS-DDI MDC analysis. The first 16% (44 of 268) of patients completed retesting of the HSS-DDI via a telephone interview and were used for another distribution-based (standard error of measurement: SEM) MDC analysis. The number of patients for the test-retest group was determined based on the previously reported minimum required sample size of reliability studies. The first 63% (169 of 268, SWAL-QOL and 168 of 268, MDADI) of patients completed two other surveys for the external validation of the HSS-DDI, and were used for the SWAL-QOL and MDADI 0.5 SD analyses. Among the patients, 86% (230 of 268) completed the Week 8 HSS-DDI survey that was used for the anchor-based HSS-DDI MCID analysis, and 56% (SWAL-QOL, 150 of 268 and MDADI, 151 of 268) of patients completed the Week 8 surveys that were used for the SWAL-QOL and MDADI MCID analyses. Subjective improvement grades from the previous assessment were used as the anchor. The MCIDs were calculated as the mean score changes among those who reported little better or greater in the improvement assessment and receiver operating characteristic (ROC) curve analyses. We adopted the higher value of these two as the MCID for each PROM.

RESULTS

The distribution-based MDCs for the HSS-DDI total score, SWAL-QOL, and MDADI were 11 of 100, 9 of 100, and 8 of 80 points, respectively, using the 0.5 SD method. Using the SEM-based method, the MDC for the HSS-DDI total score was 9 of 100 points. Regarding the anchor-based MCIDs, the values calculated with the mean score change method were consistently higher than those of ROC analysis and were adopted as the MCIDs. The MCIDs were 10 for the total HSS-DDI total score, 8 for the SWAL-QOL, and 6 for the MDADI.

CONCLUSIONS

Improvements of less than 10 points for the HSS-DDI score, 9 points for the SWAL-QOL, and 6 points for the MDADI are unlikely to be perceived by patients to be clinically important. Future studies on dysphagia after anterior cervical surgery should report between-group differences in light of this, rather than focusing on p values and statistical significance.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Reliability of the Quality of Life Inventory-Disability Measure in Children with Intellectual Disability

OBJECTIVE

To assess responsiveness and reproducibility using the estimates of test-retest reliability for the Quality of Life Inventory-Disability (QI-Disability), accounting for changes in child health and parental stress.

METHOD

Quality of Life Inventory-Disability was administered twice over a 1-month period to a sample of 55 primary caregivers of children (aged 5-19 years) with intellectual disability. Caregivers also reported their child's physical and mental health and completed a 4-item Perceived Stress Scale to assess parental stress. Fixed-effects linear regression models examined responsiveness of QI-Disability to the reported change in child health and parental stress. Reliability was then assessed using intraclass correlations (ICCs) calculated from QI-Disability scores adjusted for changes in child health and parental stress.

RESULTS

Five of 7 unadjusted ICC values indicated at least moderate agreement (>0.70), and 2 values indicated fair agreement. After accounting for changes in child health and parental stress, adjusted ICC values showed substantial agreement for the total QI-Disability score and 4 domain scores (adjusted ICC ≥ 0.80). Adjusted ICC scores indicated moderate agreement for the Physical Health domain (adjusted ICC = 0.68) and fair agreement for the Positive Emotions domain (adjusted ICC = 0.58). Improvements in a child's physical health rating were associated with higher total, Physical Health, and Positive Emotion domain scores, whereas improvements in mental health were associated with higher total and Negative Emotions domain scores, indicating better quality of life. Changes in parental stress did not have a statistically significant relationship with quality of life.

CONCLUSION

Satisfactory test-retest reliability was shown. Preliminary evidence indicates that QI-Disability is responsive to changes in child health, but not to differing levels of parental stress.

Reliability and responsiveness of virtual portion size creation tasks: Influences of context, foods, and a bariatric surgical procedure

Food portion size influences energy intake and sustained high-energy intake often leads to obesity. Virtual portion creation tasks (VPCTs), in which a participant creates portions of food on a computer screen, predict intake in healthy individuals. The objective of this study was to determine whether portions created in VPCTs are stable over time (test-retest reliability) and responsive to factors known to influence food intake, such as eating contexts and food types, and to determine if virtual portions can predict weight loss. Patients with obesity scheduled for bariatric surgery (n = 29), and individuals with a normal BMI (18.5-24.9 kg/m2, controls, n = 29), were instructed to create virtual portions of eight snack foods, which varied in energy density (low and high) and taste (sweet and salty). Portions were created in response to the following eating situations, or "contexts": What they would a) eat to stay healthy (healthy), b) typically eat (typical), c) eat to feel comfortably satisfied (satisfied), d) consider the most that they could tolerate eating (maximum), and e) eat if nothing was limiting them (desired). Tasks were completed before, and 3 months after, surgery in patients, and at two visits, 3 months apart, in controls. Body weight (kg) was recorded at both visits. Virtual portions differed significantly across groups, visits, eating contexts, energy densities (low vs. high), and tastes (sweet vs. salty). Portions created by controls did not change over time, while portions created by patients decreased significantly after surgery, for all contexts except healthy. For patients, desired and healthy portions predicted 3-month weight loss. VPCTs are replicable, responsive to foods and eating contexts, and predict surgical weight loss. These tasks could be useful for individual assessment of expectations of amounts that are eaten in health and disease and for prediction of weight loss.

The effect of caries increment on oral health-related quality of life among adolescents in Brazil: a 3-year longitudinal study

This study aimed to assess the effect of caries increment on Oral Health-related Quality of Life (OHRQoL) of Brazilian adolescents and to evaluate the responsiveness of the Child Perceptions Questionnaire (CPQ11-14) in this group. A population-based sample of 515 Brazilian 12-year-olds from a large city located in the southeast of Brazil was evaluated according to a random multistage sampling design at baseline and 291 at three years follow-up, using the DMFT index and the CPQ11-14 instrument. To evaluate the responsiveness to change, the measures of effect size and longitudinal construct validity were used. It was verified that OHRQoL among adolescents with DMFT increment across the three years worsened significantly (p<0.05) in relation to their counterparts. The effect size varied from small to moderate. The longitudinal construct validity of CPQ11-14 was satisfactory. Caries increment impacted on OHRQoL of adolescents in Brazil. The CPQ11-14 instrument demonstrated acceptable responsiveness properties.

Value-Based Healthcare in Ostomies

In order to achieve significant improvements in quality, cost, and accessibility (the health “iron triangle”), innovation in organizational and service delivery models is necessary to increase the value of healthcare. The aim of this study is to evaluate the efficiency of a model of organizational innovation based on advanced practice nurse in the care of people with ostomies (APN-O) versus usual care. An observational, exploratory, analytical, prospective study with a six-month follow-up was carried out at 12 hospitals that implemented this model in Andalusia. A total of 75 patients who had undergone a digestive elimination ostomy and/or a urinary ostomy were followed for six months. Clinical outcomes, healthcare resources, health-related quality of life, and willingness to pay (WTP) were analyzed. The economic evaluation was conducted from a societal perspective, including healthcare costs and indirect costs. The cost difference between the two models was €136.99 and the quality-adjusted life year (QALY) gained was 0.05965 (€2297 per QALY gained). At six months, the mean of WTP was €69 per APN-O consultation. This model contributes to increasing the value-based healthcare in ostomies. Results of this study suggested that APN-O is an effective patient management model for improving their health status and is highly efficient.

Five direct behavior rating multi-item scales: Sensitivity to the effects of classroom interventions

Direct Behavior Rating (DBR) is a tool designed for the assessment of behavioral changes over time. Unlike methods for summative evaluations, the development of progress monitoring tools requires evaluation of sensitivity to change. The present study aimed to evaluate this psychometric feature of five newly developed DBR Multi-Item Scales (DBR-MIS). Teachers identified students with behaviors interfering with their learning or the learning of others and implemented a Daily Report Card (DRC) intervention in the classroom settings for two months. The analyses were performed on 31 AB single case studies. Change metrics were calculated at an individual level by using Tau-U_{A vs. B + trend B} and Hedges' g and at a scale-level by using Mixed Effect Meta-Analysis, Hierarchical Linear Models (HLMs), and Between-Case Standardized Mean Difference (BC-SMD). HLMs were estimated considering both fixed and random effects of intervention and linear trend within the intervention phase. The results supported sensitivity to change for three DBR-MIS (i.e., Academic Engagement, Organizational Skills, and Disruptive Behavior), and the relative magnitudes were consistent across the metrics. Sensitivity to change of DBR-MIS Interpersonal Skills received moderate support. Conversely, empirical evidence was not provided for sensitivity to change of DBR-MIS Oppositional Behavior. Particular emphasis was placed on the intervention trend in that responses to behavioral interventions might occur gradually or require consistency over time in order to be observed by raters. Implications for the use of the new DBR-MIS in the context of progress monitoring of social-emotional behaviors are discussed.

Minimal clinically important difference and minimal detectable change of the World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) amongst patients with chronic musculoskeletal pain

OBJECTIVES

The aim of this study is to estimate a minimal clinically important difference (MCID) and a minimal detectable change (MDC) of the 12-item WHODAS 2.0 amongst patients with chronic musculoskeletal pain.

DESIGN

Cross-sectional cohort study.

SETTING

Outpatient Physical and Rehabilitation Medicine clinic.

SUBJECTS

A total of 1988 consecutive patients with musculoskeletal pain.

INTERVENTIONS

A distribution-based approach was employed to estimate a minimal clinically important difference, a minimal detectable change, and a minimal detectable percent change (MDC%).

RESULTS

The mean age of the patients was 48 years, and 65% were women. The average intensity of pain was 6,3 (2.0) points (0-10 numeric rating scale) and the mean WHODAS 2.0 total score was 13 (9) points out of 48. The minimal clinically important difference ranged between 3.1 and 4.7 points. The minimal detectable change was 8.6 points and minimal detectable % change was unacceptably high 66%.

CONCLUSIONS

Amongst patients with chronic musculoskeletal pain, the 12-item WHODAS 2.0 demonstrated a high minimal detectable change of almost nine points. As the minimal detectable change exceeded the level of minimal clinically important difference, nine points were considered to be the amount of change perceived by a respondent as clinically significant.

Rational Selection of Patient-Reported Outcomes Measures in Lumbar Spine Surgery Patients

Background

The Short Form-12 (SF-12) was developed as a shorter version of the SF-36, yet there has been limited validation of its reliability at measuring postoperative changes. The purpose of this study was to determine if the SF-12 could safely substitute for the SF-36 in measuring postoperative change in lumbar spine surgery patients and if the condition specific (Oswestry Disability Index [ODI]) or pain (visual analog scale [VAS]) instruments, provided additional utility.

Methods

A total of 972 patients from a single center who underwent lumbar spine surgery for a predominant symptom of radiating leg pain with (n = 237) or without (n = 735) fusion and prospectively completed both SF-36 and ODI instruments before and after surgery were included. The SF-12 score was calculated from the appropriate subset of SF-36 responses. The absolute sensitivity and the intraclass correlation coefficient were calculated. Reliability of each instrument to measure preoperative to postoperative change was calculated as the standardized response mean.

Results

The SF-12 and SF-36 demonstrated a strong correlation with each other ([0.97, P < .001] and [0.93, P < .001], respectively) preoperatively and postoperatively. The SF-12 and SF-36 scores were moderately to strongly inversely correlated with the ODI. The ODI showed greater reliability at measuring change than the SF-12 for both fusion (0.94 versus 0.72) and nonfusion (0.81 versus 0.33) lumbar surgery patients.

Conclusions

The SF-12 was as effective as the SF-36 to measure general health status in lumbar spine surgery patients, and both were moderate to strong predictors of ODI preoperatively and postoperatively, but lack the reliability to detect change seen with the ODI or VAS after surgical intervention.

Level of Evidence

3.

Clinical Relevance

These data suggest that the SF-12 is a valid substitute for the SF-36 to measure postoperative outcomes changes, but that the ODI should continue to be used to measure condition specific changes in function.

German version of the Chedoke McMaster arm and hand activity inventory (CAHAI-G): intra-rater reliability and responsiveness

BACKGROUND

The English version of the Chedoke Arm and Hand Activity Inventory is a validated, upper-limb measure with the purpose of assessing functional recovery of the arm and hand after a stroke. A German translation and cross-cultural adaptation was recently produced and demonstrated high validity, inter-rater reliability and internal consistency. As a follow-up, the present study evaluated the intra-rater reliability and responsiveness of the CAHAI-G for the long and all shortened versions.

METHODS

The CAHAI-G and the Action Research Arm Test were assessed on three different measurement events: upon entry (ME1), two to 3 days after entry (ME2), and after three to 4 weeks (ME3). For the intra-rater reliability analysis, the ME1 CAHAI assessments were recorded on video and rated by three therapists to obtain the intraclass coefficients (ICC). The data of all three MEs were analysed in a group of stroke inpatients for the evaluation of responsiveness. To test for responsiveness, the CAHAI-G change data were compared to concurrent instruments: The Global Rating of Change-questionnaire and the Global Rating of Concept-questionnaire. Both served as external criteria. For all CAHAI-G versions (7, 8, 9 or 13 items), the same analysis procedures for the evaluation of the responsiveness parameter were performed.

RESULTS

In total, 27 patients (9 females, age 63 ± 13.7) were enrolled in the study. The ICCs for the intra-rater reliability were calculated to be between 0.988 and 0.998 for all CAHAI versions. Responsiveness parameters were as follows from CAHAI-G 7 to 13: Minimal Detectable Change (MDC₉₀₎ 5.3, 6.0, 6.1, 8.2; Pearson's correlation coefficients CAHAI-Gs with ARAT 0.365, 0.409^*, 0.500^**, 0.597^**. The Area und Under the Curve and the Minimal Clinical Important Difference values for all CAHAI-G versions and the three external criteria ranged between 0.483 to 0.603 and 2.5 to 9.0, respectively.

CONCLUSION

In addition to the high validity, inter-rater reliability and internal consistency, the CAHAI-G revealed high intra-rater reliability. The data also suggest an adequate responsiveness of the CAHAI-G versions 9 and 13.

Minimal clinically important difference (MCID) for patient-reported shoulder outcomes

BACKGROUND

The minimal clinically important difference (MCID) is used when interpreting the importance of outcome data. However, a consensus regarding the MCID for commonly used patient-reported outcomes in shoulder surgery has not been established. The purpose of this systematic review was to evaluate the available literature on shoulder MCID to improve clinical interpretation of shoulder outcome data.

METHODS

A systematic review of the literature was conducted to identify studies reporting anchor-based MCID values for the patient-reported outcomes recommended by the American Shoulder and Elbow Surgeons (ASES): Veterans Rand 12 score, ASES score, Single Assessment Numeric Evaluation (SANE) score, Western Ontario Rotator Cuff (WORC) score, Western Ontario Osteoarthritis Score (WOOS), Western Ontario Shoulder Instability Index (WOSI), Pennsylvania Shoulder Score, and Oxford Shoulder Score (OSS).

RESULTS

A total of 14 articles reporting anchor-based MCID values were included in the final analysis. No studies reporting the Western Ontario Osteoarthritis Score (WOOS) were identified. The ASES score (6 studies), OSS (4 studies), and WORC score (2 studies) were the only instruments investigated in more than 1 study. The average reported MCID values for the ASES, OSS, and WORC scores were 15.5 (15% total difference), 275.7 (13% total difference), and 6 (13% total difference), respectively. The vast majority of studies failed to report information necessary to validate the credibility of these MCID values.

DISCUSSION AND CONCLUSION

The current utility of the MCID for patient-report shoulder outcome instruments is limited by poor study methodology, inadequate reporting, and a lack of data. Further research is needed to more clearly define the MCID values for commonly used patient-reported outcomes in shoulder surgery.

Evaluating the reproducibility of the short version of the Western Ontario Rotator Cuff Index (Short-WORC) prospectively

Background

Recently, a shorter version of the Western Ontario Rotator Cuff Index (Short-WORC) was created to reduce patient response burden. However, it has yet to be evaluated prospectively for reproducibility (reliability and agreement) and floor and ceiling effects.

Methods

Patients (N = 162) with rotator cuff disorders completed the Short-WORC at baseline. From this cohort, 47 patients underwent measurement of test-retest reliability within 2 to 7 days. We used the Cronbach α to determine internal consistency and the intraclass correlation coefficient (ICC_2,1) to assess test-retest reliability. To evaluate parameters of agreement, the standard error of measurement, minimal detectable change (based on a 90% confidence interval), and Bland-Altman plots were used.

Results

The Cronbach α was 0.82 at baseline, and the intraclass correlation coefficient (ICC_2,1) was 0.87. The agreement parameter was 8.4 for the standard error of measurement of agreement, and the limits of agreement fell within the range of –22.9 to 23.8. The Short-WORC is reliable over time and reflective of a patient’s true score after an intervention.

Conclusions

The Short-WORC demonstrated strong reproducibility parameters and can be used for patients with rotator cuff disorders. The Short-WORC indicated no systematic bias and was reflective of the true score of both individual patients and groups of patients at 2 time points.

Physical Literacy as A Framework of Assessment and Intervention for Children and Youth with Developmental Coordination Disorder: A Narrative Critical Review of Conventional Practice and Proposal for Future Directions

A framework of literacy may have roles to play in the assessment and treatment of children and youth with developmental disorders. This review aims to evaluate the conventional practice of assessment and treatment for children and youth with a developmental disorder in the physical domain, called developmental coordination disorder (DCD), and explore how the framework of physical literacy could contribute to the advancement of the current practice. This study adopts a method of narrative critical review based on a non-systematic search for its broad coverage to provide insights into the trend and future alternative directions. Over recent decades, children and youth with DCD have been typically assessed with standardized norm-referenced tests, before and after task-oriented intervention, for aiding diagnosis and evaluating the treatment effect. However, a recent high-quality systematic review showed limited evidence for the treatment effect assessed by the tests. Here, a framework of physical literacy is proposed to be used as an alternative to the conventional practice by recalibrating treatment goals and modifying the assessment and intervention approaches; criterion-referenced real-life authentic assessment and activities are encouraged with an emphasis on the enjoyment of movement and value of physical activity towards the attainment of physically active and healthy lifestyle goals from a lifespan perspective. The application of the physical literacy framework to the assessment and treatment of DCD needs to be further examined conceptually and empirically, while exploring a potential contribution of the literacy framework to transform the conventional assessment and treatment of children and youth with other developmental disorders.

A comprehensive guideline for Bland-Altman and intra class correlation calculations to properly compare two methods of measurement and interpret findings

The rapid emergence of new measurement instruments and methods requires personnel and researchers of different disciplines to know the correct statistical methods to utilize to compare their performance with reference ones and properly interpret findings. We discuss the often-made mistake of applying the inappropriate correlation and regression statistical approaches to compare methods and then explain the concepts of agreement and reliability. Then, we introduce the intraclass correlation as a measure of inter-rater reliability, and the Bland-Altman plot as a measure of agreement, and we provide formulae to calculate them along with illustrative examples for different types of study designs, specifically single measurement per subject, repeated measurement while the true value is constant, and repeated measurement when the true value is not constant. We emphasize the requirement to validate the assumptions of these statistical approaches, and also how to deal with violations and provide formulae on how to calculate the confidence interval for estimated values of agreement and intraclass correlation. Finally, we explain how to interpret and report the findings of these statistical analyses.

The responsiveness and predictive validity of the de Morton Mobility Index in geriatric rehabilitation

Introduction: The de Morton Mobility Index (DEMMI) is a unidimensional clinical instrument proven reliable and valid for measuring mobility in the elderly, but its responsiveness has not been evaluated when applying the methodology recommended by COSMIN.Purpose: To evaluate the responsiveness and the predictive validity of the DEMMI.Materials and methods: Elderly people, referred for municipality-based rehabilitation, were consecutively included. Eight hypotheses regarding the relationship between the change scores in the DEMMI and other functional measures of mobility were tested by assessing the participants before and after rehabilitation. Three months after completing the rehabilitation, the participants' functional status was evaluated.Results: 250 elderly people (mean age 85 years, 63% women) were included. The change scores in the DEMMI and the other measures correlated (r = 0.422 to 0.547), but only three out of the eight hypotheses were confirmed. A DEMMI score of less than 48 and less than 67 can predict inability of ambulating close to the participant's residence and using public transport, respectively.Conclusions: We found a moderate responsiveness and a high predictive validity of the DEMMI score. The DEMMI appears to be an appropriate instrument for monitoring change in mobility and predicting functional status among elderly patients undergoing rehabilitation.Implications for rehabilitationThe de Morton Mobility Index (DEMMI) is a clinical instrument for measuring mobility.This study shows that the DEMMI was responsive to change in mobility and predicted disability among a diverse group of elderly individuals referred to rehabilitation after hospital discharge.The DEMMI is applicable as an outcome measure in trials investigating the effect of rehabilitation and for clinical decision-making concerning geriatric rehabilitation.

Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: instrument development and reliability study

OBJECTIVE

To develop an instrument to evaluate the credibility of anchor based minimal important differences (MIDs) for outcome measures reported by patients, and to assess the reliability of the instrument.

DESIGN

Instrument development and reliability study.

DATA SOURCES

Initial criteria were developed for evaluating the credibility of anchor based MIDs based on a literature review (Medline, Embase, CINAHL, and PsycInfo databases) and the experience of the authors in the methodology for estimation of MIDs. Iterative discussions by the team and pilot testing with experts and potential users facilitated the development of the final instrument.

PARTICIPANTS

With the newly developed instrument, pairs of masters, doctoral, or postdoctoral students with a background in health research methodology independently evaluated the credibility of a sample of MID estimates.

MAIN OUTCOME MEASURES

Core credibility criteria applicable to all anchor types, additional criteria for transition rating anchors, and inter-rater reliability coefficients were determined.

RESULTS

The credibility instrument has five core criteria: the anchor is rated by the patient; the anchor is interpretable and relevant to the patient; the MID estimate is precise; the correlation between the anchor and the outcome measure reported by the patient is satisfactory; and the authors select a threshold on the anchor that reflects a small but important difference. The additional criteria for transition rating anchors are: the time elapsed between baseline and follow-up measurement for estimation of the MID is optimal; and the correlations of the transition rating with the baseline, follow-up, and change score in the patient reported outcome measures are satisfactory. Inter-rater reliability coefficients (ĸ) for the core criteria and for one item from the additional criteria ranged from 0.70 to 0.94. Reporting issues prevented the evaluation of the reliability of the three other additional criteria for the transition rating anchors.

CONCLUSIONS

Researchers, clinicians, and healthcare policy decision makers can consider using this instrument to evaluate the design, conduct, and analysis of studies estimating anchor based minimal important differences.

Measuring clinically relevant improvement after lumbar spine surgery: is it time for something new?

BACKGROUND CONTEXT

Minimum clinically important difference (MCID) for patient-reported outcome measures is commonly used to assess clinical improvement. However, recent literature suggests that an absolute point-change may not be an effective or reliable marker of response to treatment for patients with low or high baseline patient-reported outcome scores. The multitude of established MCIDs also makes it difficult to compare outcomes across studies and different spine surgery procedures.

PURPOSE

To determine whether a 30% reduction from baseline in disability and pain is a valid method for determining clinical improvement after lumbar spine surgery.

STUDY DESIGN

Retrospective analysis of prospective data from a national spine registry, the Quality Outcomes Database.

PATIENT SAMPLE

There were 23,280 participants undergoing elective lumbar spine surgery for degenerative disease who completed a baseline and follow-up assessment at 12 months.

OUTCOME MEASURES

Patient-reported disability (Oswestry Disability Index [ODI]), back and leg pain (11-point Numeric Rating Scale [NRS]), and satisfaction (NASS scale).

METHODS

Patients completed baseline and a 12-month postoperative assessment to evaluate the outcomes of disability, pain, and satisfaction. The change in ODI and NRS pain scores was categorized as met (≥30%) or not met (<30%) percent reduction MCID. The 30% reduction from baseline was compared with a wide range of well-established absolute point-change MCID values. The relationship between 30% reduction and absolute change values and satisfaction were primarily compared using receiver operating characteristic (ROC) curves, area under the curve (AUROC), and logistic regression analyses. Analyses were conducted for overall scores and for disability and pain severity categories and by surgical procedure.

RESULTS

Thirty percent reduction in ODI and back and leg pain predicted satisfaction with more accuracy than absolute point-change values for the total population and across all procedure categories (p<.001), except for when compared with the highest absolute point-change threshold for leg pain (3.5-point reduction). The largest AUROC differences, in favor of 30% reduction, were found for the lowest disability (ODI 0-20%: 21.8%) and bed-bound disability (ODI 81%-100%: 13.9%) categories. For pain, there was a 3.4%-12.4% and 1.3%-9.8% AUROC difference for no/mild back and leg pain (NRS 0-4), respectively, in favor of a 30% reduction threshold.

CONCLUSIONS

A 30% reduction MCID either outperformed or was similar to absolute point-change MCID values. Results indicate that a 30% reduction (baseline to 12 months after surgery) in disability and pain is a valid method for determining clinically relevant improvement in a broad spine surgery population. Furthermore, a 30% reduction was most accurate for patients in the lowest and highest disability and lowest pain severity categories. A 30% reduction MCID allows for a standard cut-off for disability and pain that can be used to compare outcomes across various spine surgery procedures.

Longitudinal Structural MRI in Neurologically Healthy Adults

BACKGROUND

Structural brain MRI measures are frequently examined in both healthy and clinical groups, so an understanding of how these measures vary over time is desirable.

PURPOSE

To test the stability of structural brain MRI measures over time.

POPULATION

In all, 112 healthy volunteers across four sites.

STUDY TYPE

Retrospective analysis of prospectively acquired data.

FIELD STRENGTH/SEQUENCE

3 T, magnetization prepared - rapid gradient echo, and single-shell diffusion sequence.

ASSESSMENT

Diffusion, cortical thickness, and volume data from the sensorimotor network were assessed for stability over time across 3 years. Two sites used a Siemens MRI scanner, two sites a Philips scanner.

STATISTICAL TESTS

The stability of structural measures across timepoints was assessed using intraclass correlation coefficients (ICC) for absolute agreement, cutoff ≥0.80, indicating high reliability. Mixed-factorial analysis of variance (ANOVA) was used to examine between-site and between-scanner type differences in individuals over time.

RESULTS

All cortical thickness and gray matter volume measures in the sensorimotor network, plus all diffusivity measures (fractional anisotropy plus mean, axial and radial diffusivities) for primary and premotor cortices, primary somatosensory thalamic connections, and the cortico-spinal tract met ICC. The majority of measures differed significantly between scanners, with a trend for sites using Siemens scanners to produce larger values for connectivity, cortical thickness, and volume measures than sites using Philips scanners.

DATA CONCLUSION

Levels of reliability over time for all tested structural MRI measures were generally high, indicating that any differences between measurements over time likely reflect underlying biological differences rather than inherent methodological variability.

LEVEL OF EVIDENCE

4.

TECHNICAL EFFICACY STAGE

1.

Responsiveness, minimal detectable change, and minimal clinically important difference of the sitting balance scale and function in sitting test in people with stroke

Background: Limited studies have examined the responsiveness of the Sitting Balance Scale (SBS) and Function in Sitting Test (FIST). No studies have investigated the psychometric properties of these tools in skilled nursing facilities.Objective: The purposes of the study were (1) to examine responsiveness of the SBS and FIST, in people with stroke, receiving rehabilitation in skilled nursing facilities; and (2) to estimate the MDC and MCID of these scales.Methods: Forty participants completed the FIST, SBS, and Barthel Index (BI) at admission and discharge. Internal responsiveness of the FIST and SBS was measured using Effect Size (ES) and Standardized Response Mean (SRM). Examining the association between the difference in scores on the SBS or FIST and the difference in scores on the BI determined external responsiveness. MDC and MCID were estimated for both measures.Results: The ES and the SRM for both scales were large (1.01-2.30) indicating excellent internal responsiveness. Both scales demonstrated satisfactory external responsiveness, showing good association with change in BI scores (p < .01). MDC₉₀ of the SBS and the FIST were 2.3 and 3.9, respectively. Anchor-based MCID estimates were 4.5 and 3.5, and distribution-based were 3.5 and 4.8 for the SBS and FIST, respectively.Conclusions: Both scales demonstrate sufficient responsiveness in the skilled nursing setting. Our findings suggest a change of 5 on the SBS and 4 on the FIST are clinically important, and clinicians may use these values to assess patient progress.

How is the minimal clinically important difference established in health-related quality of life instruments? Review of anchors and methods

Background

The aim of this systematic review is to describe the different types of anchors and statistical methods used in estimating the Minimal Clinically Important Difference (MCID) for Health-Related Quality of Life (HRQoL) instruments.

Methods

PubMed and Google scholar were searched for English and French language studies published from 2010 to 2018 using selected keywords. We included original articles (reviews, meta-analysis, commentaries and research letters were not considered) that described anchors and statistical methods used to estimate the MCID in HRQoL instruments.

Results

Forty-seven papers satisfied the inclusion criteria. The MCID was estimated for 6 generic and 18 disease-specific instruments. Most studies in our review used anchor-based methods (n = 41), either alone or in combination with distribution-based methods. The most common applied anchors were non-clinical, from the viewpoint of patients. Different statistical methods for anchor-based methods were applied and the Change Difference (CD) was the most used one. Most distributional methods included 0.2 standard deviations (SD), 0.3 SD, 0.5 SD and 1 standard error of measurement (SEM). MCID values were very variable depending on methods applied, and also on clinical context of the study.

Conclusion

Multiple anchors and methods were applied in the included studies, which lead to different estimations of MCID. Using several methods enables to assess the robustness of the results. This corresponds to a sensitivity analysis of the methods. Close collaboration between statisticians and clinicians is recommended to integrate an agreement regarding the appropriate method to determine MCID for a specific context.

Evaluation of clinical meaningfulness of estrogen plus progesterone oral capsule (TX-001HR) on moderate to severe vasomotor symptoms

Objective:

The aim of this study was to determine the clinical meaningfulness of TX-001HR in reducing moderate to severe vasomotor symptoms (VMS) in menopausal women with a uterus.

Methods:

In the REPLENISH study (NCT01942668), women with moderate to severe hot flushes (≥7/d or ≥50/wk) were enrolled in a VMS substudy and randomized to four doses of daily TX-001HR (17β-estradiol/progesterone) or placebo. Participants assessed improvement of their VMS by the Clinical Global Impression and the Menopause-Specific Quality of Life (MENQOL) questionnaire, which were used to define clinical responders, clinically important differences (CIDs) or minimal CID (MCID) in VMS frequency. Response thresholds were determined by nonparametric discriminant analyses utilizing bootstrapping methods.

Results:

In the modified intent-to-treat VMS substudy population (n = 726), statistically significantly more Clinical Global Impression–based clinical responders were observed with TX-001HR than placebo for MCID (weekly reduction of ≥25 moderate to severe VMS: 82-88% vs 69%; all, P < 0.05) and CID (weekly reduction of ≥39 VMS: 68%-73% vs 52%; all, P < 0.05) at week 12. Week 4 results were similar. For Menopause Quality of Life–based analysis, significantly more clinical responders were observed with TX-001HR than placebo for MCID (weekly reduction of ≥34 VMS: 74%-81% vs 55%; all, P < 0.01) and CID (weekly reduction of ≥44 VMS: 61%-69% vs 42%; all, P < 0.01) at week 12.

Conclusions:

TX-001HR provided clinically meaningful improvements (as measured by 2 different methods), in addition to statistically significant reductions, in menopausal VMS frequency. TX-001HR may provide a new option, as a single oral capsule of estradiol and progesterone (identical to the hormones naturally occurring in women) for the treatment of moderate to severe VMS in menopausal women with a uterus.

Oral 17β-estradiol/progesterone (TX-001HR) and quality of life in postmenopausal women with vasomotor symptoms

Objective:

The aim of the study was to describe the effects of TX-001HR (17β-estradiol [E2] and natural progesterone [P4] in a single oral capsule) on menopause-specific quality of life in women with moderate to severe vasomotor symptoms (VMS).

Methods:

The REPLENISH study (NCT01942668) was a phase 3, randomized, double-blind, placebo-controlled, multicenter trial which evaluated four E2/P4 doses in postmenopausal women with VMS and a uterus. Women with moderate to severe hot flushes (≥7/d or ≥50/wk) were included in a VMS substudy. Participants self-administered the Menopause-Specific Quality of Life (MENQOL) questionnaire. Baseline changes in MENQOL overall and domains were determined as well as correlations between changes in MENQOL scores and VMS frequency or severity.

Results:

In the VMS substudy, women treated with E2/P4 had significantly greater improvements from baseline in their MENQOL overall score at week 12, and months 6 and 12, compared with placebo (all, P < 0.05, except the lowest E2/P4 dose at months 6 and 12). Improvements from baseline for the MENQOL vasomotor domain score were significantly greater with TX-001HR doses versus placebo at all time points (all, P < 0.01). Changes in MENQOL vasomotor scores moderately correlated with changes in VMS frequency (r = 0.56, P < 0.0001) and severity (r = 0.55, P < 0.0001).

Conclusion:

In the REPLENISH trial, women with moderate to severe VMS treated with most E2/P4 doses reported significant improvements in quality of life from baseline to 12 weeks compared with placebo, which were maintained up to 12 months. TX-001HR, if approved, may provide the first oral hormone therapy formulation in a single capsule containing E2 and P4 for the treatment of VMS in postmenopausal women with a uterus.

Can PROMIS domains of pain and physical functioning detect changes in health over time for children with sickle cell disease?

BACKGROUND

The Patient-Reported Outcomes Measurement Information System (PROMIS) includes multiple domains that measure pain and physical functioning which are valid and reliable for use in children with sickle cell disease. The responsiveness of these measures to detect changes in health status over time among children with sickle cell disease is unknown.

PROCEDURE

We conducted a prospective cohort study of children presenting to emergency department (ED) with vaso-occlusive crises. Children completed PROMIS surveys in the ED and at two follow-up time points (7-10 days and 1-3 months) after their acute care visit. Linear mixed models were used to determine if there were significant changes in PROMIS T scores over time. We used a patient's global assessment of change in pain question to anchor the changes in PROMIS scores (mean and 95% confidence interval). A change was considered statistically significant if the 95% CI did not include 0.

RESULTS

We found that patients improved significantly in all domains 1 to 3 months after discharge from an acute care visit for pain. In addition, the pain and physical stress experience domains were responsive to change 7 to 10 days after discharge. Using the anchor of change in pain, for children who had considerable improvement in pain, there were significant changes in PROMIS T scores ranging from 6 to 15.

CONCLUSIONS

Relevant PROMIS domains detect changes in children experiencing acute vaso-occlusive crises. These domains can be used in research and clinic settings to measure clinically relevant change in children with sickle cell disease.

Translation, Cross-cultural Adaptation, and Psychometric Properties of the Hausa Versions of the Numerical Pain Rating Scale and Global Rating of Change Scale in a Low-literate Population With Chronic Low Back Pain

STUDY DESIGN

Translation, cross-cultural adaptation, and psychometric testing.

OBJECTIVE

To translate, cross-culturally adapt, and validate the Numerical Pain Rating Scale (NPRS) and the Global Rating of Change Scale (GRCS) into Hausa language.

SUMMARY OF BACKGROUND DATA

The NPRS and GRCS are commonly used patient-reported outcome measures in a variety of pain-related conditions including low back pain. To date, neither the NPRS nor GROC are available in Hausa language.

METHODS

The Hausa versions of the NPRS (NPRS-H) and GRCS (GRCS-H) were developed using recommended guidelines. The final versions were then administered to 120 patients with chronic low back pain to access their psychometric properties. Reliability assessment included calculations of intraclass correlation coefficient (ICC) and minimal detectable change among the stable group. Construct validity and concurrent validity were assessed using the Spearman rank correlation coefficient. Internal responsiveness was assessed using mean change scores, standardized effect size, and standard response mean. Receiver operating characteristic curves were plotted to determine the external responsiveness of the NPRS-H using the area under the curve, and minimal important change for small, medium, and large improvements. Outcome measures consisted of the Visual Analogue Scale for pain and Oswestry Disability Index.

RESULTS

The NPRS-H and GRCS-H were successfully developed. High test-retest reliability was demonstrated for both the NPRS-H (ICC = 0.95) and GRCS-H (ICC = 0.94) with minimal detectable change points of 1.0 and 1.2 respectively. The scales showed moderate to strong correlation with the Visual Analogue Scale for pain and Oswestry Disability Index. The mean change of the NPRS-H scores correlated moderately with the GRCS-H. Both scales demonstrated good internal responsiveness. External responsiveness of the NPRS-H was demonstrated at three levels with area under the curve  = 0.875 to 0.972, and minimal important change  = 2.5 to 3.5.

CONCLUSIONS

The NPRS and GRCS were successfully adapted into Hausa language with acceptable reliability, validity, and responsiveness. These measures are appropriate for clinical and research use among Hausa-speaking patients.

LEVEL OF EVIDENCE

2.

The vestibulo-masseteric reflex and the acoustic-masseteric reflex: a reliability and responsiveness study in healthy subjects

The vestibulo-masseteric reflex (VMR or p11 wave), the acoustic-masseteric reflex (AMR or p1/n21 wave) and the mixed vestibulo-cochlear p11/n21 potential are responses of masseter muscles to sound that can be employed to evaluate brainstem function. This study was aimed at establishing the test-retest reliability and responsiveness of these reflex parameters according to the type of electrode configuration. Twenty-two healthy volunteers (M:F = 11:11; mean age 25.3 ± 5.2 years) participated in two testing sessions separated by one week. Zygomatic and mandibular montages were compared following unilateral and bilateral stimulations. For reliability purposes, intraclass correlation coefficient (ICC), coefficient of variation of the method error (CVME) and standard error of measurement (SEM) were calculated. The minimal detectable difference (MDD) was also determined as a measure of responsiveness. Both VMR (p11 wave) and AMR could be consistently evoked from test to retest, although the frequency rate was significantly higher (all p values ≤ 0.009) with zygomatic (VMR: 97.7-100%; AMR: 86.9-97.6%) than mandibular montage (VMR: 84.7-89.8%; AMR: 65.0-67.8%), with no significant differences between unilateral and bilateral stimulations. Good-to-excellent reliability and responsiveness (high ICC, low CV_ME, SEM and MDD scores) were detected for corrected amplitudes and peak latencies for all reflex responses, whereas raw amplitudes were associated to poor reliability. The reliability of the zygomatic montage proved superior to the mandibular montage for all reflex responses. Given their high test-retest consistency and capability to study different features of the reflex arch, both peak latencies and corrected amplitudes should be reported and considered in the interpretation of reflex testing results.

Responsiveness of Device-Based and Self-Report Measures of Physical Activity to Detect Behavior Change in Men Taking Part in the Football Fans in Training (FFIT) Program

The capacity of physical activity (PA) measures to detect changes in PA within interventions is crucial. This is the first study to examine the responsiveness of activPAL3™ and the International Physical Activity Questionnaire (IPAQ; Short Form) in detecting PA change during a 12-week group-based, men-only weight management program-Football Fans in Training (FFIT). Participants wore an activPAL3™ and completed the IPAQ pre- and post-program (n = 30). Relationships between change scores were assessed by Spearman's correlations. Mean or median changes in PA were measured using paired samples t-tests and Wilcoxon signed-rank tests. Responsiveness to change was assessed utilizing Standardized Response Mean (SRM). Both device-based and self-report measures demonstrated significant changes pre-post intervention, although these changes were not significantly correlated. The SRM values for changes in activPAL3™ metrics were: 0.54 (MET-mins/day); 0.53 (step counts/day); and 0.44 (MVPA/day), indicating a small to medium responsiveness to change. SRM values for changes in IPAQ scores were: 0.59 (for total PA mins/day); 0.54 (for total MET-mins/day); 0.59 (for walking MET-mins/day); 0.38 (for vigorous MET-mins/day); and 0.38 (for moderate MET-mins/day), revealing a small to medium responsiveness to change. These findings reveal that two commonly used device-based and self-report measures demonstrated responsiveness to changes in PA. While inclusion of both device-based and self-report measures is desirable within interventions it is not always feasible. The results from this study support that self-reported measures can detect PA change within behavioral interventions, although may have a tendency to overestimate changes compared with device-based measures on absolute values, but not standardized response values.

Psychometric Analysis of the Abdominal Score From the Diary for Irritable Bowel Syndrome Symptoms-Constipation Using Phase IIb Clinical Trial Data

OBJECTIVES

The Diary for Irritable Bowel Syndrome Symptoms-Constipation (DIBSS-C) has been developed to assess the core signs and symptoms of irritable bowel syndrome with constipation (IBS-C). This article presents the psychometric evaluation of the DIBSS-C abdominal score.

METHODS

Data for these analyses are from a multicenter phase IIb study in IBS-C patients (NCT02559206). Subjects completed a number of assessments via handheld electronic diary throughout the study. The analyses used the intent-to-treat population and were blinded to randomized treatment group. The analyses evaluated the reliability, validity, and responsiveness of the DIBSS-C abdominal score; identified an appropriate scoring algorithm; and determined thresholds for interpreting clinically meaningful changes at the individual level.

RESULTS

The correlations between the DIBSS-C abdominal symptom items (ie, abdominal pain, discomfort, and bloating) were strong (>0.75). Cronbach's alpha for the abdominal symptom severity items was very strong (.94), indicating that the 3 abdominal symptom items produce a reliable score. The intraclass correlation coefficient for the abdominal score was 0.82, exceeding the threshold of 0.70 and indicating good test-retest reliability. Guyatt's responsiveness statistic values all exceeded the threshold for a large effect of 0.80, so the DIBSS-C abdominal score can be considered highly responsive to change. Triangulation across 3 sets of anchor-based analyses indicated that a threshold of -2.0 points on the abdominal score is an appropriate threshold for identifying meaningful change.

CONCLUSIONS

Overall, this study provides evidence that the DIBSS-C abdominal score is valid, reliable, responsive to change, and interpretable for assessing treatment benefit in patients with IBS-C.

Frailty modifies the intervention effect of chair yoga on pain among older adults with lower extremity osteoarthritis: Secondary analysis of a nonpharmacological intervention trial

OBJECTIVE

In an 8-week nonpharmacological pain intervention trial among older adults with lower extremity osteoarthritis (OA), we aimed to examine: a) the baseline frailty level of the participants; b) whether such intervention is more beneficial for baseline frailer older adults than for their counterparts with less frailty; and c) whether the intervention could also alter frailty.

METHODS

Participants were randomly assigned to either chair yoga (CY) or health education program (HEP) groups and attended twice-weekly 45-minute CY or HEP sessions for 8 weeks. Following a standard procedure, 82 variables were used to construct a frailty index (FI, 0-1). Primary outcomes were: Western Ontario and McMaster Universities (WOMAC) pain and pain interference. Linear mixed-effects models were used to evaluate the modifying effect of baseline frailty on the intervention effect of CY on primary outcomes. Similar models were used to evaluate the effect of CY on frailty.

RESULTS

A total of 112 participants (n = 63 CY, n = 49 HEP; 75.3 [SD = 7.5] years) with 85 females (75.9%) were included. The mean values of baseline FI for the CY and HEP groups were similar (0.428 [0.05] and 0.433 [0.05], P = 0.355). Each 0.01 increment in baseline FI was associated with higher WOMAC pain (beta = 0.28, P < 0.001) and pain interference (beta = 0.51, P < 0.001). There was a significant interaction effect between intervention, time, and baseline FI (P = 0.020 for WOMAC pain; P = 0.010 for pain interference), indicating that participants with higher level of baseline FI had greater declines in WOMAC pain and pain interference. There was no significantly greater decline in FI for the CY group compared to the HEP group (between-group difference - 0.01; P = 0.509) and there were no significant trend changes in FI (P for interaction = 0.605).

CONCLUSIONS

Frailty modifies the intervention effect of CY on pain among older adults with lower extremity OA, underscoring the importance of assessing frailty to improve the management of pain in this population.