Interpreting change from patient reported outcome (PRO) endpoints: patient global ratings of concept versus patient global ratings of change, a case study among osteoporosis patients

Meaningful Change Thresholds for the Psoriasis Symptoms and Signs Diary: A Secondary Analysis of a Randomized Clinical Trial

Importance

Change from baseline score on the validated Psoriasis Symptoms and Signs Diary (PSSD) is a widely used, patient-reported end point in clinical trials for psoriasis. Meaningful score change thresholds anchored to patient-reported assessments have not been established in a clinical trial setting.

Objective

To evaluate meaningful within-patient score change thresholds for the PSSD using data from the phase 3 Program to Evaluate the Efficacy and Safety of Deucravacitinib, a Selective TYK2 Inhibitor (POETYK), PSO-1 clinical trial, which compared the efficacy and safety of deucravacitinib vs placebo and apremilast among adults with moderate to severe plaque psoriasis.

Design, Setting, and Participants

In this predefined analysis using data from the POETYK PSO-1 multicenter, randomized, double-blind, placebo-controlled phase 3 clinical trial, conducted from August 7, 2018, to September 2, 2020, 666 adults with moderate to severe plaque psoriasis completed the PSSD daily throughout the trial. Meaningful change thresholds were derived by anchoring mean PSSD score change from baseline to week 16 to category improvements on the Patient Global Impression of Change (PGI-C) and the Patient Global Impression of Severity (PGI-S).

Interventions

Deucravacitinib, 6 mg, once daily; placebo; or apremilast, 30 mg, twice daily.

Main Outcome and Measures

The main outcome was score change from baseline to week 16 on the PSSD, anchored to the PGI-C and PGI-S.

Results

The trial included 666 patients (mean [SD] age, 46.1 [13.4] years; 453 men [68.0%]). Three thresholds were identified using an analysis set of 609 patients. Score improvement of at least 15 points from baseline reflected meaningful within-patient change anchored to the PGI-C. Score improvements of 25 points were supported by both the PGI-C and the PGI-S, while a 30-point score change identified patients with greater improvements in their psoriasis symptoms and signs.

Conclusions and Relevance

This analysis suggests that PSSD score improvements of 15, 25, or 30 points represent increasing improvements in disease burden that are meaningful to patients with psoriasis.

Psychometric evaluation of the NTDT-PRO questionnaire for assessing symptoms in patients with non-transfusion-dependent beta-thalassaemia

OBJECTIVES

The non-transfusion-dependent beta-thalassaemia-patient-reported outcome (NTDT-PRO) questionnaire was developed for assessing anaemia-related tiredness/weakness (T/W) and shortness of breath (SoB) among patients with NTDT. Psychometric properties were evaluated using blinded data from the BEYOND trial (NCT03342404).

DESIGN

Analysis of a phase 2, double-blind, randomised, placebo-controlled trial.

SETTING

USA, Greece, Italy, Lebanon, Thailand and the UK.

PARTICIPANTS

Adults (≥18 years) (N=145) with NTDT who had not received a red blood cell transfusion within 8 weeks prior to randomisation, with mean baseline haemoglobin level ≤100 g/L.

MEASURES

NTDT-PRO daily scores from baseline until week 24, and scores at select time points for the 36-Item Short Form Health Survey version 2 (SF-36v2), Functional Assessment of Chronic Illness Therapy-Fatigue (FACIT-F) and Patient Global Impression of Severity (PGI-S).

RESULTS

Cronbach's alpha at weeks 13-24 was 0.95 and 0.84 for the T/W and SoB domains, respectively, indicating acceptable internal consistency reliability. Among participants self-reporting no change in thalassaemia symptoms via the PGI-S between baseline and week 1, intraclass correlation coefficients were 0.94 and 0.92 for the T/W and SoB domains, respectively, indicating excellent test-retest reliability. In a known-groups validity analysis, least-squares mean T/W and SoB scores at weeks 13-24 were worse in participants with worse scores for the FACIT-F Fatigue Subscale (FS), SF-36v2 vitality or PGI-S. Indicating responsiveness, changes in T/W and SoB domain scores were moderately correlated with changes in haemoglobin levels, and strongly correlated with changes in SF-36v2 vitality, FACIT-F FS, select FACIT-F items and the PGI-S. Improvements in least-squares mean T/W and SoB scores were higher in participants with greater improvements in scores on other PROs measuring similar constructs.

CONCLUSIONS

The NTDT-PRO demonstrated adequate psychometric properties to assess anaemia-related symptoms in adults with NTDT and can be used to evaluate treatment efficacy in clinical trials.

Triangulation of multiple meaningful change thresholds for patient-reported outcome scores

PURPOSE

The notion of what constitutes meaningful differences or changes in patient-reported outcome scores is represented by meaningful change thresholds (MCTs). Applying multiple methods to estimate MCTs inevitably results in a range of estimates; however, a single estimate or small range is sought in practice to enable consistent interpretation of scores. While current recommendations for triangulation are appropriate in principle, the vital step of moving from all estimates to a value or small range lacks clarity and is subjective in nature. This article aims to review current triangulation approaches and provide more robust recommendations than what is currently available.

METHODS

Current approaches to perform triangulation are described and discussed. Anchor-based estimates are focussed upon due to their recognition as the most valid and developed approach. Recommendations for triangulation are provided.

RESULTS

A correlation-weighted average of MCT estimates is recommended to triangulate multiple MCT estimates derived from a single study into a single value, where increased weighting is given to stronger anchor measures. The choice of method to triangulate estimates from several published studies is highly dependent on the availability of information within the publications. MCTs designed for between-group differences, within-group changes, and within-individual changes should be considered separately.

CONCLUSION

The recommendations within this article provide a reliable and transparent approach to triangulation when a single value is sought, based on meta-analytic approaches. This approach is preferable to a simple mean of estimates where all are weighted equally, or through 'eyeballing' plotted estimates which is unreliable. We encourage researchers to adopt these methods, but to remain aware of the limitations within each method and further nuances in study design that result in heterogeneity. Sensitivity analyses with a range of plausible values are encouraged; however, the recommendations provide a suitable starting value for inferences. Unresolved issues in triangulation, requiring further exploration, are highlighted.

Psychometric evaluation of the Advanced Systemic Mastocytosis Symptom Assessment Form (AdvSM-SAF)

BACKGROUND

The Advanced Systemic Mastocytosis Symptom Assessment Form (AdvSM-SAF) was developed to evaluate symptoms of advanced systemic mastocytosis (AdvSM). This study aimed to psychometrically evaluate AdvSM-SAF scores and provide score interpretation guidelines.

METHODS

The 10-item AdvSM-SAF was administered daily (scored as a seven-day average) in EXPLORER, an open-label Phase 1 study in AdvSM. Score distribution, reliability, construct-related validity, sensitivity to change, and interpretation guidelines were evaluated for AdvSM-SAF items, gastrointestinal symptom score (GSS), skin symptom score (SSS), and total symptom score (TSS).

RESULTS

Thirty-one patients contributed to the analyses. At Baseline, the GSS, SSS, and TSS had adequate internal consistency (α > 0.7) and test-retest reliability (intraclass correlation coefficients >0.7). AdvSM-SAF scores were moderately to strongly correlated with variables as expected, and distinguished among clinically distinct groups. Observed relationships between change scores in the AdvSM-SAF and other assessments reflect evidence that AdvSM-SAF scores change in concert with other assessments designed to measure similar constructs. The magnitude of AdvSM-SAF weekly TSS mean change scores based on different anchor groupings was as expected (improvement > stable > worsening). Candidate clinically meaningful between-group difference estimates (GSS = 2-4, SSS = 2-3, and TSS = 4-7 points) and within-person change estimates (GSS = 6-9, SSS = 1-4, TSS = 9-14) for AdvSM-SAF weekly scores were generated.

CONCLUSION

The AdvSM-SAF produced reliable, construct-valid, and sensitive scores when administered in the target patient population. These results, along with its strong development history and evidence of content validity, indicate that the AdvSM-SAF is fit for the purpose of measuring treatment benefit in individuals with AdvSM.

German version of the Chedoke McMaster arm and hand activity inventory (CAHAI-G): intra-rater reliability and responsiveness

BACKGROUND

The English version of the Chedoke Arm and Hand Activity Inventory is a validated, upper-limb measure with the purpose of assessing functional recovery of the arm and hand after a stroke. A German translation and cross-cultural adaptation was recently produced and demonstrated high validity, inter-rater reliability and internal consistency. As a follow-up, the present study evaluated the intra-rater reliability and responsiveness of the CAHAI-G for the long and all shortened versions.

METHODS

The CAHAI-G and the Action Research Arm Test were assessed on three different measurement events: upon entry (ME1), two to 3 days after entry (ME2), and after three to 4 weeks (ME3). For the intra-rater reliability analysis, the ME1 CAHAI assessments were recorded on video and rated by three therapists to obtain the intraclass coefficients (ICC). The data of all three MEs were analysed in a group of stroke inpatients for the evaluation of responsiveness. To test for responsiveness, the CAHAI-G change data were compared to concurrent instruments: The Global Rating of Change-questionnaire and the Global Rating of Concept-questionnaire. Both served as external criteria. For all CAHAI-G versions (7, 8, 9 or 13 items), the same analysis procedures for the evaluation of the responsiveness parameter were performed.

RESULTS

In total, 27 patients (9 females, age 63 ± 13.7) were enrolled in the study. The ICCs for the intra-rater reliability were calculated to be between 0.988 and 0.998 for all CAHAI versions. Responsiveness parameters were as follows from CAHAI-G 7 to 13: Minimal Detectable Change (MDC₉₀₎ 5.3, 6.0, 6.1, 8.2; Pearson's correlation coefficients CAHAI-Gs with ARAT 0.365, 0.409^*, 0.500^**, 0.597^**. The Area und Under the Curve and the Minimal Clinical Important Difference values for all CAHAI-G versions and the three external criteria ranged between 0.483 to 0.603 and 2.5 to 9.0, respectively.

CONCLUSION

In addition to the high validity, inter-rater reliability and internal consistency, the CAHAI-G revealed high intra-rater reliability. The data also suggest an adequate responsiveness of the CAHAI-G versions 9 and 13.

Psychometric evaluation of a patient-reported outcome measure in pancreatic exocrine insufficiency (PEI)

BACKGROUND/OBJECTIVES

Pancreatic exocrine insufficiency (PEI) is commonly caused by chronic pancreatitis (CP) or cystic fibrosis (CF). There are no PEI-specific patient-reported assessments of symptoms and impacts. The PEI Questionnaire (PEI-Q) was developed through qualitative research with PEI patients and expert clinical input. This study evaluated the psychometric properties of the PEI-Q.

METHODS

162 PEI patients (CF = 71 and CP = 91), 62 diarrhoea-specific irritable bowel syndrome (IBS-D) patients and 60 healthy controls completed the 26-item PEI-Q and the Gastrointestinal Quality of Life Index (GIQLI) at baseline. PEI patients completed the measures again two weeks later to assess the test-retest reliability of the PEI-Q. Analyses supported item reduction and scoring algorithm development, followed by psychometric evaluation.

RESULTS

Over 90% of PEI patients completed at least 23 of the 26 items at baseline. Item responses and clinical relevance supported retention of 18 items. Factor analysis supported a three-factor solution (abdominal symptoms, bowel movements, impacts) with adequate model fit. PEI-Q scores had good internal consistency (Cronbach's alpha: 0.77-0.82) and test-retest reliability (ICC: 0.73-0.87). Correlations between PEI-Q and GIQLI supported convergent validity. Known-groups and receiver operating characteristic analyses demonstrated that PEI-Q scores discriminated (p < 0.001) between differing PEI severities, and PEI patients and controls.

CONCLUSIONS

The PEI-Q has good validity and reliability. Results indicate that the PEI-Q could be used to aid identification and diagnosis of PEI, assist in the management of patients already diagnosed with PEI, ensuring correct and optimum treatment as well as enhance patient-clinician communication.

Chronic kidney disease predicts a lower probability of improvement in patient-reported experience measures among patients with fractures: a prospective multicenter cohort study

Patient-reported experience measures (PREMs) are integral component of care for fracture patients. Using a multicenter cohort, we showed that the presence of chronic kidney disease (CKD) attenuated the probability of PREM improvement in fracture patients.

INTRODUCTION

Assessing PREM can assist physicians in improving patients' experiences. Patients with CKD are at an increased risk of exhibiting poor PREM and developing fractures. We aimed to assess whether CKD influences the probability of PREM improvement during follow-up among patients with fractures.

METHODS

We prospectively enrolled patients with hip or vertebral fractures from different institutes into a fracture liaison service program. After registering clinical histories, they received a baseline PREM assessment based on EuroQol group-5 dimension content, including self-care, daily activity, and pain severity using a 5-point Likert scale. A follow-up PREM assessment was arranged 4 months later, and we evaluated whether baseline CKD was predictive of PREM improvement.

RESULTS

Among 593 fracture patients (18% with CKD), 37.3% and 62.7% presented with hip and vertebral fractures, respectively. Self-care, daily activity, and pain severity improved after follow-up in 32%, 27%, and 43% participants; those with CKD exhibited worse self-care ability and daily activity than those without. Multivariate logistic regression analyses showed that baseline CKD was significantly associated with lower possibility of improvement in daily activity (odds ratio [OR] 0.58, p = 0.049) and pain severity (OR 0.52, p = 0.01), and an insignificant change in the possibility of improvement in self-care ability (OR 0.61, p = 0.09).

CONCLUSIONS

The presence of CKD predicts a significantly lower probability of PREM improvement among fracture patients. An early emphasis on renal function during fracture care should be considered.

Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores

PURPOSE

Clinical outcome assessments (COAs) require evidence not only of reliability, validity, and ability to detect change, but also a definition of what constitutes a meaningful change on the instrument. The responder definition specifies the amount of change on the COA that may be interpreted as a treatment benefit and is critical for interpreting what constitutes a meaningful change on the COA scores. However, the literature that describes methods for developing and applying responder definitions can be difficult to navigate. Clear and concise guidelines regarding which methods to apply under what circumstances and how to interpret the results are lacking. This article provides a guide to the variety of available methods and issues that should be considered when establishing responder definitions for interpreting meaningful changes in COA scores.

METHODS

An overview is provided for selecting anchors, developing study designs, planning psychometric analyses, using psychometric results to set responder thresholds, and applying responder thresholds in demonstrating treatment efficacy.

RESULTS

There are a variety of anchor-based methods for consideration, but they all rely on a preference for strongly related and easily interpretable anchors. The benefits of applying multiple anchors and multiple analytic methods are discussed. The process of triangulation can synthesize results across multiple sources to gain confidence in a proposed responder definition. Though a link to meaningfulness from the patient's perspective is absent, distribution-based methods provide lower bound estimates of score precision and have a role in triangulation. Responder definitions are typically required within regulatory review, but their application may differ across clinical trial programs.

CONCLUSIONS

By careful planning of anchor selection, study design, and psychometric methods, COA researchers can establish defensible responder thresholds that ultimately aid patients and clinicians in making informed treatment decisions.

Confounding factors associated with oral mucositis assessment in patients receiving chemoradiotherapy for head and neck cancer

PURPOSE

The aim of this study was to identify patient-centered, mucositis-associated adverse impact factors and events that might confound physician-assessed oral mucositis (OM) in head and neck cancer (HNC) patients receiving chemoradiotherapy.

METHODS

This was a post hoc analysis of a previously conducted randomized trial to determine the efficacy of 5% phenylbutyrate mouthwash in preventing chemoradiotherapy-induced OM. This analysis identified patient-centered symptomatic, observable, and measurable factors that may confound physician scoring of the severity of OM during chemoradiotherapy. Confounding factors were then combined with physician-rated OM scores according to World Health Organization (WHO) and OM Assessment Scale (OMAS) criteria to investigate the therapeutic implications of OM treatment.

RESULTS

The original analysis found no significant differences between experimental and placebo groups with respect to the cumulative incidence of physician-recorded severe OM (WHO ≥3 or OMAS ≥2), patient-reported adverse events, and opioid use. However, patients in the experimental arm had relatively lower rates of OM-associated adverse clinical issues including unplanned short radiation breaks, skipping of chemotherapy, nausea/vomiting, late loss of body weight, and early opioid use, all of which could potentially interfere with physician-assessed OM scoring. When WHO OM grade (functional impact and pain), OMAS ulceration size (organic impact), and prolonged radiation treatment time (cancer treatment impact) were combined, there were significantly fewer interruptions of chemoradiotherapy treatment in symptomatic OM patients in the experimental compared to the placebo group. The benefits conferred by reducing the amount of chemoradiotherapy-related, OM-associated adverse impacts in the experimental group were reflected by better 5-year locoregional recurrence-free survival.

CONCLUSIONS

This exploratory study raises questions as to whether the severity reflected by physician-rated OM scores is in concordance with OM-induced adverse impacts on HNC patients. Further investigations are warranted to identify patient-related and cancer-associated symptom burdens that may affect tolerance, compliance, and outcome of chemoradiotherapy and confound the evaluation of therapeutic effects on chemoradiotherapy-induced OM.