Quantitative challenges facing patient-centered outcomes research

Psychometric evaluation of a caregiver diary for the assessment of symptoms of respiratory syncytial virus

BACKGROUND

There are no clinical outcome assessment (COA) tools developed in accordance with Food and Drug Administration (FDA) guidance suitable for the evaluation of symptoms associated with respiratory syncytial virus (RSV) infection among infants. The Gilead RSV Caregiver Diary (GRCD) is being developed to fulfill this need; the present research evaluates the GRCD and documents its reliability, validity, and responsiveness among children < 24 months of age with acute RSV infection.

METHODS

A prospective, observational study was conducted in the United States during the 2014-2015 northern hemisphere winter season. Subjects were < 24-month, full-term, previously healthy infants with confirmed RSV infection and ≤5 days of symptoms. The GRCD was completed twice daily for 14 days by caregivers. Additional data were collected during the initial visit, subsequent visits, and end-of-study interview. Test-retest reliability (kappa and intraclass correlation coefficients [ICCs]), construct validity (correlations and factor analyses), discriminating ability (analyses of variance and chi-square), and responsiveness (effect sizes and standardized response means) were evaluated.

RESULTS

A total of 103 subjects were enrolled (mean age 7.4 ± 5.3 months). GRCD items were grouped into different subscales according to question content, which, with the exception of the behavior impact domain (ICC = 0.43), demonstrated internal consistency (alphas = 0.78-0.94) and test-retest reliability (ICCs = 0.77-0.94). Hypothesized correlations with parent global ratings of RSV severity ranged from 0.45 to 0.70 and provided support for construct validity. Support for discriminating ability was limited. Effect sizes ranged from - 1.48 to - 4.40, indicating the GRCD was responsive to change.

CONCLUSIONS

These psychometric analyses support the validity, reliability, and responsiveness of the GRCD for assessing RSV symptoms in children < 24 months of age.

Maintaining primacy of the patient perspective in the development of patient-centered patient reported outcomes

The objectives of this study were to describe and demonstrate a new model of developing patient reported outcomes (PROs) that are patient-centered, and to test the hypothesis that following this model would result in a qualitatively different PRO than if the typical PRO development model were followed. The typical process of developing PROs begins with an initial list of signs or symptoms originating from clinicians or PRO developers; patient validation of this list ensures that the list (i.e., the new PRO) is interpretable by patients, but not that patient perspectives are central or even represented. The new model begins with elicitation from clinicians and patients independently and separately. These perspectives are formally analyzed qualitatively, and the results are iteratively integrated by researchers, supporting clinical relevance and patient centeredness. We describe the application of this new model to the development of a PRO for urinary signs and symptoms in individuals with neuropathic bladder, and test the hypothesis that the two processes generate qualitatively different instruments using a national validation sample of 300 respondents. Of its 29 items, the new instrument included 13 signs/symptoms derived from existing clinical practice guidelines, with 16 others derived from the patient/focus groups. The three most-endorsed items came from the patients, and the three least-endorsed items came from clinical guidelines. Thematic qualitative analysis of the elicitation process, as well as the results from our national sample, support the conclusion that the new model yields an instrument that is clinically interpretable, but more patient-centered, than the typical model would have done in this context.

Advances in the Evaluation of Longitudinal Construct Validity of Clinical Outcome Assessments

Historically, the assessment of longitudinal construct validity in the field of psychosocial measurement involved defining hypotheses and calculating correlation coefficients using scores based on 2 measures at 2 or more time points. In the context of patient-reported outcomes, this evolved into sensitivity to change and responsiveness, including the computation of effect size estimates of change, standardized response means, and indices such as Guyatt's statistic. Cross-sectional analyses or analyses based on 2 time points have been the standard practice. Evolving conceptualizations have incorporated more than 2 time points and have included depictions of individual trajectories of change in multiple measures, structural equation models, and mixed modeling techniques. The focus of this article is on methods to evaluate longitudinal construct validity. We describe a sample of these methods and provide considerations and recommendations for designing a thoughtful longitudinal construct validity evaluation of clinical outcome assessments.