Using structural equation modeling to investigate change and response shift in patient-reported outcomes: practical considerations and recommendations

How to select and understand guidelines for patient-reported outcomes: a scoping review of existing guidance

BACKGROUND

Over the past few decades, patient-reported outcomes (PROs) have been used to understand patient health conditions better. Therefore, numerous PRO measures (questionnaires) and guidelines or guidance have been developed. However, it is challenging to select target guidance from among the many available guidance and to understand the chosen guidance. This study comprehensively collected the existing PRO guidance for clinical trials or studies and practices to support novice PRO users in academia, industry, clinical practice, and regulatory and reimbursement decision-making.

METHODS

For the scoping review, we searched the MEDLINE, Embase, Google Books, WorldCat, and the National Library of Medicine (NLM) Bookshelf databases from 2009 to 2023. The eligibility criteria were PRO guidance for clinical trials, clinical practice, or application such as health technology assessment. Those guidance cover aspects such as quality of life (QOL), PRO, health-related QOL, health state utilities, psychometric requirements, implementation methods, analysis and interpretation, or clinical practice applications. After the systematic search, three researchers individually reviewed the collected data, and the reviewed articles and books were scrutinized using the same criteria.

RESULTS

We collected the PRO guidance published in articles and books between 2009 and 2023. From the database searches, 1,455 articles and 387 books were identified, of which one book and 33 articles were finally selected. The collected PRO guidance was categorized into the adoption of PRO measures, design and reporting of trials or studies using PROs, implementation of PRO evaluation in clinical trials or studies or clinical practice, analysis and interpretation of PROs, and application of PRO evaluation. Based on this categorization, we suggest the following for novices: When selecting guidance, novices should clarify the "place" and "purpose" where the guidance will be used. Additionally, they should know that the terminology related to PRO and the scope and expectations of PROs vary by "places" and "purposes".

CONCLUSIONS

From this scoping review of existing PRO guidance, we provided summaries and caveats to assist novices in selecting guidance that fits their purpose and understanding it.

Social and therapeutic decline earlier than physical and psychological domains after discharge in heart failure patients: A patient-reported outcome measurements of latent transition analysis

Background

Among patients with chronic heart failure (CHF), response shifts are common in assessing treatment effects. However, few studies focused on potential response shifts in these patients.

Materials and methods

Data of CHF patient-reported outcome measures (PROMs) were obtained from three hospitals in Shanxi, China, from 2017 to 2019. A total of 497 patients were enrolled and followed up at 1 month and 6 months after discharge. Latent transition analysis (LTA) was employed to determine the longitudinal transition trajectories of latent subtypes in CHF patients in the physiological, psychological, social, and therapeutic domains.

Results

The patients were divided into high- and low-level groups in the four domains according to the LTA. One month after discharge, the physiological and psychological domains improved, while the social and therapeutic domains remained unchanged. Six months after discharge, the former remained stable, but the latter deteriorated. The factors affecting the state transition in four domains were as follows. The influencing factor of the physiological domains are gender, age, tea consumption, smoking, alcohol consumption, physical activity, and light diet; those of the psychological domain are gender, occupation, smoking, alcohol consumption, and physical activity; those of the social domains are age; those of the therapeutic domains are education and income.

Conclusion

The disease status of CHF patients has shifted over time. Risk factors accelerate the deterioration of patients’ condition. Furthermore, the risk factors of social and therapeutic domains deteriorate patients’ condition faster than those of physiological and psychological domains. Therefore, individualized intervention programs should be given for CHF patients who may be transferred to the low-level groups to maintain the treatment effect and improve the prognosis.

Midwives' empathy and shared decision making from women's perspective - sensitivity of an assessment to compare quality of care in prenatal and obstetric care

Background

For quality-oriented evaluation of prenatal and obstetric care, it is important to systematically consider the perspective of the women receiving care in order to comprehensively assess and optimize quality in a woman-centered manner. Empathy and Shared Decision Making (SDM) are essential components of woman-centered midwifery care. The aim of the study was to analyze measurement invariance of the items of the Consultation and Relational Empathy (CARE) and Shared Decision Making-Questionnaire (SDM-Q-9) scales depending on the prenatal versus obstetric care setting.

Methods

One hundred fifty women retrospectively assessed aspects of woman-centered midwifery care in both prenatal and obstetric care setting. The birth of the child was a maximum of 12 months ago. A structural equation modelling approach was adopted to separate true effects from response shift (RS) effects depending on care setting. The latter were analyzed in terms of recalibration (changing women’s internal measurement standards), Reprioritization (changing associations of items and construct) as well as Reconceptualization (redefining the target construct).

Results

A response shift model was identified for both assessments (pregnancy/birth: CFI = .96/.96; SRMR = .046/.051). At birth, both scales indicated lower quality of care compared with prenatal care (SDM-Q-9-M/CARE-8-M:|d| = 0.190/0.392). Although no reconceptualization is required for the items of both scales, RS effects are evident for individual items. Due to recalibration and reprioritization effects, the true differences in the items are partly underestimated (SDM-Q-9-M/CARE-8-M: 3/2 items) or overestimated (4/2 items).

Conclusion

The structure of the constructs SDM and Empathy, indicating woman-centered midwifery care, are moderated by the care settings. To validly assess midwives’ empathy and shared decision making from women’s perspective, setting-dependent response shift effects have to be considered. The proven item-specific response effects contribute to a better understanding of construct characteristics in woman-centered care by midwives during pregnancy and childbirth.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12884-022-05041-y.

Performance of a Rasch-based method for group comparisons of longitudinal change and response shift at the item level in PRO data: A simulation study

The growing interest in patient perception and experience in healthcare has led to an increase in the use of patient-reported outcomes (PRO) data. However, chronically ill patients may regularly adapt to their disease and, as a consequence, might change their perception of the PRO being measured. This phenomenon named response shift (RS) may occur differently depending on clinical and individual characteristics. The RespOnse Shift ALgorithm at the Item level (ROSALI), a method for RS analysis at the item level based on Rasch models, has recently been extended to explore heterogeneity of item-level RS between two groups of patients. The performances of ROSALI in terms of RS detection at the item level and biases of estimated differences in latent variable means were assessed. A simulation study was performed to investigate four scenarios: no RS, RS in only one group, RS affecting both groups either in a similar or a different way. Performances of ROSALI were assessed using rates of false detection of RS when no RS was simulated and a set of criteria (presence of RS, correct identification of items and groups affected by RS) when RS was simulated. Rates of false detection of RS were low indicating that ROSALI satisfactorily prevents from mistakenly inferring RS. ROSALI is able to detect RS and identify the item and group(s) affected when RS affects all response categories of an item in the same way. The performances of ROSALI depend mainly on the sample size and the degree of heterogeneity of item-level RS.

Longitudinal measurement invariance of the international spinal cord injury quality of life basic data set (SCI-QoL-BDS) during spinal cord injury/disorder inpatient rehabilitation

Purpose

This study aimed at testing the internal consistency and longitudinal measurement invariance of a brief quality of life questionnaire—the spinal cord injury quality of life basic data set (SCI-QoL-BDS)—among individuals with spinal cord injury/disorder undergoing first inpatient rehabilitation.

Methods

Longitudinal data from the Swiss spinal cord injury inception cohort study were used. Participants (n = 218) completed the SCI-QoL-BDS at one and three months post injury and at discharge. The SCI-QoL-BDS consists of three items assessing satisfaction with life as a whole, physical health, and psychological health. Internal consistency was examined at each time point and longitudinal measurement invariance was tested using longitudinal confirmatory factor analysis.

Results

Internal consistency coefficients ranged between .82 and .90. The confirmatory factor analysis revealed invariance of the factor structure and of all factor loadings across time. Additionally, all item intercepts except the one of satisfaction with physical health were invariant across time, suggesting partial intercept invariance of the SCI-QoL-BDS. Indeed, a response shift was observed in satisfaction with physical health. This item was evaluated more negatively in the early phase of inpatient rehabilitation, indicating the change of the evolving physical situation after the onset of a spinal cord injury.

Conclusion

The SCI-QoL-BDS is a consistent and valid measure to assess quality of life among individuals undergoing first spinal cord injury/disorder inpatient rehabilitation. However, we recommend using latent variable frameworks instead of mean scores when examining longitudinal changes in the measure to account for potential response shift.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-021-03027-5.

Service Quality in Early Intervention Centres: An Analysis of Its Influence on Satisfaction and Family Quality of Life

Early Intervention (EI) is a set of interventions focused on responding to the needs of children with or at risk of developmental problems. This study aimed to investigate the relationships between the perceived quality of service, satisfaction and family quality of life.

METHODS

to conduct a multi-centre, transversal study with a non-probabilistic sample. The participants (N = 1551) were families from 24 Early Intervention Centres (EICs) located in Spain.

RESULTS

The results indicated an adequate fit of the measurement and structural models, with the latter showing a capacity of 73% to predict the family quality of life. The structural model established that the perceived service quality was a positive and significant predictor of satisfaction (β = 0.85; p < 0.001). Both the perceived quality of service (β = 0.28; p < 0.001) and satisfaction (β = 0.33; p < 0.001) obtained a similar positive and significant relationship with family quality of life, which was slightly stronger than satisfaction.

CONCLUSIONS

This study provided a better understanding of the importance of the services offered in EICs and their influence on the satisfaction and family quality of life of their users. Thus, delving into these relationships was highly relevant for decision-making in the context of EI.

Response shift after coronary revascularization

PURPOSE

The aims of this study were to investigate (1) the extent to which response shift occurs among patients with coronary artery disease (CAD) after coronary revascularization, (2) whether the assessment of changes in health-related quality of life (HRQoL), controlled for response shift, yield more valid estimates of changes in HRQoL, as indicated by stronger associations with criterion measures of change, than without controlling for response shift, and (3) if occurrences of response shift are related to patient characteristics.

METHODS

Patients with CAD completed the SF-36 and the Seattle Angina Questionnaire (SAQ7) at baseline and 3 months after coronary revascularization. Sociodemographic, clinical and psychosocial variables were measured with the patient version of the New York Heart Association-class, Subjective Significance Questionnaire, Reconstruction of Life Events Questionnaire (RE-LIFE), and HEXACO personality inventory. Oort's Structural Equation Modeling (SEM) approach was used to investigate response shift.

RESULTS

191 patient completed questionnaires at baseline and at 3 months after treatment. The SF-36 showed recalibration and reprioritization response shift and the SAQ7 reconceptualization response shift. Controlling for these response shift effects did not result in more valid estimates of change. One significant association was found between reprioritization response shift and complete integration of having CAD into their life story, as indicated by the RE-LIFE.

CONCLUSION

Results indicate response shift in HRQoL following coronary revascularization. While we did not find an impact of response shift on the estimates of change, the SEM approach provides a more comprehensive insight into the different types of change in HRQoL following coronary revascularization.

Critical examination of current response shift methods and proposal for advancing new methods

Purpose

This work is part of an international, interdisciplinary initiative to synthesize research on response shift in results of patient-reported outcome measures. The objective is to critically examine current response shift methods. We additionally propose advancing new methods that address the limitations of extant methods.

Methods

Based on literature reviews, this critical examination comprises design-based, qualitative, individualized, and preference-based methods, latent variable models, and other statistical methods. We critically appraised their definition, operationalization, the type of response shift they can detect, whether they can adjust for and explain response shift, their assumptions, and alternative explanations. Overall limitations requiring new methods were identified.

Results

We examined 11 methods that aim to operationalize response shift, by assessing change in the meaning of one’s self-evaluation. Six of these methods distinguish between change in observed measurements (observed change) and change in the construct that was intended to be measured (target change). The methods use either (sub)group-based or individual-level analysis, or a combination. All methods have underlying assumptions to be met and alternative explanations for the inferred response shift effects. We highlighted the need to address the interpretation of the results as response shift and proposed advancing new methods handling individual variation in change over time and multiple time points.

Conclusion

No single response shift method is optimal; each method has strengths and limitations. Additionally, extra steps need to be taken to correctly interpret the results. Advancing new methods and conducting computer simulation studies that compare methods are recommended to move response shift research forward.

Supplementary Information

The online version of this article (10.1007/s11136-020-02755-4) contains supplementary material, which is available to authorized users.