Receptivity to Instructional Feedback

Abstract. The purpose of this study was to report validity evidence for the instrument intended to measure receptivity to instructional feedback in a sample of secondary school students from Singapore ( N = 314). We tested a nested hierarchy of hypotheses for addressing the cross-group (i.e., gender) invariance and compared means on the receptivity to feedback subscales between gender groups. We also examined whether receptivity to feedback predicted student grades. The four-factor hypothesized model comprising experiential attitudes, instrumental attitudes, cognitive engagement, and behavioral engagement with feedback had a good model fit. Multi-group confirmatory factor analysis supported configural, metric, partial scalar, partial strict as well as variance and covariance invariance across gender groups. After controlling for gender, cognitive engagement, and experiential attitudes predicted increments in grades, suggesting evidence for discriminant validity among the receptivity factors as well as their relevance for the prediction of meaningful educational outcomes.

Assessing the measurement invariance and antecedents of legal cynicism in São Paulo, Zurich, and Montevideo

INTRODUCTION

This paper accomplishes two goals. First, we assesses the measurement invariance of legal cynicism among adolescents in São Paulo, Brazil, Montevideo, Uruguay, and Zurich, Switzerland. Second, we evaluate a series of social and individual antecedents that are expected to influence legal cynicism across contexts.

METHODS

This paper first evaluates the measurement invariance of legal cynicism using Multigroup Confirmatory Factor Analysis with three randomized clustered samples of adolescents in Zurich (n = 1447), São Paulo (n = 2680) and Montevideo (n = 2204). Second, we assessed the correlates for legal cynicism in each city using structural equation modelling techniques.

RESULTS

The results demonstrated metric invariance, but not scalar invariance among adolescents in São Paulo, Zurich, and Montevideo. We were able to establish partial measurement invariance for legal cynicism in São Paulo and Zurich, and therefore proceeded with the comparison of latent means and antecedents. The results show that on average legal cynicism is higher in Zurich, but that the size and strength of antecedents were similar across cities. Low self-control was by far the strongest correlate of legal cynicism.

CONCLUSIONS

Overall, our results suggest that current operationalizations of legal cynicism may not be rooted in social structural context and experiences with legal authorities, but rather reflect how individuals interpret legal boundaries and dispositions towards rule-breaking. Researchers must reconsider how legal cynicism fits into models of legal socialization, and whether developmental models of self-control may help us understand the origins and nature of legal cynicism, as it is currently measured.

Consequences of ignoring the response-shift and measure non-invariant items in sleep studies: an empirical data based simulation of the treatment effect of CBT-I on dysfunctional sleep beliefs

INTRODUCTION

Previous studies have shown that psychological interventions do not only improve patients' mental symptoms (i.e., true change) but may also change the internal standards patients use to evaluate their symptoms (i.e., response shifts). Although the response shifts could reflect patients' cognitive changes toward their disorders as the interventions aim to achieve, failing to differentiate them from the true change during data analyses could bias the research conclusions. Considering this issue is seldom discussed in sleep studies, this study thus examined the impacts of response-shift items in an intervention study of cognitive behavioral therapy for insomnia (CBT-I) via empirical-data based simulations.

METHOD

We used longitudinal measurement invariance tests to identify the items in an abbreviated version of the Dysfunctional Beliefs and Attitudes about Sleep Scale that are non-invariant (response shifted) against CBT-I based on data from 114 insomnia patients. The partial invariance model built accordingly was then used as a population model for simulations to examine the impacts of the response-shift items on follow-up paired t-tests.

RESULTS

Invariance tests indicate CBT-I would lift the intercept of one item in DBAS-10 and cause non-uniform calibrations in three items. The following up simulations showed that failing to exclude the intercept-lifted item from the calculations of the subscale scores would lower the probability of using paired t-test to correctly detect the treatment effect by up to 53%.

CONCLUSIONS

We recommend sleep researchers to consider the issues of response-shift when assessing sleep-related constructs in interventional studies for insomnia.

Are the Insomnia Severity Index and Pittsburgh Sleep Quality Index valid outcome measures for Cognitive Behavioral Therapy for Insomnia? Inquiry from the perspective of response shifts and longitudinal measurement invariance in their Chinese versions

OBJECTIVE

The purpose of this study was to examine whether the Insomnia Severity Index (ISI) and Pittsburgh Sleep Quality Index (PSQI) are valid outcome measures for Cognitive Behavioral Therapy for Insomnia (CBT-I). Specifically, we tested whether the factorial parameters of the ISI and the PSQI could remain invariant against CBT-I, which is a prerequisite to using their change scores as an unbiased measure of the treatment outcome of CBT-I.

METHODS

A clinical data set including scores on the Chinese versions of the ISI and the PSQI obtained from 114 insomnia patients prior to and after a 6-week CBT-I program in Taiwan was analyzed. A series of measurement invariance (MI) tests were conducted to compare the factorial parameters of the ISI and the PSQI before and after the CBT-I treatment program.

RESULTS

Most factorial parameters of the ISI remained invariant after CBT-I. However, the factorial model of the PSQI changed after CBT-I treatment. An extra loading with three residual correlations was added into the factorial model after treatment.

CONCLUSIONS

The partial strong invariance of the ISI supports that it is a valid outcome measure for CBT-I. In contrast, various changes in the factor model of the PSQI indicate that it may not be an appropriate outcome measure for CBT-I. Some possible causes for the changes of the constructs of the PSQI following CBT-I are discussed.