Examining reactivity to the measurement of physical activity and sedentary behavior among women in midlife with elevated risk for cardiovascular disease

OBJECTIVES

To estimate the extent of reactivity to measurement of physical activity (PA) and sedentary behavior among women in midlife with elevated risk for cardiovascular disease (CVD).

DESIGN

Secondary analysis of a 10-day observational study of PA and sedentary behavior.

MAIN OUTCOME MEASURES

PA (steps, minutes of light PA, total minutes of moderate-to- vigorous PA [MVPA]) and percent time in sedentary behavior per day were assessed using ActiGraph GT3X tri-axial accelerometers in 75 women in midlife with elevated CVD risk (e.g. hypertension; MAge = 51.61, MBMI = 34.02 kg/m2). Two-level multilevel models were used to test for evidence of reactivity, with the addition of random effects to test for evidence of individual differences in observed trends.

RESULTS

All outcomes showed linear trends across days (ps < 0.001), though this masked what appeared to be meaningful dropoff after Day 1 or Day 2 (with little difference between subsequent days; srs ranging from 0.15 to 0.32). The random effect was significant only for percent time in sedentary behavior (χ2[1] = 10.40, p = 0.02).

CONCLUSIONS

Consistent small to medium effects were found for all PA and sedentary behavior outcomes, underscoring the importance of considering measurement reactivity in populations with elevated CVD risk.

Exploring the Role of Attentional Reorienting in the Reactive Effects of Judgments of Learning on Memory Performance

Making judgments of learning (JOLs) while studying related word pairs can enhance performance on tests that rely on cue-target associations (e.g., cued recall) compared to studying alone. One possible explanation for this positive JOL reactivity effect is that the prompt to make JOLs, which typically occurs halfway through the presentation of each pair, may encourage learners to devote more attention to the pair during the second half of the encoding episode, which may contribute to enhanced recall performance. To investigate this idea, an online sample of participants (Experiment 1) and undergraduate students (Experiment 2) studied a set of moderately related word pairs (e.g., dairy-cow) in preparation for a cued recall test. Some participants made JOLs for each pair halfway through the presentation, whereas other participants did not. Also, some participants were presented with a fixation point halfway through the presentation, whereas other participants were not. The goal of this fixation point was to simulate the possible "reorienting" effect of a JOL prompt halfway through each encoding episode. In both an unsupervised online context and a supervised laboratory context, cued recall performance was higher for participants who made JOLs compared to those who did not make JOLs. However, presenting a fixation point halfway through the presentation of each pair did not lead to reactive effects on memory. Thus, JOLs are more effective than a manipulation that reoriented participants to the word pairs in another way (i.e., via a fixation point), which provides some initial evidence that positive reactivity for related pairs is not solely driven by attentional reorienting during encoding.

A Novel Experience Sampling Method Tool Integrating Momentary Assessments of Cognitive Biases: Two Compliance, Usability, and Measurement Reactivity Studies

BACKGROUND

Experience sampling methods (ESMs) are increasingly being used to study ecological emotion dynamics in daily functioning through repeated assessments taken over several days. However, most of these ESM approaches are only based on self-report assessments, and therefore, studies on the ecological trajectories of their underlying mechanisms are scarce (ie, cognitive biases) and require evaluation through experimental tasks. We developed a novel ESM tool that integrates self-report measures of emotion and emotion regulation with a previously validated app-based cognitive task that allows for the assessment of underlying mechanisms during daily functioning.

OBJECTIVE

The objective of the study is to test this new tool and study its usability and the possible factors related to compliance with it in terms of latency and missing responses. Among the compliance predictors, we considered psychological and time-related variables, as well as usability, measurement reactivity, and participants' satisfaction with the tool.

METHODS

We conducted 2 extensive ESM studies-study 1 (N=84; a total of 3 assessments per day for 5 days) and study 2 (N=135; a total of 3 assessments per day for 10 days).

RESULTS

In both studies, participants found the tool highly usable (average usability score >81). By using mixed regression models, we found both common and specific results for the compliance predictors. In both study 1 and study 2, latency was significantly predicted by the day (P<.001 and P=.003, respectively). Participants showed slower responses to the notification as the days of the study progressed. In study 2 but not in study 1, latency was further predicted by individual differences in overload with the use of the app, and missing responses were accounted for by individual differences in stress reactivity to notifications (P=.04). Thus, by using a more extensive design, participants who experienced higher overload during the study were characterized by slower responses to notifications (P=.01), whereas those who experienced higher stress reactivity to the notification system were characterized by higher missing responses.

CONCLUSIONS

The new tool had high levels of usability. Furthermore, the study of compliance is of enormous importance when implementing novel ESM methods, including app-based cognitive tasks. The main predictors of latency and missing responses found across studies, specifically when using extensive ESM protocols (study 2), are methodology-related variables. Future research that integrates cognitive tasks in ESM designs should take these results into consideration by performing accurate estimations of participants' response rates to facilitate the optimal quality of novel eHealth approaches, as in this study.

Investigating memory reactivity with a within-participant manipulation of judgments of learning: support for the cue-strengthening hypothesis

When learners make judgments of learning (JOLs) for some word pairs but not others, how and why is recall performance affected? Participants studied related and unrelated word pairs and made JOLs for a randomly selected half of the pairs. We evaluated two hypotheses. The changed-goal hypothesis states that making JOLs leads learners to notice differences in pair difficulty and to change their learning goal. Because JOLs are manipulated within participants, such a goal change should influence how all (judged or non-judged) pairs are processed on the list, which should lead to no JOL reactivity. The cue-strengthening hypothesis predicts greater positive reactivity (i.e., higher recall for judged versus non-judged pairs) for related than unrelated pairs, because making a JOL strengthens the relationship between the two words in a pair, which would be more beneficial for pairs with an a priori relationship. Across experiments, we found positive reactivity for both related and unrelated pairs (albeit to a lesser degree for the latter). We also found no evidence that learners make qualitative changes in their reported strategy use when judging pairs. Making JOLs for some pairs on a list influenced memory performance and the pattern of reactivity provided support for the cue-strengthening hypothesis.

Changes in mood after screening for antenatal anxiety and depression

BACKGROUND

Screening programmes with referral are a valuable strategy for mitigating consequences of perinatal depression on mothers and their families. The effectiveness of these screening programmes needs to be measured. One potential problem in assessing outcomes is measurement reactivity where the actual measure results in changes in the people being measured.

AIM

The aim of this article is to explain the mechanisms and circumstances by which measurement reactivity occurred in a sample of antenatal women who participated in a screening interview.

METHODS

Fifty-five women who participated in an antenatal screening interview in their second trimester were re-interviewed in their third trimester. These qualitative interviews were conducted between September 2015 and April 2016.

RESULTS

The qualitative data suggested that measurement reactivity occurred through mechanisms such as the disclosure, gaining self-knowledge, validation of experiences, and personal agency which resulted in them seeking out support from others.

CONCLUSION

Although the screening interview appeared to improve women's outcomes, this may have occurred through measurement reactivity. This needs to be considered when designing studies that aim to assess the effectiveness of screening with intervention for antenatal depression.

Conducting ethical internet-based research with vulnerable populations: a qualitative study of bereaved participants' experiences of online questionnaires

Background: Bereavement can be considered a potentially traumatic experience, and concerns have been raised about conducting grief research responsibly online. Objective: Given that online research introduces new methodological opportunities and challenges, we aimed to develop a greater understanding of how bereaved individuals experience participation in online research. Method: One day after participation in an online grief study, 876 participants, bereaved on average for 40 months, received a 'check-in' email to support well-being and offer further contact if needed. Although not explicitly asked to respond if no help was needed, 300 participants sent email replies, with only six requesting support. These responses were analysed qualitatively using content analysis. Results: Results suggested that participants found it acceptable to be asked about their grief and while difficult emotions were frequently described in response to the questionnaires, these reactions were temporary. A range of positive reactions was also reported, including new realizations arising from completing the research and changes in thinking related to grief. Participants also wrote about their appreciation for the study and how it was carried out, as well as a desire to contribute more to the study and to help others in a similar position. Conclusions: We suggest that the use of the check-in email to support well-being following study completion, along with advice on preparing to take part, contributed to positive experiences of participation and we recommend these strategies for future studies. These findings could allay clinical concerns about conducting online research with vulnerable populations, as well as raising questions about the possible therapeutic impact of measurement.

Are Improvements in Measured Performance Driven by Better Treatment or "Denominator Management"?

BACKGROUND

Process measures of healthcare quality are usually formulated as the number of patients who receive evidence-based treatment (numerator) divided by the number of patients in the target population (denominator). When the systems being evaluated can influence which patients are included in the denominator, it is reasonable to wonder if improvements in measured quality are driven by expanding numerators or contracting denominators.

OBJECTIVE

In 2003, the US Department of Veteran Affairs (VA) based executive compensation in part on performance on a substance use disorder (SUD) continuity-of-care quality measure. The first goal of this study was to evaluate if implementing the measure in this way resulted in expected improvements in measured performance. The second goal was to examine if the proportion of patients with SUD who qualified for the denominator contracted after the quality measure was implemented, and to describe the facility-level variation in and correlates of denominator contraction or expansion.

DESIGN

Using 40 quarters of data straddling the implementation of the performance measure, an interrupted time series design was used to evaluate changes in two outcomes.

PARTICIPANTS

All veterans with an SUD diagnosis in all VA facilities from fiscal year 2000 to 2009.

MAIN MEASURES

The two outcomes were 1) measured performance-patients retained/patients qualified and 2) denominator prevalence-patients qualified/patients with SUD program contact.

KEY RESULTS

Measured performance improved over time (P < 0.001). Notably, the proportion of patients with SUD program contact who qualified for the denominator decreased more rapidly after the measure was implemented (p = 0.02). Facilities with higher pre-implementation denominator prevalence had steeper declines in denominator prevalence after implementation (p < 0.001).

CONCLUSIONS

These results should motivate the development of measures that are less vulnerable to denominator management, and also the exploration of "shadow measures" to monitor and reduce undesirable denominator management.

Is intensive measurement of body image reactive? A two-study evaluation using Ecological Momentary Assessment suggests not

Intensive assessment methods (e.g., Ecological Momentary Assessment [EMA]) are increasingly used to capture body image experiences in daily life. One concern with EMA is multiple assessments may increase reactivity to internal or external cues, potentially biasing measurement. Reactivity to EMA was evaluated in two studies (Study 1: N=63 female undergraduates, Study 2: N=131 women with high body dissatisfaction/disordered eating). Participants completed five daily surveys on handheld computers for 1-2 weeks and body image-related questionnaires at the start and end of each study. Results showed no systematic changes in pre- and post-EMA measures or momentary EMA reports, suggesting women were not reactive to the EMA protocols. Completing 1-2 weeks of EMA does not appear to affect body dissatisfaction, mood, or attitudes in non-clinical or at-risk samples of women. These studies provide evidence that EMA methods can be used to assess real-world body image experiences without undue concern about measurement reactivity.