Improving the touchscreen-based food approach-avoidance task: remediated block-order effects and initial findings regarding validity

How pre-processing decisions affect the reliability and validity of the approach-avoidance task: Evidence from simulations and multiverse analyses with six datasets

Reaction time (RT) data are often pre-processed before analysis by rejecting outliers and errors and aggregating the data. In stimulus-response compatibility paradigms such as the approach-avoidance task (AAT), researchers often decide how to pre-process the data without an empirical basis, leading to the use of methods that may harm data quality. To provide this empirical basis, we investigated how different pre-processing methods affect the reliability and validity of the AAT. Our literature review revealed 108 unique pre-processing pipelines among 163 examined studies. Using empirical datasets, we found that validity and reliability were negatively affected by retaining error trials, by replacing error RTs with the mean RT plus a penalty, and by retaining outliers. In the relevant-feature AAT, bias scores were more reliable and valid if computed with D-scores; medians were less reliable and more unpredictable, while means were also less valid. Simulations revealed bias scores were likely to be less accurate if computed by contrasting a single aggregate of all compatible conditions with that of all incompatible conditions, rather than by contrasting separate averages per condition. We also found that multilevel model random effects were less reliable, valid, and stable, arguing against their use as bias scores. We call upon the field to drop these suboptimal practices to improve the psychometric properties of the AAT. We also call for similar investigations in related RT-based bias measures such as the implicit association task, as their commonly accepted pre-processing practices involve many of the aforementioned discouraged methods. HIGHLIGHTS: • Rejecting RTs deviating more than 2 or 3 SD from the mean gives more reliable and valid results than other outlier rejection methods in empirical data • Removing error trials gives more reliable and valid results than retaining them or replacing them with the block mean and an added penalty • Double-difference scores are more reliable than compatibility scores under most circumstances • More reliable and valid results are obtained both in simulated and real data by using double-difference D-scores, which are obtained by dividing a participant's double mean difference score by the SD of their RTs.

An experimental test of integrating imagery with approach bias modification for alcohol: A cautionary tale

OBJECTIVES

In terms of dual process models, behaviour can be conceived of as the outcome of an interplay between reflective, top-down and impulsive, bottom-up processes. Behaviour change interventions may benefit from targeting both types of processes in a coherent way. One approach to this, in the context of reducing hazardous drinking, is to combine imagery involving real-life situations involving alcohol with the simple actions involved in Approach Bias Modification (ApBM), a form of Cognitive Bias Modification.

DESIGN

We developed and tested a version of this Imagery-enhanced Approach Bias Modification (IApBM) in an experimental design, with two independent factors: imagery versus control and ApBM versus control training components (N = 139).

METHODS

An effect of integrating the training factors was hypothesized on the alcohol-approach bias of an alcohol Approach-Avoidance Task. Further exploratory analyses were performed for the bias on alcohol-related Single Attribute Implicit Association Tests and on alcohol-related questionnaires. Finally, the psychometric properties of an imagery interference effect during training were explored.

RESULTS

Results showed no benefit of the training and in fact suggested a negative interaction in which combining the training components appeared to block reductions in craving effected by each in isolation. The reliability of the imagery-related interference effect was high and the effect was correlated with alcohol-related scales.

CONCLUSIONS

In conclusion, it appears that interference between training components decreases their individual effects when combining imagery and ApBM in the current way. The imagery-related interference effects that could be measured during training conditions may be useful as an implicit measure of automatic processes underlying hazardous drinking.