Promises and Perils of Experimentation: The Mutual-Internal-Validity Problem

An experimental manipulation of the value of effort

People who take on challenges and persevere longer are more likely to succeed in life. But individuals often avoid exerting effort, and there is limited experimental research investigating whether we can learn to value effort. We developed a paradigm to test the hypothesis that people can learn to value effort and will seek effortful challenges if directly incentivized to do so. We also dissociate the effects of rewarding people for choosing effortful challenges and performing well. The results provide limited evidence that rewarding effort increased people's willingness to choose harder tasks when rewards were no longer offered (near transfer). There was also mixed evidence that rewarding effort increased willingness to choose harder tasks in another unrelated and unrewarded task (far transfer). These heterogeneous results highlight the need for further research to understand when this paradigm may be the most effective for increasing and generalizing the value of effort.

A Meta-Analytic Review of the Validity of the Tangram Help/Hurt Task

The Tangram Help/Hurt Task (THHT) allows participants to help another participant win a prize (by assigning them easy tangrams), to hurt another participant by preventing them from winning the prize (by assigning them difficult tangrams), or to do neither (by assigning them medium tangrams) in offline or online studies. Consistent with calls for continued evidence supporting psychological measurement, we conducted a meta-analytic review of the THHT that included 52 independent studies involving 11,060 participants. THHT scores were associated with helping and hurting outcomes in theoretically predicted ways. Results showed that THHT scores were not only associated with short-term (experimental manipulations, state measures) and long-term (trait measures) helping and hurting outcomes, but also with helping and harming intentions. We discuss the strengths and limitations of the THHT relative to other laboratory measures of prosocial behavior and aggression, discuss unanswered questions about the task, and offer suggestions for the best use of the task.

When and Why Do People Accept Public-Policy Interventions? An Integrative Public-Policy-Acceptance Framework

The successful introduction of public policies to prompt behavior change hinges on the degree to which citizens endorse the proposed policies. Although there is a large body of research on psychological determinants of public policy acceptance, these determinants have not yet been synthesized into an integrative framework that proposes hypotheses about their interplay. In this article, we develop a review-based, integrative public-policy-acceptance framework that introduces the desire for governmental support as a motivational foundation in public-policy acceptance. The framework traces the route from problem awareness to policy acceptance and, ultimately, policy compliance. We propose this relationship to be mediated by a desire for governmental support. We integrate numerous key variables assumed to qualify the relationship between problem awareness and the desire for governmental support, such as control attributions, trust, and value fit, as well as the relationship between the desire for governmental support and policy acceptance, such as perceived policy effectiveness, intrusiveness, and fairness. We exemplify the use of the proposed framework by applying it to climate policies.

The scientific value of explanation and prediction

Deep neural network models have revived long-standing debates on the value of explanation versus prediction for advancing science. Bowers et al.'s critique will not make these models go away, but it is likely to prompt new work that seeks to reconcile explanatory and predictive models, which could change how we determine what constitutes valuable scientific knowledge.

Keeping track of reality: embedding visual memory in natural behaviour

Since immersive virtual reality (IVR) emerged as a research method in the 1980s, the focus has been on the similarities between IVR and actual reality. In this vein, it has been suggested that IVR methodology might fill the gap between laboratory studies and real life. IVR allows for high internal validity (i.e., a high degree of experimental control and experimental replicability), as well as high external validity by letting participants engage with the environment in an almost natural manner. Despite internal validity being crucial to experimental designs, external validity also matters in terms of the generalizability of results. In this paper, we first highlight and summarise the similarities and differences between IVR, desktop situations (both non-immersive VR and computer experiments), and reality. In the second step, we propose that IVR is a promising tool for visual memory research in terms of investigating the representation of visual information embedded in natural behaviour. We encourage researchers to carry out experiments on both two-dimensional computer screens and in immersive virtual environments to investigate visual memory and validate and replicate the findings. IVR is valuable because of its potential to improve theoretical understanding and increase the psychological relevance of the findings.

A New Way to Think About Internal and External Validity

Researchers have been concerned with internal and external validity for decades, and the discussion continues. The present proposal is that there are less important and more important senses in which one can interpret internal and external validity, and these can be integrated with a taxonomy that includes theoretical, auxiliary, statistical, and inferential assumptions. The integration sheds light on recent exchanges in the literature on validity and suggests that the vaunted internal-external validity trade-off is false for more important senses of internal and external validity; internal and external validity increase or decrease together when there is an emphasis on underlying theories. Finally, the integration implies the desirability of some changes in typical research advice and practice.

Improving the Generalizability of Behavioral Science by Using Reality Checks: A Tool for Assessing Heterogeneity in Participants' Consumership of Study Stimuli

In attempting to draw bigger conclusions, researchers in psychology open their labs to more diverse groups of people. Yet even the most far-reaching theories must be tested with specific stimuli, materials, and methodology. To the extent that a study's stimuli are familiar beyond the lab to groups of people writ large, an experiment is said to have mundane realism-a type of external validity. We propose that an experiment's stimuli will vary in their relevance to each individual participant (such as how much they consume the stimuli outside the lab) and can be assessed using a tool: reality checks. We found that accounting for a study's mundane realism, at the individual level, significantly altered a study's results-which we found to be the case in testing well-established findings in psychology and behavioral economics. Our work suggests that measuring mundane realism (in addition to creating it) is a useful way of testing effects in psychology among the participants for whom the studies' scenarios and decisions will matter most outside of the lab.

Countering Antivax Misinformation via Social Media: Message-Testing Randomized Experiment for Human Papillomavirus Vaccination Uptake

BACKGROUND

Suboptimal adolescent human papillomavirus (HPV) vaccination rates have been attributed to parental perceptions of the HPV vaccine. The internet has been cited as a setting where misinformation and controversy about HPV vaccination have been amplified.

OBJECTIVE

We aimed to test message effectiveness in changing parents' attitudes and behavioral intentions toward HPV vaccination.

METHODS

We conducted a web-based message-testing experiment with 6 control messages and 25 experimental messages and 5 from each of the 5 salient themes about HPV vaccination (theme 1: safety, side effects, risk, and ingredient concerns and long-term or major adverse events; theme 2: distrust of the health care system; theme 3: HPV vaccine effectiveness concerns; theme 4: connection to sexual activity; and theme 5: misinformation about HPV or HPV vaccine). Themes were identified from previous web-based focus group research with parents, and specific messages were developed by the study team using content from credible scientific sources. Through an iterative process of message development, the messages were crafted to be appropriate for presentation on a social media platform. Among the 1713 participants recruited via social media and crowdsourcing sites, 1043 eligible parents completed a pretest survey questionnaire. Participants were then randomly assigned to 1 of the 31 messages and asked to complete a posttest survey questionnaire that assessed attitudes toward the vaccine and perceived effectiveness of the viewed message. A subgroup of participants (189/995, 19%) with unvaccinated children aged 9 to 14 years was also assessed for their behavioral intention to vaccinate their children against HPV.

RESULTS

Parents in the experimental group had increased positive attitudes toward HPV vaccination compared with those in the control group (t969=3.03, P=.003), which was associated with increased intention to vaccinate among parents of unvaccinated children aged 9 to 14 years (r=1.14, P=.05). At the thematic level, we identified 4 themes (themes 2-5) that were relatively effective in increasing behavioral intentions by positively influencing attitudes toward the HPV vaccine (χ25=5.97, P=.31, root mean square error of approximation [RMSEA]=0.014, comparative fit index [CFI]=0.91, standardized root mean square residual [SRMR]=0.031). On the message level, messages that provided scientific evidence from government-related sources (eg, the Centers for Disease Control and Prevention) and corrected misinformation (eg, "vaccines like the HPV vaccine are simply a way for pharmaceutical companies to make money. That isn't true") were effective in forming positive perceptions toward the HPV vaccination messages.

CONCLUSIONS

Evidence-based messages directly countering misinformation and promoting HPV vaccination in social media environments can positively influence parents' attitudes and behavioral intentions to vaccinate their children against HPV.

Cognitive effort for self, strangers, and charities

Effort is aversive and often avoided, even when earning benefits for oneself. Yet, people sometimes work hard for others. How do people decide who is worth their effort? Prior work shows people avoid physical effort for strangers relative to themselves, but invest more physical effort for charity. Here, we find that people avoid cognitive effort for others relative to themselves, even when the cause is a personally meaningful charity. In two studies, participants repeatedly decided whether to invest cognitive effort to gain financial rewards for themselves and others. In Study 1, participants (N = 51; 150 choices) were less willing to invest cognitive effort for a charity than themselves. In Study 2, participants (N = 47; 225 choices) were more willing to work cognitively for a charity than an intragroup stranger, but again preferred cognitive exertion that benefited themselves. Computational modeling suggests that, unlike prior physical effort findings, cognitive effort discounted the subjective value of rewards linearly. Exploratory machine learning analyses suggest that people who represented others more similarly to themselves were more willing to invest effort on their behalf, opening up new avenues for future research.

Whither Inhibition?

Inhibition is considered a process essential to goal pursuit and as a result has become a central construct in many disciplines in psychology and adjacent fields. Despite a century’s worth of debate, however, there is little consensus about what inhibition actually is. We suggest that it is time to abandon the concept of inhibition as it currently stands, given that its definition has been problematic. Instead, we propose an alternative framework in which inhibition is the target outcome, rather than a process to obtain a goal. We leverage existing process models to elucidate how people can achieve an inhibition goal by actively regulating impulses and desires. Although the field has been led astray by classifying inhibition as a process, our framework is intended to provide greater practical utility to the study of goal pursuit moving forward.

Longitudinal evidence that Event Related Potential measures of self-regulation do not predict everyday goal pursuit

Self-regulation has been studied across levels of analysis; however, little attention has been paid to the extent to which self-report, neural, and behavioral indices predict goal pursuit in real-life. We use a mixed-method approach (N = 201) to triangulate evidence among established measures of different aspects of self-regulation to predict both the process of goal pursuit using experience sampling, as well as longer-term goal progress at 1, 3, and 6-month follow-ups. While self-reported trait self-control predicts goal attainment months later, we observe a null relationship between longitudinal goal attainment and ERPs associated with performance-monitoring and reactivity to positive/rewarding stimuli. Despite evidence that these ERPs are reliable and trait-like, and despite theorizing that suggests otherwise, our findings suggest that these ERPs are not meaningfully associated with everyday goal attainment. These findings challenge the ecological validity of brain measures thought to assess aspects of self-regulation.

Self-regulation helps people to achieve their goals, and has been studied across modalities. Here, the authors present longitudinal evidence suggesting that common neural and behavioral measures of self-regulation derived from laboratory tasks do not predict everyday goal pursuit.

Intolerance of Transgressive Protest Actions: The Differential Roles of Deontological and Utilitarian Morality

The current research examines intolerance of protest actions by focusing on two major questions: (a) How intolerant are people of transgressive protest actions of their least-liked versus most-liked groups? and (b) how do individual differences in deontological and utilitarian moral predisposition relate to intolerance of transgressive protest actions by these two groups? In two survey-embedded experiments using nationally representative samples from two West European countries (Germany, Netherlands), we found that people were overwhelmingly intolerant of morally transgressive protest actions by both their most-liked and least-liked groups, although slightly less so for the former. In addition, deontological moral predisposition was related to increased intolerance of protest actions regardless of whether it was committed by a most-liked or least-liked group. Individual difference in utilitarian moral predisposition was related to increased acceptance of protest actions regardless of group, but especially when the actions were perceived as serving the greater good.

Something Distressing This Way Comes: The Effects of Trigger Warnings on Avoidance Behaviors in an Analogue Trauma Task

Avoidance is one of the purported benefits and harms of trigger warnings-alerts that upcoming content may contain traumatic themes. Yet, previous research has focused primarily on emotional responses. Here, we used a trauma analogue design to assess people's avoidance behavior in response to stimuli directly related to an analogue trauma event. University undergraduates (n = 199) watched a traumatic film and then viewed film image stills preceded by either a trigger warning or a neutral task instruction. Participants had the option to "cover" and avoid each image. Apart from a minor increase in avoidance when a warning appeared in the first few trials, we found that participants did not overall avoid negative stimuli prefaced with a trigger warning any more than stimuli without a warning. In fact, participants were reluctant overall to avoid distressing images; only 12.56% (n = 25) of participants used the option to cover such images when given the opportunity to do so. Furthermore, we did not find any indication that trigger warning messages help people to pause and emotionally prepare themselves to view negative content. Our results contribute to the growing body of literature demonstrating that warnings seem trivially effective in achieving their purported goals.

Effectiveness of a Self-Fitting Tool for User-Driven Fitting of Hearing Aids

Hearing aids can be effective devices to compensate for age- or non-age-related hearing losses. Their overall adoption in the affected population is still low, especially in underdeveloped countries in the subpopulation experiencing milder hearing loss. One of the major reasons for low adoption is the need for repeated complex fitting by professional audiologists, which is often not completed for various reasons. As a result, self-fitting procedures have been appearing as an alternative. Key open questions with these digital tools are linked to their effectiveness, utilized algorithms, and achievable end-results. A digital self-fitting prototype tool with a novel quick four-step fitting workflow was evaluated in a study on 19 individuals with moderate hearing loss. The tool was evaluated in a double-blinded, randomized study, having two study aims: comparing traditional audiological fitting with the new self-fitting tool, which can also be used as a remote tool. The main reported results show moderately high usability and user satisfaction obtained during self-fitting, and quasi-equivalence of the performance of the classical audiological fitting approach. The digital self-fitting tool enables multiple sessions and easy re-fitting, with the potential to outperform the classical fitting approach.