Is psychology suffering from a replication crisis? What does "failure to replicate" really mean?

Psychological Resilience and Hardiness as Protective Factors in the Relationship Between Depression/Anxiety and Well-Being: Exploratory and Confirmatory Evidence

Previous research shows depression and anxiety are negatively correlated with subjective well-being. Additionally, there is evidence psychological resilience positively influences well-being. The present study explored whether the relationship between depression/anxiety and subjective well-being might also be moderated by aspects of psychological resilience - such that depression and anxiety do not reduce well-being to the same extent in individuals high in psychological resilience traits. Participants from an exploratory sample (N = 236, Mage = 23.49) and confirmatory sample (N = 196, Mage = 24.99) completed self-report measures of depression, anxiety, well-being, resilience, and hardiness (i.e., CDRISC and DRS-15). As expected, results showed strong negative correlations between anxiety/depression and both well-being and resilience/hardiness, as well as positive correlations between well-being and resilience/hardiness. A significant interaction was also present between both resilience/hardiness and depression/anxiety in predicting well-being in the first sample. Results partially replicated in the confirmatory sample (i.e., for hardiness but not resilience). These findings add to prior work by highlighting hardiness (as measured by the DRS-15), one aspect of psychological resilience, as an important protective factor in mental health. Namely, results suggest individuals with symptoms of affective disorders may remain capable of living subjectively fulfilling lives if they possess traits of psychological resilience such as hardiness.

Measuring the replicability of our own research

In the study of transgenic mouse models of neurodevelopmental and neurodegenerative disorders, we use batteries of tests to measure deficits in behaviour and from the results of these tests, we make inferences about the mental states of the mice that we interpret as deficits in "learning", "memory", "anxiety", "depression", etc. This paper discusses the problems of determining whether a particular transgenic mouse is a valid mouse model of disease X, the problem of background strains, and the question of whether our behavioural tests are measuring what we say they are. The problem of the reliability of results is then discussed: are they replicable between labs and can we replicate our results in our own lab? This involves the study of intra- and inter- experimenter reliability. The variables that influence replicability and the importance of conducting a complete behavioural phenotype: sensory, motor, cognitive and social emotional behaviour are discussed. Then the thorny question of failure to replicate is examined: Is it a curse or a blessing? Finally, the role of failure in research and what it tells us about our research paradigms is examined.

Experiments as Code and its application to VR studies in human-building interaction

Experiments as Code (ExaC) is a concept for reproducible, auditable, debuggable, reusable, & scalable experiments. Experiments are a crucial tool to understand Human-Building Interactions (HBI) and build a coherent theory around it. However, a common concern for experiments is their auditability and reproducibility. Experiments are usually designed, provisioned, managed, and analyzed by diverse teams of specialists (e.g., researchers, technicians, engineers) and may require many resources (e.g., cloud infrastructure, specialized equipment). Although researchers strive to document experiments accurately, this process is often lacking. Consequently, it is difficult to reproduce these experiments. Moreover, when it is necessary to create a similar experiment, the "wheel is very often reinvented". It appears easier to start from scratch than trying to reuse existing work. Thus valuable embedded best practices and previous experiences are lost. In behavioral studies, such as in HBI, this has contributed to the reproducibility crisis. To tackle these challenges, we propose the ExaC paradigm, which not only documents the whole experiment, but additionally provides the automation code to provision, deploy, manage, and analyze the experiment. To this end, we define the ExaC concept, provide a taxonomy for the components of a practical implementation, and provide a proof of concept with an HBI desktop VR experiment that demonstrates the benefits of its "as code" representation, that is, reproducibility, auditability, debuggability, reusability, & scalability.

A registered report survey of open research practices in psychology departments in the UK and Ireland

Open research practices seek to enhance the transparency and reproducibility of research. While there is evidence of increased uptake in these practices, such as study preregistration and open data, facilitated by new infrastructure and policies, little research has assessed general uptake of such practices across psychology university researchers. The current study estimates psychologists' level of engagement in open research practices across universities in the United Kingdom and Ireland, while also assessing possible explanatory factors that may impact their engagement. Data were collected from 602 psychology researchers in the United Kingdom and Ireland on the extent to which they have implemented various practices (e.g., use of preprints, preregistration, open data, open materials). Here we present the summarized descriptive results, as well as considering differences between various categories of researcher (e.g., career stage, subdiscipline, methodology), and examining the relationship between researcher's practices and their self-reported capability, opportunity, and motivation (COM-B) to engage in open research practices. Results show that while there is considerable variability in engagement of open research practices, differences across career stage and subdiscipline of psychology are small by comparison. We observed consistent differences according to respondent's research methodology and based on the presence of institutional support for open research. COM-B dimensions were collectively significant predictors of engagement in open research, with automatic motivation emerging as a consistently strong predictor. We discuss these findings, outline some of the challenges experienced in this study, and offer suggestions and recommendations for future research. Estimating the prevalence of responsible research practices is important to assess sustained behaviour change in research reform, tailor educational training initiatives, and to understand potential factors that might impact engagement.

The analysis of collective orientation and process feedback in relation to coordination and performance in interdependently working teams

Effective teamwork is not only essential for teams themselves, but also for organizations and our society. To facilitate team processes and enhance team performance, feedback interventions are a widely used means. However, different types of feedback (i.e., individual vs. team-level feedback, performance vs. process feedback) can have various effects leaving the question of their effectiveness unanswered. This is especially important when team members' attitudes (namely collective orientation) are considered. Thus, understanding the interplay between types of feedback and team members' attitudes would reveal new opportunities for fostering reliable teamwork. The methodology of the present study is based on a laboratory approach. Teams (N = 142) of two worked together over four scenarios to extinguish forest fires in a microworld. We examined the effects of collective orientation on team coordination and team performance. To understand the interplay between feedback and attitudes we examined the effect of different feedback interventions on team performance and on a change in collective orientation. For analyzing multilevel mediation and changes over time, Bayesian multilevel models were applied. Results show a positive relationship between collective orientation and team performance mediated by coordination. Additionally, team-level process and performance feedback seem to be slightly more beneficial for maintaining performance over time with increasing difficulty of the task compared to individual-level process feedback. Feedback can lead to an increase in collective orientation if these values are low at the beginning. Our research highlights the importance of collective orientation and feedback interventions on team processes and performance for interdependently working teams.

The "bubbles"-study: Validation of ultra-short scales for the assessment of depression, anxiety and stress symptoms

Depression, anxiety and stress symptoms cause substantial psychological and economic burdens around the globe. To mitigate the negative consequences, the negative symptoms should be identified at an early stage. Therefore, the implementation of very brief valid screening tools in mental health prevention programs and in therapeutic settings is advantageous. In two studies on representative German population samples, we developed and validated three ultra-short scales-the "bubbles"-that consist of only one item based on the Depression Anxiety Stress 21 subscales (DASS-21) for the assessment of depression, anxiety and stress symptoms. The results of Study 1 (N = 1,001) and Study 2 (N = 894) revealed that the bubbles are valid instruments that fit the DASS-21 subscales on the factor level. Moreover, the bubbles replicated the association pattern of the DASS-21 subscales with demographic variables, and with variables that belong to the negative and the positive dimension of mental health. Thus, due to their time- and cost-efficiency, the bubbles can be used as brief screening tools in research (e.g., large-scale studies, longitudinal studies, experience sampling paradigms) and in praxis. Their shortness can prevent fatigue, motivation decrease, and participants' drop-out.

Association between resting-state connectivity patterns in the defensive system network and treatment response in spider phobia-a replication approach

Although highly effective on average, exposure-based treatments do not work equally well for all patients with anxiety disorders. The identification of pre-treatment response-predicting patient characteristics may enable patient stratification. Preliminary research highlights the relevance of inhibitory fronto-limbic networks as such. We aimed to identify pre-treatment neural signatures differing between exposure treatment responders and non-responders in spider phobia and to validate results through rigorous replication. Data of a bi-centric intervention study comprised clinical phenotyping and pre-treatment resting-state functional connectivity (rsFC) data of n = 79 patients with spider phobia (discovery sample) and n = 69 patients (replication sample). RsFC data analyses were accomplished using the Matlab-based CONN-toolbox with harmonized analyses protocols at both sites. Treatment response was defined by a reduction of >30% symptom severity from pre- to post-treatment (Spider Phobia Questionnaire Score, primary outcome). Secondary outcome was defined by a reduction of >50% in a Behavioral Avoidance Test (BAT). Mean within-session fear reduction functioned as a process measure for exposure. Compared to non-responders and pre-treatment, results in the discovery sample seemed to indicate that responders exhibited stronger negative connectivity between frontal and limbic structures and were characterized by heightened connectivity between the amygdala and ventral visual pathway regions. Patients exhibiting high within-session fear reduction showed stronger excitatory connectivity within the prefrontal cortex than patients with low within-session fear reduction. Whereas these results could be replicated by another team using the same data (cross-team replication), cross-site replication of the discovery sample findings in the independent replication sample was unsuccessful. Results seem to support negative fronto-limbic connectivity as promising ingredient to enhance response rates in specific phobia but lack sufficient replication. Further research is needed to obtain a valid basis for clinical decision-making and the development of individually tailored treatment options. Notably, future studies should regularly include replication approaches in their protocols.

Many nonnormalities, one simulation: Do different data generation algorithms affect study results?

Monte Carlo simulation studies are among the primary scientific outputs contributed by methodologists, guiding application of various statistical tools in practice. Although methodological researchers routinely extend simulation study findings through follow-up work, few studies are ever replicated. Simulation studies are susceptible to factors that can contribute to replicability failures, however. This paper sought to conduct a meta-scientific study by replicating one highly cited simulation study (Curran et al., Psychological Methods, 1, 16-29, 1996) that investigated the robustness of normal theory maximum likelihood (ML)-based chi-square fit statistics under multivariate nonnormality. We further examined the generalizability of the original study findings across different nonnormal data generation algorithms. Our replication results were generally consistent with original findings, but we discerned several differences. Our generalizability results were more mixed. Only two results observed under the original data generation algorithm held completely across other algorithms examined. One of the most striking findings we observed was that results associated with the independent generator (IG) data generation algorithm vastly differed from other procedures examined and suggested that ML was robust to nonnormality for the particular factor model used in the simulation. Findings point to the reality that extant methodological recommendations may not be universally valid in contexts where multiple data generation algorithms exist for a given data characteristic. We recommend that researchers consider multiple approaches to generating a specific data or model characteristic (when more than one is available) to optimize the generalizability of simulation results.

Patterns of Neural Network Functional Connectivity Associated With Mania/Hypomania and Depression Risk in 3 Independent Young Adult Samples

Importance

Mania/hypomania is the pathognomonic feature of bipolar disorder (BD). Established, reliable neural markers denoting mania/hypomania risk to help with early risk detection and diagnosis and guide the targeting of pathophysiologically informed interventions are lacking.

Objective

To identify patterns of neural responses associated with lifetime mania/hypomania risk, the specificity of such neural responses to mania/hypomania risk vs depression risk, and the extent of replication of findings in 2 independent test samples.

Design, Setting, and Participants

This cross-sectional study included 3 independent samples of young adults aged 18 to 30 years without BD or active substance use disorder within the past 3 months who were recruited from the community through advertising. Of 603 approached, 299 were ultimately included and underwent functional magnetic resonance imaging at the University of Pittsburgh, Pittsburgh, Pennsylvania, from July 2014 to May 2023.

Main Outcomes and Measures

Activity and functional connectivity to approach-related emotions were examined using a region-of-interest mask supporting emotion processing and emotional regulation. The Mood Spectrum Self-Report assessed lifetime mania/hypomania risk and depression risk. In the discovery sample, elastic net regression models identified neural variables associated with mania/hypomania and depression risk; multivariable regression models identified the extent to which selected variables were significantly associated with each risk measure. Multivariable regression models then determined whether associations in the discovery sample replicated in both test samples.

Results

A total of 299 participants were included. The discovery sample included 114 individuals (mean [SD] age, 21.60 [1.91] years; 80 female and 34 male); test sample 1, 103 individuals (mean [SD] age, 21.57 [2.09] years; 30 male and 73 female); and test sample 2, 82 individuals (mean [SD] age, 23.43 [2.86] years; 48 female, 29 male, and 5 nonbinary). Associations between neuroimaging variables and Mood Spectrum Self-Report measures were consistent across all 3 samples. Bilateral amygdala-left amygdala functional connectivity and bilateral ventrolateral prefrontal cortex-right dorsolateral prefrontal cortex functional connectivity were positively associated with mania/hypomania risk: discovery omnibus χ2 = 1671.7 (P < .001); test sample 1 omnibus χ2 = 1790.6 (P < .001); test sample 2 omnibus χ2 = 632.7 (P < .001). Bilateral amygdala-left amygdala functional connectivity and right caudate activity were positively associated and negatively associated with depression risk, respectively: discovery omnibus χ2 = 2566.2 (P < .001); test sample 1 omnibus χ2 = 2935.9 (P < .001); test sample 2 omnibus χ2 = 1004.5 (P < .001).

Conclusions and Relevance

In this study of young adults, greater interamygdala functional connectivity was associated with greater risk of both mania/hypomania and depression. By contrast, greater functional connectivity between ventral attention or salience and central executive networks and greater caudate deactivation were reliably associated with greater risk of mania/hypomania and depression, respectively. These replicated findings indicate promising neural markers distinguishing mania/hypomania-specific risk from depression-specific risk and may provide neural targets to guide and monitor interventions for mania/hypomania and depression in at-risk individuals.

Are we really Bayesian? Probabilistic inference shows sub-optimal knowledge transfer

Numerous studies have found that the Bayesian framework, which formulates the optimal integration of the knowledge of the world (i.e. prior) and current sensory evidence (i.e. likelihood), captures human behaviours sufficiently well. However, there are debates regarding whether humans use precise but cognitively demanding Bayesian computations for behaviours. Across two studies, we trained participants to estimate hidden locations of a target drawn from priors with different levels of uncertainty. In each trial, scattered dots provided noisy likelihood information about the target location. Participants showed that they learned the priors and combined prior and likelihood information to infer target locations in a Bayes fashion. We then introduced a transfer condition presenting a trained prior and a likelihood that has never been put together during training. How well participants integrate this novel likelihood with their learned prior is an indicator of whether participants perform Bayesian computations. In one study, participants experienced the newly introduced likelihood, which was paired with a different prior, during training. Participants changed likelihood weighting following expected directions although the degrees of change were significantly lower than Bayes-optimal predictions. In another group, the novel likelihoods were never used during training. We found people integrated a new likelihood within (interpolation) better than the one outside (extrapolation) the range of their previous learning experience and they were quantitatively Bayes-suboptimal in both. We replicated the findings of both studies in a validation dataset. Our results showed that Bayesian behaviours may not always be achieved by a full Bayesian computation. Future studies can apply our approach to different tasks to enhance the understanding of decision-making mechanisms.

Better Accuracy for Better Science . . . Through Random Conclusions

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.

Developmental psychologists should adopt citizen science to improve generalization and reproducibility

Widespread failures of replication and generalization are, ironically, a scientific triumph, in that they confirm the fundamental metascientific theory that underlies our field. Generalizable and replicable findings require testing large numbers of subjects from a wide range of demographics with a large, randomly-sampled stimulus set, and using a variety of experimental parameters. Because few studies accomplish any of this, meta-scientists predict that findings will frequently fail to replicate or generalize. We argue that to be more robust and replicable, developmental psychology needs to find a mechanism for collecting data at greater scale and from more diverse populations. Luckily, this mechanism already exists: Citizen science, in which large numbers of uncompensated volunteers provide data. While best-known for its contributions to astronomy and ecology, citizen science has also produced major findings in neuroscience and psychology, and increasingly in developmental psychology. We provide examples, address practical challenges, discuss limitations, and compare to other methods of obtaining large datasets. Ultimately, we argue that the range of studies where it makes sense *not* to use citizen science is steadily dwindling.

Contamination in Observational Research on Child Maltreatment: A Conceptual and Empirical Review With Implications for Future Research

Contamination is a methodological phenomenon occurring in child maltreatment research when individuals in an established comparison condition have, in reality, been exposed to maltreatment during childhood. The current paper: (1) provides a conceptual and methodological introduction to contamination in child maltreatment research, (2) reviews the empirical literature demonstrating that the presence of contamination biases causal estimates in both prospective and retrospective cohort studies of child maltreatment effects, (3) outlines a dual measurement strategy for how child maltreatment researchers can address contamination, and (4) describes modern statistical methods for generating causal estimates in child maltreatment research after contamination is controlled. Our goal is to introduce the issue of contamination to researchers examining the effects of child maltreatment in an effort to improve the precision and replication of causal estimates that ultimately inform scientific and clinical decision-making as well as public policy.

Computational social science is growing up: why puberty consists of embracing measurement validation, theory development, and open science practices

Puberty is a phase in which individuals often test the boundaries of themselves and surrounding others and further define their identity - and thus their uniqueness compared to other individuals. Similarly, as Computational Social Science (CSS) grows up, it must strike a balance between its own practices and those of neighboring disciplines to achieve scientific rigor and refine its identity. However, there are certain areas within CSS that are reluctant to adopt rigorous scientific practices from other fields, which can be observed through an overreliance on passively collected data (e.g., through digital traces, wearables) without questioning the validity of such data. This paper argues that CSS should embrace the potential of combining both passive and active measurement practices to capitalize on the strengths of each approach, including objectivity and psychological quality. Additionally, the paper suggests that CSS would benefit from integrating practices and knowledge from other established disciplines, such as measurement validation, theoretical embedding, and open science practices. Based on this argument, the paper provides ten recommendations for CSS to mature as an interdisciplinary field of research.

Assessing the Evidential Value of Mental Fatigue and Exercise Research

It has often been reported that mental exertion, presumably leading to mental fatigue, can negatively affect exercise performance; however, recent findings have questioned the strength of the effect. To further complicate this issue, an overlooked problem might be the presence of publication bias in studies using underpowered designs, which is known to inflate false positive report probability and effect size estimates. Altogether, the presence of bias is likely to reduce the evidential value of the published literature on this topic, although it is unknown to what extent. The purpose of the current work was to assess the evidential value of studies published to date on the effect of mental exertion on exercise performance by assessing the presence of publication bias and the observed statistical power achieved by these studies. A traditional meta-analysis revealed a Cohen's dz effect size of - 0.54, 95% CI [- 0.68, - 0.40], p < .001. However, when we applied methods for estimating and correcting for publication bias (based on funnel plot asymmetry and observed p-values), we found that the bias-corrected effect size became negligible with most of publication-bias methods and decreased to - 0.36 in the more optimistic of all the scenarios. A robust Bayesian meta-analysis found strong evidence in favor of publication bias, BFpb > 1000, and inconclusive evidence in favor of the effect, adjusted dz = 0.01, 95% CrI [- 0.46, 0.37], BF10 = 0.90. Furthermore, the median observed statistical power assuming the unadjusted meta-analytic effect size (i.e., - 0.54) as the true effect size was 39% (min = 19%, max = 96%), indicating that, on average, these studies only had a 39% chance of observing a significant result if the true effect was Cohen's dz = - 0.54. If the more optimistic adjusted effect size (- 0.36) was assumed as the true effect, the median statistical power was just 20%. We conclude that the current literature is a useful case study for illustrating the dangers of conducting underpowered studies to detect the effect size of interest.

No effect of hunger on attentional capture by food cues: Two replication studies

Food cues potently capture human attention, and it has been suggested that hunger increases their propensity to do so. However, the evidence for such hunger-related attentional biases is weak. We focus on one recent study that did show significantly greater attentional capture by food cues when participants were hungry, using an Emotional Blink of Attention (EBA) task [Piech, Pastorino, & Zald, 2010. Appetite, 54, 579-582]. We conducted online (N = 29) and in-person (N = 28) replications of this study with British participants and a Bayesian analytical approach. For the EBA task, participants tried to identify a rotated target image in a Rapid Serial Visual Presentation (RSVP). Targets were preceded by "neutral", "romantic", or "food" distractor images. Participants completed the task twice, 6-11 days apart, once hungry (overnight plus 6h fast) and once sated (after a self-selected lunch in the preceding 1h). We predicted that food images would create a greater attentional blink when participants were hungry than when they were sated, but romantic and neutral images would not. We found no evidence that hunger increased attentional capture by food cues, despite our experiments passing manipulation and quality assurance checks. Our sample and stimuli differed from the study we were replicating in several ways, but we were unable to identify any specific factor responsible for the difference in results. The original finding may not be generalisable. The EBA is more sensitive to the physical distinctiveness of distractors from filler and target images than their emotional valence, undermining the sensitivity of the EBA task for picking up subtle changes in motivational state. Moreover, hunger-related attentional bias shifts may not be substantial over the intensities and durations of hunger typically induced in laboratory experiments.

Null regions: a unified conceptual framework for statistical inference

Ruling out the possibility that there is absolutely no effect or association between variables may be a good first step, but it is rarely the ultimate goal of science. Yet that is the only inference provided by traditional null hypothesis significance testing (NHST), which has been a mainstay of many scientific fields. Reliance on NHST also makes it difficult to define what it means to replicate a finding, and leads to an uncomfortable quandary in which increasing precision in data reduces researchers' ability to perform theory falsification. To solve these problems, in recent years several alternatives to traditional NHST have been proposed. However, each new test is described using its own terminology and practiced in different fields. We describe a simple, unified framework for conceptualizing all these tests so that it is not necessary to learn them separately. Moreover, the framework allows researchers to conduct any of these tests by asking just one question: is the confidence interval entirely outside the null region(s)? This framework may also help researchers choose the test(s) that best answers their research question when simply ruling out 'no effect at all' is not enough.

Standard experimental paradigm designs and data exclusion practices in cognitive psychology can inadvertently introduce systematic "shadow" biases in participant samples

Standard cognitive psychology research practices can introduce inadvertent sampling biases that reduce the reliability and generalizability of the findings. Researchers commonly acknowledge and understand that any given study sample is not perfectly generalizable, especially when implementing typical experimental constraints (e.g., limiting recruitment to specific age ranges or to individuals with normal color vision). However, less obvious systematic sampling constraints, referred to here as "shadow" biases, can be unintentionally introduced and can easily go unnoticed. For example, many standard cognitive psychology study designs involve lengthy and tedious experiments with simple, repetitive stimuli. Such testing environments may 1) be aversive to some would-be participants (e.g., those high in certain neurodivergent symptoms) who may self-select not to enroll in such studies, or 2) contribute to participant attrition, both of which reduce the sample's representativeness. Likewise, standard performance-based data exclusion efforts (e.g., minimum accuracy or response time) or attention checks can systematically remove data from participants from subsets of the population (e.g., those low in conscientiousness). This commentary focuses on the theoretical and practical issues behind these non-obvious and often unacknowledged "shadow" biases, offers a simple illustration with real data as a proof of concept of how applying attention checks can systematically skew latent/hidden variables in the included population, and then discusses the broader implications with suggestions for how to manage and reduce, or at a minimum acknowledge, the problem.

A Digital Music-Based Mindfulness Intervention ("healing attempt") for Race-Based Anxiety in Black Americans

This study replicates and extends findings that "healing attempt"-a brief digital music-based mindfulness intervention-represents a feasible and potentially effective intervention for race-based anxiety in the Black community.

Adolescent affective psychopathic traits: the long-term outcomes of mother-infant attachment across 14 years

Affective psychopathic traits have been associated with increased violence and aggression. The scholarly interest in the pathways leading to affective psychopathic traits in adolescence has been growing. Infant attachment security may be one of the early life factors that contribute to the development of affective psychopathic traits. In this study, we test the infant attachment styles measured with the Strange Situation Procedure as a predictor of adolescent affective psychopathic traits measured in three dimensions: remorselessness, unemotionality, and callousness, among 1149 families in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Study of Early Child Care and Youth Development (SECCYD). Propensity scores for the four attachment styles were estimated with iterative tree-based regression models. After accounting for the potential confounding effects of demographic characteristics and child temperament with the inverse probability of treatment weighting, weighted generalized linear models revealed the association between insecure/avoidant attachment style measured one year after childbirth and higher levels of remorselessness, unemotionality, and callousness at 15 years. These findings provide insights into the long-term outcomes for attachment relationships established in early life.

Improving the stability of bivariate correlations using informative Bayesian priors: a Monte Carlo simulation study

Objective

Much of psychological research has suffered from small sample sizes and low statistical power, resulting in unstable parameter estimates. The Bayesian approach offers a promising solution by incorporating prior knowledge into statistical models, which may lead to improved stability compared to a frequentist approach.

Methods

Simulated data from four populations with known bivariate correlations (ρ = 0.1, 0.2, 0.3, 0.4) was used to estimate the sample correlation as samples were sequentially added from the population, from n = 10 to n = 500. The impact of three different, subjectively defined prior distributions (weakly, moderately, and highly informative) was investigated and compared to a frequentist model.

Results

The results show that bivariate correlation estimates are unstable, and that the risk of obtaining an estimate that is exaggerated or in the wrong direction is relatively high, for sample sizes for below 100, and considerably so for sample sizes below 50. However, this instability can be constrained by informative Bayesian priors.

Conclusion

Informative Bayesian priors have the potential to significantly reduce sample size requirements and help ensure that obtained estimates are in line with realistic expectations. The combined stabilizing and regularizing effect of a weakly informative prior is particularly useful when conducting research with small samples. The impact of more informative Bayesian priors depends on one's threshold for probability and whether one's goal is to obtain an estimate merely in the correct direction, or to obtain a high precision estimate whose associated interval falls within a narrow range. Implications for sample size requirements and directions for future research are discussed.

The trustworthiness of the cumulative knowledge in industrial/organizational psychology: The current state of affairs and a path forward

The goal of industrial/organizational (IO) psychology, is to build and organize trustworthy knowledge about people-related phenomena in the workplace. Unfortunately, as with other scientific disciplines, our discipline may be experiencing a "crisis of confidence" stemming from the lack of reproducibility and replicability of many of our field's research findings, which would suggest that much of our research may be untrustworthy. If a scientific discipline's research is deemed untrustworthy, it can have dire consequences, including the withdraw of funding for future research. In this focal article, we review the current state of reproducibility and replicability in IO psychology and related fields. As part of this review, we discuss factors that make it less likely that research findings will be trustworthy, including the prevalence of scientific misconduct, questionable research practices (QRPs), and errors. We then identify some root causes of these issues and provide several potential remedies. In particular, we highlight the need for improved research methods and statistics training as well as a re-alignment of the incentive structure in academia. To accomplish this, we advocate for changes in the reward structure, improvements to the peer review process, and the implementation of open science practices. Overall, addressing the current "crisis of confidence" in IO psychology requires individual researchers, academic institutions, and publishers to embrace system-wide change.

From "below chance" to "a single error is one too many": Evaluating various thresholds for invalid performance on two forced choice recognition tests

This study was designed to empirically evaluate the classification accuracy of various definitions of invalid performance in two forced-choice recognition performance validity tests (PVTs; FCRCVLT-II and Test of Memory Malingering [TOMM-2]). The proportion of at and below chance level responding defined by the binomial theory and making any errors was computed across two mixed clinical samples from the United States and Canada (N = 470) and two sets of criterion PVTs. There was virtually no overlap between the binomial and empirical distributions. Over 95% of patients who passed all PVTs obtained a perfect score. At chance level responding was limited to patients who failed ≥2 PVTs (91% of them failed 3 PVTs). No one scored below chance level on FCRCVLT-II or TOMM-2. All 40 patients with dementia scored above chance. Although at or below chance level performance provides very strong evidence of non-credible responding, scores above chance level have no negative predictive value. Even at chance level scores on PVTs provide compelling evidence for non-credible presentation. A single error on the FCRCVLT-II or TOMM-2 is highly specific (0.95) to psychometrically defined invalid performance. Defining non-credible responding as below chance level scores is an unnecessarily restrictive threshold that gives most examinees with invalid profiles a Pass.

Perceived illness consequences predict the long-term course of handicap in patients with vertigo and dizziness beyond vestibular abnormality

OBJECTIVE

Vertigo and dizziness (VD) affect one third of the population during their lifetime. VD patients are oftentimes severely handicapped. One current study showed that illness perceptions, emotional as well as behavioral responses to illness were associated with VD-related handicap at 3-months follow-up. However, no study has yet investigated this association for a period longer than six months. This study aimed to investigate long-term associations of cognitive, emotional, and behavioral factors with VD-related handicap.

METHODS

In a naturalistic longitudinal study design, n = 161 patients with VD were examined at baseline, at 6-months follow-up, and at 12-months follow-up. Participants underwent neurological and psychiatric examinations as well as comprehensive psychological assessments using self-report questionnaires.

RESULTS

During the study period VD-related handicap decreased significantly (Cohen's d = .35, p < .001). Cognitive, emotional, and behavioral factors remained without significant change during the study period. Vestibular testing as well as the type of diagnosis were not associated with changes in VD-related handicap. Changes in perceived illness consequences (ß = .265, p < .001), depression (ß = .257, p < .001), and anxiety (ß = .206, p = .008) significantly predicted the course of VD-related handicap over 12 months, while the presence vs. absence of vestibular abnormality did not.

CONCLUSION

Our results extend findings that cognitive and emotional factors including perceived illness consequences, depression, and anxiety are associated with the long-term course of VD-related handicap and may provide therapeutic targets to improve long-term outcomes in patients with VD.

How do psychology researchers interpret the results of multiple replication studies?

Employing two vignette studies, we examined how psychology researchers interpret the results of a set of four experiments that all test a given theory. In both studies, we found that participants' belief in the theory increased with the number of statistically significant results, and that the result of a direct replication had a stronger effect on belief in the theory than the result of a conceptual replication. In Study 2, we additionally found that participants' belief in the theory was lower when they assumed the presence of p-hacking, but that belief in the theory did not differ between preregistered and non-preregistered replication studies. In analyses of individual participant data from both studies, we examined the heuristics academics use to interpret the results of four experiments. Only a small proportion (Study 1: 1.6%; Study 2: 2.2%) of participants used the normative method of Bayesian inference, whereas many of the participants' responses were in line with generally dismissed and problematic vote-counting approaches. Our studies demonstrate that many psychology researchers overestimate the evidence in favor of a theory if one or more results from a set of replication studies are statistically significant, highlighting the need for better statistical education.

Evaluating the Evidence for Enclothed Cognition: Z-Curve and Meta-Analyses

Enclothed cognition refers to the systematic influence that clothes can have on the wearer's feelings, thoughts, and behaviors through their symbolic meaning. It has attracted considerable academic and nonacademic interest, with the 2012 article that coined the phrase cited more than 600 times and covered in more than 160 news outlets. However, a recent high-powered replication failed to replicate one of the original effects. To determine whether the larger body of research on enclothed cognition possesses evidential value and replicable effects, we performed z-curve and meta-analyses using 105 effects from 40 studies across 24 articles (N = 3,789). Underscoring the marked improvement of psychological research practices in the mid-2010s, our results raise concerns about the replicability of early enclothed cognition studies but affirm the evidential value for effects published after 2015. These later studies support the core principle of enclothed cognition-what we wear influences how we think, feel, and act.

Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

BACKGROUND

The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation.

METHODS

Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses.

RESULTS

A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (β =  - 2.75, p < .001, R²_adj = 0.40; r =  - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF₁₀ > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance.

CONCLUSIONS

Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.

Improving Scale Equivalence by Increasing Access to Scale-Specific Information

Measures of the same phenomenon should produce the same results; this principle is fundamental because it allows for replication-the basis of science. Unfortunately, measures of a psychological construct in one language can often measure something a bit different in another language (i.e., low "scale equivalence"). Historically, the problem was thought to stem from insufficient knowledge of best-practice translation procedures. Yet solutions based on this diagnosis and their widespread adoption have not resolved the issue. In this article, we suggest that an additional problem might be insufficient information about the measure being translated. If so, low scale equivalence is a problem that translators and cross-cultural psychologists cannot solve on their own. We explore the possibility that measure-specific translation guides be created by original scale builders for the most widely used measures of important psychological constructs. We describe why such guides are needed, when they are needed, what they might look like, their feasibility, and next steps, providing a complete example guide and test case in a supplement concerning the Primals Inventory. In this article, we seek to spark discussion on translation practices happening behind the scenes and how greater transparency can improve scale equivalence, in the spirit of open science.

Thinking About Reasons for One's Choices Increases Sensitivity to Moral Norms in Moral-Dilemma Judgments

Whereas norm-conforming (deontological) judgments have been claimed to be rooted in automatic emotional responses, outcome-maximizing (utilitarian) judgments are assumed to require reflective reasoning. Using the CNI model to disentangle factors underlying moral-dilemma judgments, the current research investigated effects of thinking about reasons on sensitivity to consequences, sensitivity to moral norms, and general action preferences. Three experiments (two preregistered) found that thinking about reasons (vs. responding intuitively or thinking about intuitions) reliably increased sensitivity to moral norms independent of processing time. Thinking about reasons had no reproducible effects on sensitivity to consequences and general action preferences. The results suggest that norm-conforming responses in moral dilemmas can arise from reflective thoughts about reasons, challenging the modal view on the role of cognitive reflection in moral-dilemma judgment. The findings highlight the importance of distinguishing between degree (high vs. low elaboration) and content (intuitions vs. reasons) as distinct aspects of cognitive reflection.

Developmental manifestations of polygenic risk for bipolar disorder from infancy to middle childhood

Knowledge on how genetic risk for bipolar disorder manifests in developmental, emotional or behavioral traits during childhood is lacking. This issue is important to address to inform early detection and intervention efforts. We investigated whether polygenic risk for bipolar disorder is associated with developmental outcomes during early to middle childhood in the general population, and if associations differ between boys and girls. Our sample consisted of 28 001 children from the Norwegian Mother, Father and Child Cohort study, a prospective pregnancy cohort with available genotype and developmental data. Mothers reported on a range of developmental outcomes in their children at 6 and 18 months, 3, 5 and 8 years. Polygenic risk scores reflecting common variant liability to bipolar disorder were calculated. Linear regression models were used in a multi-group framework to investigate associations between polygenic risk score and developmental outcomes, using sex as a grouping variable. We found robust evidence for an association between polygenic risk scores for bipolar disorder and conduct difficulties (β = 0.041, CI = 0.020-0.062) and oppositional defiant difficulties (β = 0.032, CI = 0.014-0.051) at 8 years. Associations with most other outcomes were estimated within the region of practical equivalence to zero (equivalence range D = -0.1 to 0.1), with the exceptions of negative association for activity levels (β = -0.028, CI = -0.047- -0.010) at age 5 and benevolence (β = -0.025, CI = -0.043 to -0.008) at age 8, and positive association for motor difficulties (β = 0.025, CI = 0.008-0.043) at age 3, inattention (β = 0.021, CI = 0.003-0.041) and hyperactivity (β = 0.025, CI = 0.006-0.044) at age 8. Our results suggest that genetic risk for bipolar disorder manifests as disruptive behaviors like oppositional defiant and conduct difficulties in childhood in the general population.

Understanding and improving the language of business: How accounting and corporate reporting research can better serve business and society

Financial accounting, the core of corporate reporting, is often characterized as the 'language of business'. Over the last roughly 100 years, and using an evolving set of theories, methods, and data, scholarly work in this area has been contributing to our understanding of this language and how to improve it. This paper seeks, first, to characterize the field with a focus on its evolution in the German-speaking area, where, like elsewhere, normative research traditions interested in improving practice have been making way for positivist approaches that seek a detached understanding of 'what is.' Second, we discuss the changing users and institutional parameters that are reshaping corporate reporting, followed by our personal view of 'wicked' societal problems and challenges that corporate reporting might be able to help alleviate. Finally, we discuss directions in which research might evolve in order to address these issues, in order to make corporate reporting more useful for serving not only economic actors, but also society and the environment more broadly.

Operationalizing anthropological theory: four techniques to simplify networks of co-occurring ethnographic codes

The use of data and algorithms in the social sciences allows for exciting progress, but also poses epistemological challenges. Operations that appear innocent and purely technical may profoundly influence final results. Researchers working with data can make their process less arbitrary and more accountable by making theoretically grounded methodological choices. We apply this approach to the problem of simplifying networks representing ethnographic corpora, in the interest of visual interpretation. Network nodes represent ethnographic codes, and their edges the co-occurrence of codes in a corpus. We introduce and discuss four techniques to simplify such networks and facilitate visual analysis. We show how the mathematical characteristics of each one are aligned with an identifiable approach in sociology or anthropology: structuralism and post-structuralism; identifying the central concepts in a discourse; and discovering hegemonic and counter-hegemonic clusters of meaning. We then provide an example of how the four techniques complement each other in ethnographic analysis.

On the Accuracy of Replication Failure Rates

A prominent approach to studying the replication crisis has been to conduct replications of several different scientific findings as part of the same research effort. The reported proportion of findings that these programs determined failed to replicate have become important statistics in the replication crisis. However, these "failure rates" are based on decisions about whether individual studies replicated, which are themselves subject to statistical uncertainty. In this article, we examine how that uncertainty impacts the accuracy of reported failure rates and find that the reported failure rates can be substantially biased and highly variable. Indeed, very high or very low failure rates could arise from chance alone.

Testing Replicability and Generalizability of the Time on Task Effect

The time on task (ToT) effect describes the relationship of the time spent on a cognitive task and the probability of successful task completion. The effect has been shown to vary in size and direction across tests and even within tests, depending on the test taker and item characteristics. Specifically, investing more time has a positive effect on response accuracy for difficult items and low ability test-takers, but a negative effect for easy items and high ability test-takers. The present study sought to test the replicability of this result pattern of the ToT effect across samples independently drawn from the same populations of persons and items. Furthermore, its generalizability was tested in terms of differential correlations across ability tests. To this end, ToT effects were estimated for three different reasoning tests and one test measuring natural sciences knowledge in 10 comparable subsamples with a total N = 2640. Results for the subsamples were highly similar, demonstrating that ToT effects are estimated with sufficient reliability. Generally, faster answers tended to be more accurate, suggesting a relatively effortless processing style. However, with increasing item difficulty and decreasing person ability, the effect flipped to the opposite direction, i.e., higher accuracy with longer processing times. The within-task moderation of the ToT effect can be reconciled with an account on effortful processing or cognitive load. By contrast, the generalizability of the ToT effect across different tests was only moderate. Cross-test relations were stronger in relative terms if performance in the respective tasks was more strongly related. This suggests that individual differences in the ToT effect depend on test characteristics such as their reliabilities but also similarities and differences of their processing requirements.

Neural correlates of the attention training technique as used in metacognitive therapy – A randomized sham-controlled fMRI study in healthy volunteers

Introduction

The Attention Training Technique (ATT) developed as part of metacognitive therapy is a psychotherapeutic treatment method used to enhance top-down attentional flexibility and control. This study investigated potential neurocognitive changes due to ATT and its underlying neural mechanisms using pre-to-post functional magnetic resonance imaging (fMRI).

Materials and methods

Fifty-four healthy participants were subjected to a randomized, sham-controlled attention training and evaluated using a neurocognitive test battery that partly took place in an fMRI environment. Participants received two doses ATT or sham ATT daily for 1 week. On day eight, all subjects completed the neurocognitive test battery again.

Results

After the training, the ATT group showed a significant improvement in reaction times regarding attentional disengagement compared to the sham ATT group. fMRI data showed decreased levels of activation in the anterior cingulate cortex (ACC) when comparing the ATT group to the sham ATT group during attentional disengagement post intervention. No ATT &gt; sham ATT effects were found regarding selective auditory attention, working memory performance and inhibitory control.

Discussion

These findings putatively indicate that ATT facilitates faster attention allocation and increased attentional flexibility in healthy subjects. The fMRI results suggest this ATT-dependent improvement is accompanied by reduced ACC activity, indicating a more flexible attentional state.

The Influence of Diagnostic Labels on the Evaluation of Students: a Multilevel Meta-Analysis

Research suggests that children suffering from different types of disorders (learning disorders, behavioral disorders, or intellectual disabilities) are sometimes evaluated differently simply due to the presence of a diagnostic label. We conducted a multilevel meta-analysis of experimental studies (based on data from 8,295 participants and on 284 effects nested in 60 experiments) to examine the magnitude and robustness of such label effects and to explore the impact of potential moderators (type of evaluation, diagnostic category, expertise, student’s gender, and amount and type of information). We found a moderately negative overall label effect (Hedges’ g = −0.42), which was robust across several types of evaluation, different samples, and different diagnostic categories. There was no indication that expertise and the gender of the child moderated the effect. Presenting participants with only a label yielded the strongest negative effect of g = −1.26, suggesting that the effect was dependent on the amount of information being presented to participants. We conclude that labeling a child can exacerbate negative academic evaluations, behavioral evaluations, evaluations of personality, and overall assessments of the child. Further implications for theory and future research are discussed.

Psychopathy and dangerousness: An umbrella review and meta-analysis

Psychopathy has traditionally been linked with heightened criminality, but the relationship of psychopathy with increased risk for dangerousness is contested. To address this debated issue, we conducted an umbrella review (PROSPERO CRD42020214761) of all available meta-analyses of psychopathy and indices of 'dangerousness' (e.g., violent or sexual recidivism, self-reported aggression). We searched PsycINFO, Web of Science, PubMed, and Scopus from inception to August 19, 2022, to identify systematic reviews and meta-analyses on psychopathy and dangerousness. Our review included 33 studies, with the quantitative synthesis including 17 effect sizes extracted from 10 studies (N = ∼77,000 participants). Overall, we observed a pooled correlation coefficient r = 0.284 [95% CI = 0.233, 0.336] for the association of psychopathy with dangerousness (equivalent to Cohen's d = 0.592). Despite considerable heterogeneity (I² = 89.9%), leave-one-out analyses had minimal impact. The effect was robust to examination of potential moderators such as study quality and sample ages. However, the relationship was stronger when psychopathy was assessed using self-report compared to clinical rating scales. The association should be interpreted as meaningful in both the short-term and the long-term and suggests that psychopathy is one of the strongest predictors of dangerousness in the realm of psychopathology.

Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory

Terror Management Theory (TMT) postulates that humans, in response to awareness of their death, developed complex defenses to remove the salience and discomfort stemming from those thoughts. In a standard paradigm to test this theory, an individual is presented with a death-related prime (Mortality Salience; MS), such as writing the details of their own death, or something neutral, such as watching television. After a distractor task (for delay), participants complete the dependent variable, such as rating how much they like or agree with a pro- or anti-national essay and its author. Individuals in the MS condition typically exhibit greater worldview defense than control conditions by rating the pro-national essay more positively and the anti-national essay more negatively. We completed five separate studies across five unique samples with the goal of replicating and extending this well-established pattern to provide further understanding of the phenomena that underlie the effects of MS. However, despite using standard procedures, we were unable to replicate basic patterns of the dependent variable in the MS conditions. We also pooled all responses into two meta-analyses, one examining all dependent variables and one focusing on the anti-national essay; yet the effect sizes in these analyses did not significantly differ from zero. We discuss the methodological and theoretical implications of these (unintended) failures to replicate. It is not clear if these null findings were due to methodological limitations, restraints of online/crowd-sourced recruitment, or ever-evolving sociocultural factors.

Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science

INTRODUCTION

To improve the rigor of science, experimental evidence for scientific claims ideally needs to be replicated repeatedly with comparable analyses and new data to increase the collective confidence in the veracity of those claims. Large replication projects in psychology and cancer biology have evaluated the replicability of their fields but no collaborative effort has been undertaken in sports and exercise science. We propose to undertake such an effort here. As this is the first large replication project in this field, there is no agreed-upon protocol for selecting studies to replicate. Criticism of previous selection protocols include claims they were non-randomised and non-representative. Any selection protocol in sports and exercise science must be representative to provide an accurate estimate of replicability of the field. Our aim is to produce a protocol for selecting studies to replicate for inclusion in a large replication project in sports and exercise science.

METHODS

The proposed selection protocol uses multiple inclusion and exclusion criteria for replication study selection, including: the year of publication and citation rankings, research disciplines, study types, the research question and key dependent variable, study methods and feasibility. Studies selected for replication will be stratified into pools based on instrumentation and expertise required, and will then be allocated to volunteer laboratories for replication. Replication outcomes will be assessed using a multiple inferential strategy and descriptive information will be reported regarding the final number of included and excluded studies, and original author responses to requests for raw data.

Relationships between task awareness, comprehension strategies, and literacy outcomes

Reading is typically guided by a task or goal (e.g., studying for a test, writing a paper). A reader's task awareness arises from their mental representation of the task and plays an important role in guiding reading processes, ultimately influencing comprehension outcomes and task success. As such, a better understanding of how task awareness arises and how it affects comprehension is needed. The present study tested the Task Awareness Mediation Hypothesis. This hypothesis assumes that the strategies that support reading comprehension (e.g., paraphrasing, bridging, and elaborative strategies) also support a reader's task awareness while engaged in a literacy task. Further, it assumes that the reader's level of task awareness partially mediates the relationship between these comprehension strategies and a comprehension outcome. At two different time points in a semester, college students completed an assessment of their propensity to engage in comprehension strategies and a complex academic literacy task that provided a measure of comprehension outcomes and an assessment of task awareness. Indirect effects analyses provided evidence for the Task Awareness Mediation Hypothesis showing that the propensity to engage in paraphrasing and elaboration was positively predictive of task awareness, and that task awareness mediated the relationships between these comprehension strategies and performance on the complex academic literacy task. These results indicate that task awareness has complex relationships with comprehension strategies and performance on academic literacy tasks and warrants further consideration as a possible malleable factor to improve student success.

How to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice

Hypothesis testing is an essential statistical method in experimental psychology and the cognitive sciences. The problems of traditional null hypothesis significance testing (NHST) have been discussed widely, and among the proposed solutions to the replication problems caused by the inappropriate use of significance tests and p-values is a shift toward Bayesian data analysis. However, Bayesian hypothesis testing is concerned with various posterior indices for significance and the size of an effect. This complicates Bayesian hypothesis testing in practice, as the availability of multiple Bayesian alternatives to the traditional p-value causes confusion which one to select and why. In this paper, various Bayesian posterior indices which have been proposed in the literature are compared and their benefits and limitations are discussed. The comparison shows that conceptually not all proposed Bayesian alternatives to NHST and p-values are beneficial, and the usefulness of some indices strongly depends on the study design and research goal. However, the comparison also reveals that there exist at least two candidates among the available Bayesian posterior indices which have appealing theoretical properties and are widely underused in the cognitive sciences.

The Drunk Utilitarian Revisited: Does Alcohol Really Increase Utilitarianism in Moral Judgment?

The "drunk utilitarian" phenomenon suggests that people are more likely to accept harm for the greater good when they are under the influence of alcohol. This phenomenon conflicts with the ideas that (a) acceptance of pro-sacrificial harm requires inhibitory control of automatic emotional responses to the idea of causing harm and (b) alcohol impairs inhibitory control. This preregistered experiment aimed to provide deeper insights into the effects of alcohol on moral judgments by using a formal modeling approach to disentangle three factors in moral dilemma judgments and by distinguishing between instrumental harm and impartial beneficence as two distinct dimensions of utilitarian psychology. Despite the use of a substantially larger sample and higher doses of alcohol compared with the ones in prior studies, alcohol had no significant effect on moral judgments. The results pose a challenge to the idea that alcohol increases utilitarianism in moral judgments.

Accessing acute medical care to protect health: the utility of community treatment orders

Background

The conclusion that people with severe mental illness require involuntary care to protect their health (including threats due to physical-non-psychiatric-illness) is challenged by findings indicating that they often lack access to general healthcare and the assertion that they would access such care voluntarily if available and effective. Victoria, Australia's single-payer healthcare system provides accessible medical treatment; therefore, it is an excellent context in which to test these challenges.

Aims

This study replicates a previous investigation in considering whether, in Australia's easy-access single-payer healthcare system, patients placed on community treatment orders, specifically involuntary community treatment, are more likely to access acute medical care addressing potentially life-threatening physical illnesses than voluntary patients with and without severe mental illness.

Methods

Replicating methods used in 2000-2010, for the years 2010-2017, this study compared the acute medical care access of three new cohorts: 7826 hospitalised patients with severe mental illness who received a post-hospitalisation, community treatment order; 13 896 patients with severe mental illness released from the hospital without a community treatment order and 12 101 outpatients who were never psychiatrically hospitalised (individuals with less morbidity risk who were not considered to have severe mental illness) during periods when they were under versus outside community mental health supervision. Logistic regression was used to determine the influence of community-based community mental health supervision and the type of community mental health supervision (community treatment order vs non-community treatment order) on the likelihood of receiving an initial diagnosis of a life-threatening physical illness requiring acute care.

Results

Validating their shared elevated morbidity risk, 43.7% and 46.7%, respectively, of each hospitalised cohort (community treatment order and non-community treatment order patients) accessed an initial acute-care diagnosis for a life-threatening condition vs 26.3% of outpatients. Outside community mental health supervision, the likelihood that a community treatment order patient would receive a diagnosis of physical illness was 36% lower than non-community treatment order patients-1.30 times that of outpatients. Under community mental health supervision, their likelihood was two times greater than that of non-community treatment order patients and 6.6 times that of outpatients. Each community treatment order episode was associated with a 14.6% increase in the likelihood of a community treatment order patient receiving a diagnosis. The results replicate those found in an independent 2000-2010 cohort comparison.

Conclusions

Community mental health supervision, notably community treatment order supervision, in two independent investigations over two decades appeared to facilitate access to physical healthcare in acute care settings for patients with severe mental illness who were refusing treatment-a group that has been subject to excess morbidity and mortality.

A critique of using the labels confirmatory and exploratory in modern psychological research

Psychological science is experiencing a rise in the application of complex statistical models and, simultaneously, a renewed focus on applying research in a confirmatory manner. This presents a fundamental conflict for psychological researchers as more complex forms of modeling necessarily eschew as stringent of theoretical constraints. In this paper, I argue that this is less of a conflict, and more a result of a continued adherence to applying the overly simplistic labels of exploratory and confirmatory. These terms mask a distinction between exploratory/confirmatory research practices and modeling. Further, while many researchers recognize that this dichotomous distinction is better represented as a continuum, this only creates additional problems. Finally, I argue that while a focus on preregistration helps clarify the distinction, psychological research would be better off replacing the terms exploratory and confirmatory with additional levels of detail regarding the goals of the study, modeling details, and scientific method.