Caregiver-Youth intergeneration acculturation conflict moderates the relationship between depression severity and suicidality among female Mexican-Descent college students

OBJECTIVE

Suicide is the second leading cause of death among college-age students in the U.S., with disparities in suicide ideation and attempts among Latinas. The current study aims to examine if depression severity predicts suicide ideation and attempts and to examine if caregiver intergeneration acculturation conflict (IAC) moderates this link.

PARTICIPANTS

A sample of 246 Mexican-descent female college students.

METHODS

Self-reported measures for depression severity, male and female caregiver IAC, and suicide ideation and attempts in the previous 12-months were ascertained.

RESULTS

In our sample, 31.1% endorsed suicide ideation and 15.9% suicide attempts. Controlling for age, both male and female caregiver IAC moderated the relationship between depression severity and suicide ideation. A similar moderation pattern emerged for the suicide attempts outcome.

CONCLUSIONS

Understanding this exacerbating contextual factor can help inform prevention/intervention efforts targeting Mexican-descent college students who are experiencing depressive symptoms by focusing on decreasing IAC with both caregivers.Suicide is the second leading cause of death among emerging adults, defined as 18-29-year-olds,1 in the United States (U.S.) and is responsible for more deaths than any single major medical illness.2 Significant gender and ethnic/racial disparities have been well-documented and highlight that Latina emerging adults experience some of the highest rates of suicide ideation (SI) and suicide attempts (SA) and the greatest increases in SA over time, when compared to non-Latinx White, female emerging adults.3,4 In fact, recent research suggests that Latina college students report a 1.7% prevalence rate of suicide attempts compared to 1.2% among non-Latina White college students and data trends report a nearly double increase in the percent of suicide attempts from 2011 to 2015 (from 0.9% to 1.7%) for Latinas versus a minimal change (from 1.1% to 1.2%) among non-Latina White college students.4 Key research examining this disparity have cited that elevated depressive symptoms, which are also experienced at higher levels among Latinx groups in the U.S.,5 are strongly linked to SI and SA among Latinx college students and emerging adults.6,7 Developmentally, the highest risk period for the onset of SI and SA is during late adolescence or emerging adulthood8 and deaths due to suicide increase as adolescents move into emerging adulthood.2,9 Emerging adulthood is an even riskier developmental period for minoritized college students, like Latinx college students, because this period is marked by identity formation processes that are exacerbated by intercultural interactions on college campuses and cultural expectations at home.10 These data underscore the significance of detecting how unique contextual factors may interact with elevated depressive symptoms, and importantly, how these factors are associated with the increased suicide risk among Latinx college-age youth, as they represent a high-risk developmental and ethnic group.

The multiverse of data preprocessing and analysis in graph-based fMRI: A systematic literature review of analytical choices fed into a decision support tool for informed analysis

The large number of different analytical choices used by researchers is partly responsible for the challenge of replication in neuroimaging studies. For an exhaustive robustness analysis, knowledge of the full space of analytical options is essential. We conducted a systematic literature review to identify the analytical decisions in functional neuroimaging data preprocessing and analysis in the emerging field of cognitive network neuroscience. We found 61 different steps, with 17 of them having debatable parameter choices. Scrubbing, global signal regression, and spatial smoothing are among the controversial steps. There is no standardized order in which different steps are applied, and the parameter settings within several steps vary widely across studies. By aggregating the pipelines across studies, we propose three taxonomic levels to categorize analytical choices: 1) inclusion or exclusion of specific steps, 2) parameter tuning within steps, and 3) distinct sequencing of steps. We have developed a decision support application with high educational value called METEOR to facilitate access to the data in order to design well-informed robustness (multiverse) analysis.

Toward a more credible assessment of the credibility of science by many-analyst studies

We discuss a relatively new meta-scientific research design: many-analyst studies that attempt to assess the replicability and credibility of research based on large-scale observational data. In these studies, a large number of analysts try to answer the same research question using the same data. The key idea is the greater the variation in results, the greater the uncertainty in answering the research question and, accordingly, the lower the credibility of any individual research finding. Compared to individual replications, the large crowd of analysts allows for a more systematic investigation of uncertainty and its sources. However, many-analyst studies are also resource-intensive, and there are some doubts about their potential to provide credible assessments. We identify three issues that any many-analyst study must address: 1) identifying the source of variation in the results; 2) providing an incentive structure similar to that of standard research; and 3) conducting a proper meta-analysis of the results. We argue that some recent many-analyst studies have failed to address these issues satisfactorily and have therefore provided an overly pessimistic assessment of the credibility of science. We also provide some concrete guidance on how future many-analyst studies could provide a more constructive assessment.

Heterogeneity in effect size estimates

A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty, limiting the generalizability of published scientific findings. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population, design, and analytical heterogeneity. Our framework suggests that after accounting for heterogeneity, the probability that the tested hypothesis is true for the average population, design, and analysis path can be much lower than implied by nominal error rates of statistically significant individual studies. We estimate each type's heterogeneity from 70 multilab replication studies, 11 prospective meta-analyses of studies employing different experimental designs, and 5 multianalyst studies. In our data, population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. Our results should, however, be interpreted cautiously due to the limited number of studies and the large uncertainty in the heterogeneity estimates. We discuss several ways to parse and account for heterogeneity in the context of different methodologies.

The multiverse of universes: A tutorial to plan, execute and interpret multiverses analyses using the R package multiverse

Even when guided by strong theories and sound methods, researchers must often choose a singular course of action from multiple viable alternatives. Regardless of the choice, it, along with all other choices made during the research process, individually and collectively affects study results, often in unpredictable ways. The inability to disentangle how much of an observed effect is attributable to the phenomenon of interest, and how much is attributable to what have come to be known as researcher degrees of freedom (RDF), slows theoretical progress and stymies practical implementation. However, if one could examine the results from a particular set of RDF (known as a universe) against a systematically and comprehensively determined background of alternative viable universes (known as a multiverse), then the effects of RDF can be directly examined to provide greater context and clarity to future researchers, and greater confidence in the recommendations to practitioners. This tutorial demonstrates a means to map result variability directly and efficiently, and empirically investigate RDF impact on conclusions via multiverse analysis. Using the R package multiverse, we outline best practices in planning, executing and interpreting of multiverse analyses.

Misstatements, misperceptions, and mistakes in controlling for covariates in observational research

We discuss 12 misperceptions, misstatements, or mistakes concerning the use of covariates in observational or nonrandomized research. Additionally, we offer advice to help investigators, editors, reviewers, and readers make more informed decisions about conducting and interpreting research where the influence of covariates may be at issue. We primarily address misperceptions in the context of statistical management of the covariates through various forms of modeling, although we also emphasize design and model or variable selection. Other approaches to addressing the effects of covariates, including matching, have logical extensions from what we discuss here but are not dwelled upon heavily. The misperceptions, misstatements, or mistakes we discuss include accurate representation of covariates, effects of measurement error, overreliance on covariate categorization, underestimation of power loss when controlling for covariates, misinterpretation of significance in statistical models, and misconceptions about confounding variables, selecting on a collider, and p value interpretations in covariate-inclusive analyses. This condensed overview serves to correct common errors and improve research quality in general and in nutrition research specifically.

First steps into the pupillometry multiverse of developmental science

Pupillometry has been widely implemented to investigate cognitive functioning since infancy. Like most psychophysiological and behavioral measures, it implies hierarchical levels of arbitrariness in preprocessing before statistical data analysis. By means of an illustrative example, we checked the robustness of the results of a familiarization procedure that compared the impact of audiovisual and visual stimuli in 12-month-olds. We adopted a multiverse approach to pupillometry data analysis to explore the role of (1) the preprocessing phase, that is, handling of extreme values, selection of the areas of interest, management of blinks, baseline correction, participant inclusion/exclusion and (2) the modeling structure, that is, the incorporation of smoothers, fixed and random effects structure, in guiding the parameter estimation. The multiverse of analyses shows how the preprocessing steps influenced the regression results, and when visual stimuli plausibly predicted an increase of resource allocation compared with audiovisual stimuli. Importantly, smoothing time in statistical models increased the plausibility of the results compared to those nested models that do not weigh the impact of time. Finally, we share theoretical and methodological tools to move the first steps into (rather than being afraid of) the inherent uncertainty of infant pupillometry.

Exploring the multiverse of analysis options for the alcohol Stroop

The alcohol Stroop is a widely used task in addiction science to measure the theoretical concept of attentional bias (a selective attention to alcohol-related cues in the environment), which is thought to be associated with clinical outcomes (craving and consumption). However, recent research suggests findings from this task can be equivocal. This may be because the task has many different potential analysis pipelines, which increase researcher degrees of freedom when analysing data and reporting results. These analysis pipelines largely come from how outlying reaction times on the task are identified and handled (e.g. individual reaction times > 3 standard deviations from the mean are removed from the distribution; removal of all participant data if > 25% errors are made). We used specification curve analysis across two alcohol Stroop datasets using alcohol-related stimuli (one published and one novel) to examine the robustness of the alcohol Stroop effect to different analytical decisions. We used a prior review of this research area to identify 27 unique analysis pipelines. Across both data sets, the pattern of results was similar. The alcohol Stroop effect was present and largely robust to different analysis pipelines. Increased variability in the Stroop effect was observed when implementing outlier cut-offs for individual reaction times, rather than the removal of participants. Stricter outlier thresholds tended to reduce the size of the Stroop interference effect. These specification curve analyses are the first to examine the robustness of the alcohol Stroop to different analysis strategies, and we encourage researchers to adopt such analytical methods to increase confidence in their inferences across cognitive and addiction science.

Incorporating causal inference perspectives into psychoneuroimmunology: A simulation study highlighting concerns about controlling for adiposity in immunopsychiatry

Psychoneuroimmunology and immunopsychiatry are quickly approaching a critical point where the clinical translatability of their evidence base will be tested. To maximize chances for translational success, we believe researchers must adopt causal inference techniques that augment the causal relevance of estimates given theorized causal structures. To illustrate the utility of incorporating causal inference perspectives into psychoneuroimmunology, we applied directed acyclic graphs and a combination of empirical and simulated data to demonstrate the consequences of controlling for adiposity when testing the association between inflammation and depression under the plausible causal structure of increases in adipose tissue leading to greater inflammation that in turn promotes depression. Effect size estimates were pulled from a dataset combining the Midlife in the United States 2 (MIDUS-2) and MIDUS Refresher datasets. Data were extracted and used to simulate data reflecting an adiposity → inflammation → depression causal structure. Next, a Monte Carlo simulation study with 1,000 iterations and three sample size scenarios (Ns = 100, 250, and 500) was conducted testing whether controlling for adiposity when estimating the relation between inflammation and depression influenced the precision of this estimate. Across all simulation scenarios, controlling for adiposity reduced precision of the inflammation → depression estimate, suggesting that researchers primarily interested in quantifying inflammation → depression associations should not control for adiposity. This work thus underscores the importance of incorporating causal inference approaches into psychoneuroimmunological research.

Using Multiverse Analysis to Highlight Differences in Convergent Correlation Outcomes Due to Data Analytical and Study Design Choices

In neuropsychological research, there are a near limitless number of different approaches researchers can choose when designing studies. Here we showcase the multiverse/specification curve technique to establish the robustness of analytical pathways choices within classic psychometric test validation in an example test of executive function. We examined the impact of choices regarding sample groups, sample sizes, test metrics, and covariate inclusions on convergent validation correlations between tests of executive function. Data were available for 87 neurologically healthy adults and 117 stroke survivors, and a total of 2,220 different analyses were run in a multiverse analysis. We found that the type of sample group, sample size, and test metric used for analyses affected validation outcomes. Covariate inclusion choices did not affect the observed coefficients in our analyses. The present analysis demonstrates the importance of carefully justifying every aspect of a psychometric test validation study a priori with theoretical and statistical factors in mind. It is essential to thoroughly consider the purpose and use of a new tool when designing validation studies.

Enhancing precision in human neuroscience

Human neuroscience has always been pushing the boundary of what is measurable. During the last decade, concerns about statistical power and replicability - in science in general, but also specifically in human neuroscience - have fueled an extensive debate. One important insight from this discourse is the need for larger samples, which naturally increases statistical power. An alternative is to increase the precision of measurements, which is the focus of this review. This option is often overlooked, even though statistical power benefits from increasing precision as much as from increasing sample size. Nonetheless, precision has always been at the heart of good scientific practice in human neuroscience, with researchers relying on lab traditions or rules of thumb to ensure sufficient precision for their studies. In this review, we encourage a more systematic approach to precision. We start by introducing measurement precision and its importance for well-powered studies in human neuroscience. Then, determinants for precision in a range of neuroscientific methods (MRI, M/EEG, EDA, Eye-Tracking, and Endocrinology) are elaborated. We end by discussing how a more systematic evaluation of precision and the application of respective insights can lead to an increase in reproducibility in human neuroscience.

Reproducibility in Neuroimaging Analysis: Challenges and Solutions

Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience, and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research-one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish 3 main types of reproducibility, discussing each in turn. Analytical reproducibility is the ability to reproduce findings using the same data and methods. Replicability is the ability to find an effect in new datasets, using the same or similar methods. Finally, robustness to analytical variability refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in more reproducible, replicable, and robust psychological and brain research and a stronger scientific foundation across fields of inquiry.

Multiverse analyses of fear acquisition and extinction retention in posttraumatic stress disorder

Persistent fear is a cardinal feature of posttraumatic stress disorder (PTSD), and deficient fear extinction retention is a proposed illness mechanism and target of exposure-based therapy. However, evidence for deficient fear extinction in PTSD has been mixed using laboratory paradigms, which may relate to underidentified methodological variation across studies. We reviewed the literature to identify parameters that differ across studies of fear extinction retention in PTSD. We then performed Multiverse Analysis in a new sample, to quantify the impact of those methodological parameters on statistical findings. In 25 PTSD patients (15 female) and 36 trauma-exposed non-PTSD controls (TENC) (20 female), we recorded skin conductance response (SCR) during fear acquisition and extinction learning (day 1) and extinction recall (day 2). A first Multiverse Analysis examined the effects of methodological parameters identified by the literature review on comparisons of SCR-based fear extinction retention in PTSD versus TENC. A second Multiverse Analysis examined the effects of those methodological parameters on comparisons of SCR to a danger cue (CS+) versus safety cue (CS-) during fear acquisition. Both the literature review and the Multiverse Analysis yielded inconsistent findings for fear extinction retention in PTSD versus TENC, and most analyses found no statistically significant group difference. By contrast, significantly elevated SCR to CS+ versus CS- was consistently found across all analyses in the literature review and the Multiverse Analysis of new data. We discuss methodological parameters that may most contribute to inconsistent findings of fear extinction retention deficit in PTSD and implications for future clinical research.

Higher amplitudes in steady-state visual evoked potentials driven by square-wave versus sine-wave contrast modulation - A dual-laboratory study

Steady-state visual evoked potentials (ssVEPs) are an established tool for assessing visuocortical responses in visual perception and attention. They have the same temporal frequency characteristics as a periodically modulated stimulus (e.g., in contrast or luminance) that drives them. It has been hypothesized that the amplitude of a given ssVEP may depend on the shape of the stimulus modulation function, but the size and robustness of these effects is not well established. The current study systematically compared the effect of the two most common functions in the ssVEP literature, square-wave and sine-wave functions. Across two laboratories, we presented mid-complex color patterns to 30 participants with square-wave or sine-wave contrast modulation and at different driving frequencies (6 Hz, 8.57 Hz, 15 Hz). When ssVEPs were analyzed independently for the samples, with each laboratory's standard processing pipeline, ssVEP amplitudes in both samples decreased at higher driving frequencies and square-wave modulation evoked higher amplitudes at lower frequencies (i.e., 6 Hz, 8.57 Hz) compared to sine-wave modulation. These effects were replicated when samples were aggregated and analyzed with the same processing pipeline. In addition, when using signal-to-noise ratios as outcome measures, this joint analysis indicated a somewhat weaker effect of increased ssVEP amplitudes to square-wave modulation at 15 Hz. The present study suggests that square-wave modulation should be used in ssVEP research when the goal is to maximize signal amplitude or signal-to-noise ratio. Given effects of modulation function across laboratories, and data processing pipelines, the findings appear robust to differences in data collection and analysis.

Worker aging, control, and well-being: A specification curve analysis

Among the many work (and life) characteristics of relevance to adult development and aging, various forms of control are some of the most extensively and diversely studied. Indeed, "control," whether objectively held (i.e., "actual" control), perceived, or enacted through self-regulation, is a concept central to our understanding of person-environment interactions, development, and well-being within and across life domains. However, variability in conceptualization and analysis in the literature on control presents challenges to integration. To partially address these gaps, the present study sought to explore the effects of conceptual and analytical specification decisions (e.g., construct types, time, covariates) on observed control-well-being relationships in a large, age-diverse, longitudinal sample (Midlife in the United States I, II, and III datasets), providing a specification curve analysis (SCA) tutorial and guidance in the process. Results suggest that construct types and operationalizations, particularly predictor variables, have bearing on observed results, with certain types of control serving as better predictors of various forms of well-being than others. These findings and identified gaps are summarized to provide direction for theoretical clarification and reconciliation in the control and lifespan development literatures, construct selection and operationalization in future aging and work research, and inclusive, well-specified interventions to improve employee well-being.

Conspiratorial Ideation Is Associated with Lower Perceptions of Policy Effectiveness: Views from Local Governments during the COVID-19 Pandemic

Governments around the world struggled to formulate an effective response to the coronavirus disease 2019 pandemic, which was hampered by the widespread diffusion of various conspiracy theories about the virus. Local governments are often responsible for the implementing mitigation measures such as mask mandates and curfews but have received very limited attention in the scholarly literature. In this article, the authors use data from local policy actors in Colorado to evaluate the relationship between conspiratorial beliefs and perceptions of mitigation policy effectiveness. The authors find that many local policy actors hold conspiratorial beliefs, which combine with partisanship to contribute to lower perceptions of policy effectiveness. The authors conclude by discussing future research directions, noting that the broad adoption of conspiracy theories likely changes enforcement at the local scale.

Reducing bias, increasing transparency and calibrating confidence with preregistration

Flexibility in the design, analysis and interpretation of scientific studies creates a multiplicity of possible research outcomes. Scientists are granted considerable latitude to selectively use and report the hypotheses, variables and analyses that create the most positive, coherent and attractive story while suppressing those that are negative or inconvenient. This creates a risk of bias that can lead to scientists fooling themselves and fooling others. Preregistration involves declaring a research plan (for example, hypotheses, design and statistical analyses) in a public registry before the research outcomes are known. Preregistration (1) reduces the risk of bias by encouraging outcome-independent decision-making and (2) increases transparency, enabling others to assess the risk of bias and calibrate their confidence in research outcomes. In this Perspective, we briefly review the historical evolution of preregistration in medicine, psychology and other domains, clarify its pragmatic functions, discuss relevant meta-research, and provide recommendations for scientists and journal editors.

A data multiverse analysis investigating non-model based SCR quantification approaches

Electrodermal signals are commonly used outcome measures in research on arousal, emotion, and habituation. Recently, we reported on heterogeneity in skin conductance response quantification approaches and its impact on replicability. Here we provide complementary work focusing on within-approach heterogeneity of specifications for skin conductance response quantification. We focus on heterogeneity within the baseline-correction approach (BLC) which appeared as particularly heterogeneous-for instance with respect to the pre-CS baseline window duration, the start, and end of the peak detection window. We systematically scrutinize the robustness of results when applying different BLC approach specifications to one representative pre-existing data set (N = 118) in a (partly) pre-registered study. We report high agreement between different BLC approaches for US and CS+ trials, but moderate to poor agreement for CS- trials. Furthermore, a specification curve of the main effect of CS discrimination during fear acquisition training from all potential and reasonable combinations of specifications (N = 150) and a prototypical trough-to-peak (TTP) approach indicates that resulting effect sizes are largely comparable. A second specification curve (N = 605 specific combinations) highlights a strong impact of different transformation types. Crucially, however, we show that BLC approaches often misclassify the peak value-particularly for CS- trials, leading to stimulus-specific biases and challenges for post-processing and replicability of CS discrimination across studies applying different approaches. Lastly, we investigate how negative skin conductance values in BLC, appearing most frequently for CS- (CS- > CS+ > US), correspond to values in TTP quantification. We discuss the results considering prospects and challenges of the multiverse approach and future directions.

Applying multiverse analysis to experience sampling data: Investigating whether preprocessing choices affect robustness of conclusions

The experience sampling method (ESM) has revolutionized our ability to conduct psychological research in the natural environment. However, researchers have a large degree of freedom when preprocessing ESM data, which may hinder scientific progress. This study illustrates the use of multiverse analyses regarding preprocessing choices related to data exclusion (i.e., based on various levels of compliance and exclusion of the first assessment day) and the calculation of constructs (i.e., composite scores calculated as the mean, median, or mode) by reanalyzing established group differences in negative affect, stress reactivity, and emotional inertia between individuals with and without psychosis. Data came from five studies and included 233 individuals with psychosis and 223 healthy individuals (in total, 26,892 longitudinal assessments). Preprocessing choices related to data exclusion did not affect conclusions. For both stress reactivity and emotional inertia of negative affect, group differences were affected when negative affect was calculated as the mean compared to the median or mode. Further analyses revealed that this could be attributed to considerable differences in the within- and between-factor structure of negative affect. While these findings show that observed differences in affective processes between individuals with and without psychosis are robust to preprocessing choices related to data exclusion, we found disagreement in conclusions between different central tendency measures. Safeguarding the validity of future experience sampling research, scholars are advised to use multiverse analysis to evaluate the robustness of their conclusions across different preprocessing scenarios.

Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

New Data on Birth Order in Homosexual Men and Women and a Reply to Vilsmeier et al. (2021a, 2021b)

The fraternal birth order effect (FBOE) is the repeated finding that older brothers increase the odds of homosexuality in later-born males. It has been our working assumption, based on the majority of previous studies, that a similar FBOE does not occur in females. In an elaborate quantitative review posted last year to a preprint server, Vilsmeier et al. (2021a) concluded that there is no valid evidence for an FBOE in men or women. Ablaza et al. (2022) subsequently published a study of population-level data from the Netherlands with conclusions completely opposite to those of Vilsmeier et al., namely, that there is robust evidence of an FBOE in both men and women. The present research was initially undertaken to refute the assertion of Vilsmeier et al. that there is no proof of an FBOE in men and to investigate how they obtained such a discrepant conclusion. We found evidence that the discrepancy may relate to Vilsmeier et al.'s use of the large and demonstrably unreliable sample published by Frisch and Hviid (2006). After the publication by Ablaza et al., we expanded our article to address their finding of an FBOE in women. We argue that our preferred explanation of the FBOE in men-that it reflects the progressive immunization of some mothers to Y-linked antigen by each succeeding male fetus and the concomitantly increasing effects of anti-male antibody on sexual differentiation in the brain in each succeeding male fetus-could plausibly be extended to female homosexuality.

Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis

There is an increasing awareness that replication should become common practice in empirical studies. However, study results might fail to replicate for various reasons. The robustness of published study results can be assessed using the relatively new multiverse-analysis methodology, in which the robustness of the effect estimates against data analytical decisions is assessed. However, the uptake of multiverse analysis in empirical studies remains low, which might be due to the scarcity of guidance available on performing multiverse analysis. Researchers might experience difficulties in identifying data analytical decisions and in summarizing the large number of effect estimates yielded by a multiverse analysis. These difficulties are amplified when applying multiverse analysis to assess the robustness of the effect estimates from a mediation analysis, as a mediation analysis involves more data analytical decisions than a bivariate analysis. The aim of this paper is to provide an overview and worked example of the use of multiverse analysis to assess the robustness of the effect estimates from a mediation analysis. We showed that the number of data analytical decisions in a mediation analysis is larger than in a bivariate analysis. By using a real-life data example from the Longitudinal Aging Study Amsterdam, we demonstrated the application of multiverse analysis to a mediation analysis. This included the use of specification curves to determine the impact of data analytical decisions on the magnitude and statistical significance of the direct, indirect, and total effect estimates. Although the multiverse analysis methodology is still relatively new and future research is needed to further advance this methodology, this paper shows that multiverse analysis is a useful method for the assessment of the robustness of the direct, indirect, and total effect estimates in a mediation analysis and thereby to inform replication studies.

Age-related change in task-evoked amygdala-prefrontal circuitry: A multiverse approach with an accelerated longitudinal cohort aged 4-22 years

The amygdala and its connections with medial prefrontal cortex (mPFC) play central roles in the development of emotional processes. While several studies have suggested that this circuitry exhibits functional changes across the first two decades of life, findings have been mixed - perhaps resulting from differences in analytic choices across studies. Here we used multiverse analyses to examine the robustness of task-based amygdala-mPFC function findings to analytic choices within the context of an accelerated longitudinal design (4-22 years-old; N = 98; 183 scans; 1-3 scans/participant). Participants recruited from the greater Los Angeles area completed an event-related emotional face (fear, neutral) task. Parallel analyses varying in preprocessing and modeling choices found that age-related change estimates for amygdala reactivity were more robust than task-evoked amygdala-mPFC functional connectivity to varied analytical choices. Specification curves indicated evidence for age-related decreases in amygdala reactivity to faces, though within-participant changes in amygdala reactivity could not be differentiated from between-participant differences. In contrast, amygdala-mPFC functional connectivity results varied across methods much more, and evidence for age-related change in amygdala-mPFC connectivity was not consistent. Generalized psychophysiological interaction (gPPI) measurements of connectivity were especially sensitive to whether a deconvolution step was applied. Our findings demonstrate the importance of assessing the robustness of findings to analysis choices, although the age-related changes in our current work cannot be overinterpreted given low test-retest reliability. Together, these findings highlight both the challenges in estimating developmental change in longitudinal cohorts and the value of multiverse approaches in developmental neuroimaging for assessing robustness of results.

Hidden talents in context: Cognitive performance with abstract versus ecological stimuli among adversity-exposed youth

Adversity-exposed youth tend to score lower on cognitive tests. However, the hidden talents approach proposes some abilities are enhanced by adversity, especially under ecologically relevant conditions. Two versions of an attention-shifting and working memory updating task-one abstract, one ecological-were administered to 618 youth (M_age  = 13.62, SD_age  = 0.81; 48.22% female; 64.56% White). Measures of environmental unpredictability, violence, and poverty were collected to test adversity × task version interactions. There were no interactions for attention shifting. For working memory updating, youth exposed to violence and poverty scored lower than their peers with abstract stimuli but almost just as well with ecological stimuli. These results are striking compared to contemporary developmental science, which often reports lowered performance among adversity-exposed youth.

Navigating the manyverse of skin conductance response quantification approaches - A direct comparison of trough-to-peak, baseline correction, and model-based approaches in Ledalab and PsPM

Raw data are typically required to be processed to be ready for statistical analyses, and processing pipelines are often characterized by substantial heterogeneity. Here, we applied seven different approaches (trough-to-peak scoring by two different raters, script-based baseline correction, Ledalab as well as four different models implemented in the software PsPM) to two fear conditioning data sets. Selection of the approaches included was guided by a systematic literature search by using fear conditioning research as a case example. Our approach can be viewed as a set of robustness analyses (i.e., same data subjected to different processing pipelines) aiming to investigate if and to what extent these different quantification approaches yield comparable results given the same data. To our knowledge, no formal framework for the evaluation of robustness analyses exists to date, but we may borrow some criteria from a framework suggested for the evaluation of "replicability" in general. Our results from seven different SCR quantification approaches applied to two data sets with different paradigms suggest that there may be no single approach that consistently yields larger effect sizes and could be universally considered "best." Yet, at least some of the approaches employed show consistent effect sizes within each data set indicating comparability. Finally, we highlight substantial heterogeneity also within most quantification approaches and discuss implications and potential remedies.

How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses

COVID-19 research has relied heavily on convenience-based samples, which-though often necessary-are susceptible to important sampling biases. We begin with a theoretical overview and introduction to the dynamics that underlie sampling bias. We then empirically examine sampling bias in online COVID-19 surveys and evaluate the degree to which common statistical adjustments for demographic covariates successfully attenuate such bias. This registered study analysed responses to identical questions from three convenience and three largely representative samples (total N = 13,731) collected online in Canada within the International COVID-19 Awareness and Responses Evaluation Study ( www.icarestudy.com ). We compared samples on 11 behavioural and psychological outcomes (e.g., adherence to COVID-19 prevention measures, vaccine intentions) across three time points and employed multiverse-style analyses to examine how 512 combinations of demographic covariates (e.g., sex, age, education, income, ethnicity) impacted sampling discrepancies on these outcomes. Significant discrepancies emerged between samples on 73% of outcomes. Participants in the convenience samples held more positive thoughts towards and engaged in more COVID-19 prevention behaviours. Covariates attenuated sampling differences in only 55% of cases and increased differences in 45%. No covariate performed reliably well. Our results suggest that online convenience samples may display more positive dispositions towards COVID-19 prevention behaviours being studied than would samples drawn using more representative means. Adjusting results for demographic covariates frequently increased rather than decreased bias, suggesting that researchers should be cautious when interpreting adjusted findings. Using multiverse-style analyses as extended sensitivity analyses is recommended.

Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society's most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

Changes in Size and Interpretation of Parameter Estimates in Within-Person Models in the Presence of Time-Invariant and Time-Varying Covariates

For several decades, cross-lagged panel models (CLPM) have been the dominant statistical model in relationship research for investigating reciprocal associations between two (or more) constructs over time. However, recent methodological research has questioned the frequent usage of the CLPM because, amongst other things, the model commingles within-person associations with between-person associations, while most developmental research questions pertain to within-person processes. Furthermore, the model presumes that there are no third variables that confound the relationships between the longitudinally assessed variables. Therefore, the usage of alternative models such as the Random-Intercept Cross-Lagged Panel Model (RI-CLPM) or the Latent Curve Model with Structured Residuals (LCM-SR) has been suggested. These models separate between-person from within-person variation and they also control for time constant covariates. However, there might also be third variables that are not stable but rather change across time and that can confound the relationships between the variables studied in these models. In the present article, we explain the differences between the two types of confounders and investigate how they affect the parameter estimates of within-person models such as the RI-CLPM and the LCM-SR.