Distinguishing cause from effect in psychological research: An independence-based approach under linear non-Gaussian models

Distinguishing cause from effect - that is, determining whether x causes y (x → y) or, alternatively, whether y causes x (y → x) - is a primary research goal in many psychological research areas. Despite its importance, determining causal direction with observational data remains a difficult task. In this study, we introduce an independence-based approach for causal discovery between two variables of interest under a linear non-Gaussian model framework. We propose a two-step algorithm based on distance correlations that provides empirical conclusions on the causal directionality of effects under realistic conditions typically seen in psychological studies, that is, in the presence of hidden confounders. The performance of the proposed algorithm is evaluated using Monte-Carlo simulations. Findings suggest that the algorithm can effectively detect the causal direction between two variables of interest, even in the presence of weak hidden confounders. Moreover, distance correlations provide useful insights into the magnitude of hidden confounding. We provide an empirical example to demonstrate the application of our proposed approach and discuss practical implications and future directions.

Detecting heterogeneity in the causal direction of dependence: A model-based recursive partitioning approach

Methods of causal discovery and direction of dependence to evaluate causal properties of variable relations have experienced rapid development. The majority of causal discovery methods, however, relies on the assumption of causal effect homogeneity, that is, the identified causal structure is expected to hold for the entire population. Because causal mechanisms can vary across subpopulations, we propose combining methods of model-based recursive partitioning and non-Gaussian causal discovery to identify such subpopulations. The resulting algorithm can discover subpopulations with potentially varying magnitude and causal direction of effects under mild parameter inequality assumptions. Feasibility conditions are described and results from synthetic data experiments are presented suggesting that large effects and large sample sizes are beneficial for detecting causally competing subgroups with acceptable statistical performance. In a real-world data example, the extraction of meaningful subgroups that differ in the causal mechanism underlying the development of numerical cognition is illustrated. Potential extensions and recommendations for best practice applications are discussed.

Covariate selection in causal learning under non-Gaussianity

Understanding causal mechanisms is a central goal in the behavioral, developmental, and social sciences. When estimating and probing causal effects using observational data, covariate adjustment is a crucial element to remove dependencies between focal predictors and the error term. Covariate selection, however, constitutes a challenging task because availability alone is not an adequate criterion to decide whether a covariate should be included in the statistical model. The present study introduces a non-Gaussian method for covariate selection and provides a forward selection algorithm for linear models (i.e., non-Gaussian forward selection; nGFS) to select appropriate covariates from a set of potential control variables to avoid inconsistent and biased estimators of the causal effect of interest. Further, we demonstrate that the forward selection algorithm has properties compatible with principles of direction of dependence, i.e., probing whether the causal target model is correctly specified with respect to the causal direction of effects. Results of a Monte Carlo simulation study suggest that the selection algorithm performs well, in particular when sample sizes are large (i.e., n ≥ 250) and data strongly deviate from Gaussianity (e.g., distributions with skewness beyond 1.5). An empirical example is given for illustrative purposes.

Sleep quality and neurohormonal and psychophysiological accompanying factors in adolescents with depressive disorders: study protocol

BACKGROUND

Depressive disorders are common mental health problems during adolescence. Many adolescents with depression describe difficulties with sleeping. Findings of previous studies regarding changes in objective sleep quality in adolescents with depressive disorders are heterogeneous.

AIMS

This study aims to investigate differences in objective and subjective sleep quality between adolescents with depressive disorders and healthy peers, and to evaluate if potential changes in sleep occur concurrently with changes in the release of cortisol and alpha-amylase after awakening.

METHOD

This non-interventional parallel study examines correlations between depressive disorders, sleep quality and release of stress hormones. Sleep quality in the past 2 weeks, severity of depressive symptoms, psychiatric comorbidities and stress response of 30 adolescents with depressive disorders and 30 healthy controls (N = 60) are assessed via questionnaires. In participants' home environments, the objective sleep quality of seven consecutive nights is measured by sleep accelerometry. After awakening, participants answer sleep questionnaires to examine the subjective sleep quality of those nights. Furthermore, salivary cortisol and alpha-amylase are measured three times after awakening (+0 min, +30 min and +45 min after awakening).

CONCLUSIONS

Sleep is an important factor for prognosis and well-being in adolescents with depression. The results of this study can be highly valuable to integrate a more detailed examination of sleep quality and sleeping impairments in the treatment of adolescent depressive disorders.

Conditional Direction Dependence Analysis: Evaluating the Causal Direction of Effects in Linear Models with Interaction Terms

Direction dependence analysis (DDA) makes use of higher than second moment information of variables (x and y) to detect potential confounding and to probe the causal direction of linear variable relations (i.e., whether x → y or y → x better approximates the underlying causal mechanism). The "true" predictor is assumed to be a continuous nonnormal exogenous variable. Existing methods compatible with DDA, however, are of limited use when the relation of a focal predictor and an outcome is affected by a moderator. This study presents a conditional direction dependence analysis (CDDA) framework which enables researchers to evaluate the causal direction of conditional regression effects. Monte-Carlo simulations were used to evaluate two different moderation scenarios: Study 1 evaluates the performance of CDDA tests when a moderator affects the strength of the causal effect x → y. Study 2 evaluates cases in which the causal direction itself (x → y vs y → x) depends on moderator values. Study 3 evaluates the robustness of DDA tests in the presence of functional model misspecifications. Results suggest that significance tests compatible with CDDA are suitable in both moderation scenarios, i.e., CDDA allows one to discern regions of a moderator in which the causal direction is uniquely identifiable. An empirical example is provided to illustrate the approach.

Confounder detection in linear mediation models: Performance of kernel-based tests of independence

It is well-known that the identification of direct and indirect effects in mediation analysis requires strong unconfoundedness assumptions. Even when the predictor is under experimental control, unconfoundedness assumptions must be imposed on the mediator-outcome relation in order to guarantee valid indirect-effect identification. Researchers are therefore advised to test for unconfoundedness when estimating mediation effects. Significance tests to evaluate unconfoundedness usually rely on an instrumental variable (IV)-that is, a variable that is nonindependent of the explanatory variable and, at the same time, independent of all exogenous factors that affect the outcome when the explanatory variable is held constant. Because IVs may be hard to come by, the present study shows that confounders of the mediator-outcome relation can be detected without making use of IVs when variables are nonnormal. We show that kernel-based tests of independence are able to detect confounding under nonnormality. Results from a simulation study are presented that suggest that these tests perform well in terms of Type I error protection and statistical power, independent of the distribution or measurement level of the confounder. A real-world data example from the Job Search Intervention Study (JOBS II) illustrates how the presented approach can be used to minimize the risk of obtaining biased indirect-effect estimates. The data requirements and role of unconfoundedness tests as diagnostic tools are discussed. A Monte Carlo-based power analysis tool for sample size planning is also provided.

The Assumptions of Direction Dependence Analysis

Direction dependence analysis attempts to discern the direction of a causal effect, using statistical features of the data, such as skew and kurtosis of variables, and their residuals in regression models. Wiedermann and Sebastian discuss the use of this analysis in the context of mediation, and introduce methods to distinguish three different causal structures. In this commentary, I highlight some connections to literature in computer science, review the assumptions of the proposed analysis critically, and provide an example in which I argue that the analysis of Wiedermann and Sebastian can yield incorrect conclusions.

Direction Dependence Analysis in the Presence of Confounders: Applications to Linear Mediation Models Using Observational Data

Statistical methods to identify mis-specifications of linear regression models with respect to the direction of dependence (i.e. whether x→y or y→x better approximates the data-generating mechanism) have received considerable attention. Direction dependence analysis (DDA) constitutes such a statistical tool and makes use of higher-moment information of variables to derive statements concerning directional model mis-specifications in observational data. Previous studies on direction of dependence mainly focused on statistical inference and guidelines for the selection from the two directionally competing candidate models (x→y versus y→x) while assuming the absence of unobserved common causes. The present study describes properties of DDA when confounders are present and extends existing DDA methodology by incorporating the confounder model as a possible explanation. We show that all three explanatory models can be uniquely identified under standard DDA assumptions. Further, we discuss the proposed approach in the context of testing competing mediation models and evaluate an organizational model proposing a mediational relation between school leadership and student achievement via school safety using observational data from an urban school district. Overall, DDA provides strong empirical support that school safety has indeed a causal effect on student achievement but suggests that important confounders are present in the school leadership-safety relation.

Sensitivity Analysis and Extensions of Testing the Causal Direction of Dependence: A Rejoinder to Thoemmes (2019)

A commentary by Thoemmes on Wiedermann and Sebastian's introductory article on Direction Dependence Analysis (DDA) is responded to in the interest of elaborating and extending direction dependence principles to evaluate causal effect directionality. Considering Thoemmes' observation that some DDA principles are already well-established in machine learning, we argue that several other connections between DDA and research lines in theoretical statistics, econometrics, and quantitative psychology exist, suggesting that DDA is best conceptualized as a framework that summarizes and extends existing knowledge on properties of linear models under non-normality. Further, Thoemmes articulates concerns about assumptions of error distributions used in our main article and presents an artificial data example in which some DDA tests have suboptimal statistical power. We present extensions of DDA to entirely relax distributional assumptions about errors and describe two sensitivity analysis approaches to critically evaluate DDA results. Both sensitivity approaches are illustrated using Thoemmes' artificial data example. Incorporating DDA sensitivity results yields correct causal conclusions. Thus, overall, we stay with our initial conclusion that the use of higher moments in causal inference constitutes an exciting open research area.

Back to the future: Considerations in use and reporting of the retrospective pretest

Retrospective pretests ask respondents to report after an intervention on their aptitudes, knowledge, or beliefs before the intervention. A primary reason to administer a retrospective pretest is that in some situations, program participants may over the course of an intervention revise or recalibrate their prior understanding of program content, with the result that their posttest scores are lower than their traditional pretest scores, even though their understanding or abilities have increased. This phenomenon is called response-shift bias. The existence of response-shift bias is undisputed, but it does not always occur, and use of the retrospective pretest in place of a traditional pretest often introduces new problems. In this commentary, I provide a brief overview of the literature on response-shift bias and discuss common pitfalls in the use and reporting of retrospective pretest results, including a failure to consider multiple factors that may affect all test scores, as well as claims that retrospective pretests are less biased than traditional pretests, provide more accurate estimates of effects, and are necessarily superior to traditional pretests in program evaluation. I comment on the article by Little et al. (2019) in this issue in light of the literature on retrospective pretests and discuss the need for a theoretical framework to guide research on response-shift bias. The goal of the commentary is to provide readers with an informed and critical lens through which to evaluate and use retrospective pretest methods.

Direction dependence analysis: A framework to test the direction of effects in linear models with an implementation in SPSS

In nonexperimental data, at least three possible explanations exist for the association of two variables x and y: (1) x is the cause of y, (2) y is the cause of x, or (3) an unmeasured confounder is present. Statistical tests that identify which of the three explanatory models fits best would be a useful adjunct to the use of theory alone. The present article introduces one such statistical method, direction dependence analysis (DDA), which assesses the relative plausibility of the three explanatory models on the basis of higher-moment information about the variables (i.e., skewness and kurtosis). DDA involves the evaluation of three properties of the data: (1) the observed distributions of the variables, (2) the residual distributions of the competing models, and (3) the independence properties of the predictors and residuals of the competing models. When the observed variables are nonnormally distributed, we show that DDA components can be used to uniquely identify each explanatory model. Statistical inference methods for model selection are presented, and macros to implement DDA in SPSS are provided. An empirical example is given to illustrate the approach. Conceptual and empirical considerations are discussed for best-practice applications in psychological data, and sample size recommendations based on previous simulation studies are provided.

Testing the Causal Direction of Mediation Effects in Randomized Intervention Studies

In a recent update of the standards for evidence in research on prevention interventions, the Society of Prevention Research emphasizes the importance of evaluating and testing the causal mechanism through which an intervention is expected to have an effect on an outcome. Mediation analysis is commonly applied to study such causal processes. However, these analytic tools are limited in their potential to fully understand the role of theorized mediators. For example, in a design where the treatment x is randomized and the mediator (m) and the outcome (y) are measured cross-sectionally, the causal direction of the hypothesized mediator-outcome relation is not uniquely identified. That is, both mediation models, x → m → y or x → y → m, may be plausible candidates to describe the underlying intervention theory. As a third explanation, unobserved confounders can still be responsible for the mediator-outcome association. The present study introduces principles of direction dependence which can be used to empirically evaluate these competing explanatory theories. We show that, under certain conditions, third higher moments of variables (i.e., skewness and co-skewness) can be used to uniquely identify the direction of a mediator-outcome relation. Significance procedures compatible with direction dependence are introduced and results of a simulation study are reported that demonstrate the performance of the tests. An empirical example is given for illustrative purposes and a software implementation of the proposed method is provided in SPSS.

Locating Event-Based Causal Effects: A Configural Perspective

Statistical models for the analysis of hypotheses that are compatible with direction dependence were originally specified based on the linear model. In these models, relations among variables reflected directional or causal hypotheses. In a number of causal theories, however, effects are defined as resulting from causes that did versus did not occur. To accommodate this type of theory, the present article proposes analyzing directional or causal hypotheses at the level of configurations. Causes thus have the effect that, in a particular sector of the data space, the density of cases increases or decreases. With reference to log-linear models of direction dependence, this article specifies base models for the configural analysis of directional or causal hypotheses. In contrast to standard configural analysis, the models are applied in a confirmatory context. Specific direction dependence hypotheses are analyzed. In a simulation study, it is shown that the proposed methods have good power to identify the sectors in the data space in which density exceeds or falls below expectation. In a data example, it is shown that the evolutionary hypothesis that body size determines brain size is confirmed in particular for higher vertebrates.

Direction of dependence in measurement error models

Methods to determine the direction of a regression line, that is, to determine the direction of dependence in reversible linear regression models (e.g., x→y vs. y→x), have experienced rapid development within the last decade. However, previous research largely rested on the assumption that the true predictor is measured without measurement error. The present paper extends the direction dependence principle to measurement error models. First, we discuss asymmetric representations of the reliability coefficient in terms of higher moments of variables and the attenuation of skewness and excess kurtosis due to measurement error. Second, we identify conditions where direction dependence decisions are biased due to measurement error and suggest method of moments (MOM) estimation as a remedy. Third, we address data situations in which the true outcome exhibits both regression and measurement error, and propose a sensitivity analysis approach to determining the robustness of direction dependence decisions against unreliably measured outcomes. Monte Carlo simulations were performed to assess the performance of MOM-based direction dependence measures and their robustness to violated measurement error assumptions (i.e., non-independence and non-normality). An empirical example from subjective well-being research is presented. The plausibility of model assumptions and links to modern causal inference methods for observational data are discussed.

Heteroscedasticity as a Basis of Direction Dependence in Reversible Linear Regression Models

Heteroscedasticity is a well-known issue in linear regression modeling. When heteroscedasticity is observed, researchers are advised to remedy possible model misspecification of the explanatory part of the model (e.g., considering alternative functional forms and/or omitted variables). The present contribution discusses another source of heteroscedasticity in observational data: Directional model misspecifications in the case of nonnormal variables. Directional misspecification refers to situations where alternative models are equally likely to explain the data-generating process (e.g., x → y versus y → x). It is shown that the homoscedasticity assumption is likely to be violated in models that erroneously treat true nonnormal predictors as response variables. Recently, Direction Dependence Analysis (DDA) has been proposed as a framework to empirically evaluate the direction of effects in linear models. The present study links the phenomenon of heteroscedasticity with DDA and describes visual diagnostics and nine homoscedasticity tests that can be used to make decisions concerning the direction of effects in linear models. Results of a Monte Carlo simulation that demonstrate the adequacy of the approach are presented. An empirical example is provided, and applicability of the methodology in cases of violated assumptions is discussed.

Testing Event-Based Forms of Causality

Three fundamental types of causal relations are those of necessity, sufficiency, and necessity and sufficiency. These types are defined in contexts of categorical variables or events. Using statement calculus or Boolean algebra, one can determine which patterns of events are in support of a particular form of causal relation. In this article, we approach the analysis of these forms of causality taking the perspective of the analyst of empirical data. It is proposed using Configural Frequency Analysis (CFA) to test hypotheses about type of causal relation. Models are proposed for two-variable and multi-variable cases. Two CFA approaches are proposed. In the first, individual patterns (configurations) are examined under the question whether they are in support of a particular type of causal relation. In the second, patterns that are in support are compared with corresponding patterns that are not in support. In an empirical example, hypotheses are tested on the prediction of sustainability of change in dietary fat intake habits.