A generalized-weights solution to sample overlap in meta-analysis

Meta-studies are often conducted on empirical findings obtained from overlapping samples. Sample overlap is common in research fields that strongly rely on aggregated observational data (eg, economics and finance), where the same set of data may be used in several studies. More generally, sample overlap tends to occur whenever multiple estimates are sampled from the same study. We show analytically how failing to account for sample overlap causes high rates of false positives, especially for large meta-sample sizes. We propose a generalized-weights (GW) meta-estimator, which solves the sample overlap problem by explicitly modeling the variance-covariance matrix that describes the structure of dependence among estimates. We show how this matrix can be constructed from information that is usually available from basic sample descriptions in the primary studies (ie, sample sizes and number of overlapping observations). The GW meta-estimator amounts to weighting each empirical outcome according to its share of independent sampling information. We use Monte Carlo simulations to (a) demonstrate how the GW meta-estimator brings the rate of false positives to its nominal level, and (b) quantify the efficiency gains of the GW meta-estimator relative to standard meta-estimators. The GW meta-estimator is fairly straightforward to implement and can solve any case of sample overlap, within or between studies. HIGHLIGHTS: Meta-analyses are often conducted on empirical outcomes based on samples containing common observations. Sample overlap induces a correlation structure among empirical outcomes that harms the statistical properties of meta-analysis methods. We derive the analytic conditions under which sample overlap causes conventional meta-estimators to exhibit high rates of false positives. We propose a generalized-weights (GW) solution to sample overlap, which involves approximating the variance-covariance matrix that describes the correlation structure between outcomes; we show how to construct this matrix from information typically reported in the primary studies. We conduct Monte Carlo simulations to quantify the efficiency gains of the proposed GW estimator and show how it brings the rate of false positives near its nominal level. Although we focus on meta-analyses of regression coefficients, our approach can, in principle, be modified and extended to effect sizes more commonly used in other research fields, such as Cohen's d or odds ratios.

Neural networks: further insights into error function, generalized weights and others

The article is a continuum of a previous one providing further insights into the structure of neural network (NN). Key concepts of NN including activation function, error function, learning rate and generalized weights are introduced. NN topology can be visualized with generic plot() function by passing a "nn" class object. Generalized weights assist interpretation of NN model with respect to the independent effect of individual input variables. A large variance of generalized weights for a covariate indicates non-linearity of its independent effect. If generalized weights of a covariate are approximately zero, the covariate is considered to have no effect on outcome. Finally, prediction of new observations can be performed using compute() function. Make sure that the feature variables passed to the compute() function are in the same order to that in the training NN.