A simple method for inference on an overall effect in meta-analysis

Jointly pooling aggregated effect sizes and their standard errors from studies with continuous clinical outcomes

The DerSimonian-Laird (DL) weighted average method for aggregated data meta-analysis has been widely used for the estimation of overall effect sizes. It is criticized for its underestimation of the standard error of the overall effect size in the presence of heterogeneous effect sizes. Due to this negative property, many alternative estimation approaches have been proposed in the literature. One of the earliest alternative approaches was developed by Hardy and Thompson (HT), who implemented a profile likelihood instead of the moment-based approach of DL. Others have further extended this likelihood approach and proposed higher-order likelihood inferences (e.g., Bartlett-type corrections). In addition, corrections factors for the estimated DL standard error, like the Hartung-Knapp-Sidik-Jonkman (HKSJ) adjustment, and the restricted maximum likelihood (REML) estimation have been suggested too. Although these improvements address the uncertainty in estimating the between-study variance better than the DL method, they all assume that the true within-study standard errors are known and equal to the observed standard errors of the effect sizes. Here, we will treat the observed standard errors as estimators for the within-study variability and we propose a bivariate likelihood approach that jointly estimates the overall effect size, the between-study variance, and the potentially heteroskedastic within-study variances. We study the performance of the proposed method by means of simulation, and compare it to DL (with and without HKSJ), HT, their higher-order likelihood methods, and REML. Our proposed approach seems to have better or similar coverages compared to the other approaches and it appears to be less biased in the case of heteroskedastic within-study variances when this heteroskedasticty is correlated with the effect size.

Laying hen mortality in different indoor housing systems: a meta-analysis of data from commercial farms in 16 countries

Societal concern with the welfare of egg laying hens housed in conventional cages is fostering a transition towards cage-free systems in many countries. However, although cage-free facilities enable hens to move freely and express natural behaviours, concerns have also been raised over the possibility that cage-free flocks experience higher mortality, potentially compromising some aspects of their welfare. To investigate this possibility, we conducted a large meta-analysis of laying hen mortality in conventional cages, furnished cages and cage-free aviaries using data from 6040 commercial flocks and 176 million hens from 16 countries. We show that except for conventional cages, mortality gradually drops as experience with each system builds up: since 2000, each year of experience with cage-free aviaries was associated with a 0.35–0.65% average drop in cumulative mortality, with no differences in mortality between caged and cage-free systems in more recent years. As management knowledge evolves and genetics are optimized, new producers transitioning to cage-free housing may experience even faster rates of decline. Our results speak against the notion that mortality is inherently higher in cage-free production and illustrate the importance of considering the degree of maturity of production systems in any investigations of farm animal health, behaviour and welfare.

Interval estimation of the overall treatment effect in random-effects meta-analyses: Recommendations from a simulation study comparing frequentist, Bayesian, and bootstrap methods

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study is still lacking. Thus, we conduct such a simulation study for continuous and binary outcomes, focusing on the medical field for application. Based on the literature review and some new theoretical considerations, a practicable number of interval estimators is selected for this comparison: the classical normal-approximation interval using the DerSimonian-Laird heterogeneity estimator, the HKSJ interval using either the Paule-Mandel or the Sidik-Jonkman heterogeneity estimator, the Skovgaard higher-order profile likelihood interval, a parametric bootstrap interval, and a Bayesian interval using different priors. We evaluate the performance measures (coverage and interval length) at specific points in the parameter space, that is, not averaging over a prior distribution. In this sense, our study is conducted from a frequentist point of view. We confirm the main finding of the literature review, the general recommendation of the HKSJ method (here with the Sidik-Jonkman heterogeneity estimator). For meta-analyses including only two studies, the high length of the HKSJ interval limits its practical usage. In this case, the Bayesian interval using a weakly informative prior for the heterogeneity may help. Our recommendations are illustrated using a real-world meta-analysis dealing with the efficacy of an intramyocardial bone marrow stem cell transplantation during coronary artery bypass grafting.

Frequentist performances of Bayesian prediction intervals for random-effects meta-analysis

The prediction interval has been increasingly used in meta-analyses as a useful measure for assessing the magnitude of treatment effect and between-studies heterogeneity. In calculations of the prediction interval, although the Higgins-Thompson-Spiegelhalter method is used most often in practice, it might not have adequate coverage probability for the true treatment effect of a future study under realistic situations. An effective alternative candidate is the Bayesian prediction interval, which has also been widely used in general prediction problems. However, these prediction intervals are constructed based on the Bayesian philosophy, and their frequentist validities are only justified by large-sample approximations even if noninformative priors are adopted. There has been no certain evidence that evaluated their frequentist performances under realistic situations of meta-analyses. In this study, we conducted extensive simulation studies to assess the frequentist coverage performances of Bayesian prediction intervals with 11 noninformative prior distributions under general meta-analysis settings. Through these simulation studies, we found that frequentist coverage performances strongly depended on what prior distributions were adopted. In addition, when the number of studies was smaller than 10, there were no prior distributions that retained accurate frequentist coverage properties. We also illustrated these methods via applications to two real meta-analysis datasets. The resultant prediction intervals also differed according to the adopted prior distributions. Inaccurate prediction intervals may provide invalid evidence and misleading conclusions. Thus, if frequentist accuracy is required, Bayesian prediction intervals should be used cautiously in practice.

Clinical, laboratory and radiological characteristics and outcomes of novel coronavirus (SARS-CoV-2) infection in humans: A systematic review and series of meta-analyses

New evidence on the COVID-19 pandemic is being published daily. Ongoing high-quality assessment of this literature is therefore needed to enable clinical practice to be evidence-based. This review builds on a previous scoping review and aimed to identify associations between disease severity and various clinical, laboratory and radiological characteristics. We searched MEDLINE, CENTRAL, EMBASE, Scopus and LILACS for studies published between January 1, 2019 and March 22, 2020. Clinical studies including ≥10 patients with confirmed COVID-19 of any study design were eligible. Two investigators independently extracted data and assessed risk of bias. A quality effects model was used for the meta-analyses. Subgroup analysis and meta-regression identified sources of heterogeneity. For hospitalized patients, studies were ordered by overall disease severity of each population and this order was used as the modifier variable in meta-regression. Overall, 86 studies (n = 91,621) contributed data to the meta-analyses. Severe disease was strongly associated with fever, cough, dyspnea, pneumonia, any computed tomography findings, any ground glass opacity, lymphocytopenia, elevated C-reactive protein, elevated alanine aminotransferase, elevated aspartate aminotransferase, older age and male sex. These variables typically increased in prevalence by 30-73% from mild/early disease through to moderate/severe disease. Among hospitalized patients, 30-78% of heterogeneity was explained by severity of disease. Elevated white blood cell count was strongly associated with more severe disease among moderate/severe hospitalized patients. Elevated lymphocytes, low platelets, interleukin-6, erythrocyte sedimentation rate and D-dimers showed potential associations, while fatigue, gastrointestinal symptoms, consolidation and septal thickening showed non-linear association patterns. Headache and sore throat were associated with the presence of disease, but not with more severe disease. In COVID-19, more severe disease is strongly associated with several clinical, laboratory and radiological characteristics. Symptoms and other variables in early/mild disease appear non-specific and highly heterogeneous. Clinical Trial Registration: PROSPERO CRD42020170623.

Road Traffic Noise Exposure and Depression/Anxiety: An Updated Systematic Review and Meta-Analysis

Unlike other World Health Organization evidence reviews, the systematic review on mental disorders could not provide a quantitative estimate of the effect of environmental noise. With that in mind, we aimed to update it with additional studies published through to 18 August 2019 in order to allow for a formal meta-analysis of the association of residential road traffic noise with anxiety and depression. The quality effects and random effects estimators were used for meta-analysis and the robustness of findings was tested in several sensitivity analyses. Ten studies were included in the qualitative synthesis, from which we extracted 15 estimates for depression (n = 1,201,168) and five for anxiety (n = 372,079). Almost all studies were cross-sectional and the risk of bias in them was generally high. We found 4% (95% CI: -3%, 11%) higher odds of depression and 12% (95% CI: -4%, 30%) of anxiety associated with a 10 dB(A) increase in day-evening-night noise level (Lden). Both models suffered from moderate heterogeneity (55% and 54%), but there was evidence of publication bias only in the depression model. These findings were robust with no evidence of study-level moderators. A sensitivity analysis on an alternative set of categorically-reported estimates supported a linear relationship between Lden and depression. Taking into account an overall quality assessment for the included studies, we conclude that there is evidence of "very low" quality that increasing exposure to road traffic noise may be associated with depression and anxiety.

Road Traffic Noise Exposure and Birth Outcomes: An Updated Systematic Review and Meta-Analysis

Unlike the other WHO evidence reviews, the systematic review on birth outcomes could not provide a quantitative estimate of the effect of environmental noise. With that in mind, we aimed to update it with additional studies published through to 12 May, 2019 to allow for a formal meta-analysis of the association of residential road traffic noise with birth weight, low birth weight (LBW), small for gestational age (SGA), and preterm birth (PTB). The quality effects and random effects estimators were used for meta-analysis and the robustness of findings was tested in several sensitivity analyses. Nine studies were included in the qualitative synthesis, from which we extracted seven estimates for birth weight (n = 718,136 births) and LBW (n = 620,221), and five for SGA (n = 547,256) and PTB (n = 74,609). We found -8.26 g (95% CI: -20.61 g, 4.10 g) (I2 = 87%) lower birth weight associated with a 10 dB(A) increase in day-evening-night noise level (Lden), and this effect became significant in sensitivity analyses. No evidence of significant effects was found for LBW (OR = 1.06; 95% CI: 0.91, 1.23) (I2 = 49%), SGA (OR = 1.02; 95% CI: 0.86, 1.21) (I2 = 90%), or PTB (OR = 1.00; 95% CI: 0.79, 1.27) (I2 = 69%). The quality of evidence for continuous birth weight was graded as "moderate", while for the other outcomes it was deemed "very low". Finally, we discuss limitations of the risk of bias assessment criteria employed by Nieuwenhuijsen et al.

Residential road traffic noise as a risk factor for hypertension in adults: Systematic review and meta-analysis of analytic studies published in the period 2011-2017

Multiple cross-sectional studies indicated an association between hypertension and road traffic noise and they were recently synthetized in a WHO systematic evidence review. However, recent years have seen a growing body of high-quality, large-scale research, which is missing from the WHO review. Therefore, we aimed to close that gap by conducting an updated systematic review and meta-analysis on the exposure-response relationship between residential road traffic noise and the risk of hypertension in adults. Studies were identified by searching MEDLINE, EMBASE, the Internet, conference proceedings, reference lists, and expert archives in English, Russian, and Spanish through August 5, 2017. The risk of bias for each extracted estimate and the overall quality of evidence were evaluated using a list of predefined safeguards against bias related to different study characteristics and the Grading of Recommendations Assessment, Development and Evaluation system, respectively. The inverse variance heterogeneity (IVhet) model was used for meta-analysis. The possibility of publication bias was evaluated by funnel and Doi plots, and asymmetry in these was tested with Egger's test and the Luis Furuya-Kanamori index, respectively. Sensitivity analyses included leave-one-out meta-analysis, subgroup meta-analysis with meta-regressions, and non-linear exposure-response meta-analysis. Based on seven cohort and two case-control studies (n = 5 514 555; 14 estimates; L_den range ≈ 25-90 dB(A)), we found "low" evidence of RR _{per 10 dB(A)} = 1.018 (95% CI: 0.984, 1.053), moderate heterogeneity (I² = 46%), and no publication bias. In the subgroup of cohort studies, we found "moderate" evidence of RR _{per 10 dB(A)} = 1.018 (95% CI: 0.987, 1.049), I² = 31%, and no publication bias. In conclusion, residential road traffic noise was associated with higher risk of hypertension in adults, but the risk was lower than previously reported in the systematic review literature.

Prediction intervals for random-effects meta-analysis: A confidence distribution approach

Prediction intervals are commonly used in meta-analysis with random-effects models. One widely used method, the Higgins–Thompson–Spiegelhalter prediction interval, replaces the heterogeneity parameter with its point estimate, but its validity strongly depends on a large sample approximation. This is a weakness in meta-analyses with few studies. We propose an alternative based on bootstrap and show by simulations that its coverage is close to the nominal level, unlike the Higgins–Thompson–Spiegelhalter method and its extensions. The proposed method was applied in three meta-analyses.

A comparison of seven random-effects models for meta-analyses that estimate the summary odds ratio

Comparative trials that report binary outcome data are commonly pooled in systematic reviews and meta-analyses. This type of data can be presented as a series of 2-by-2 tables. The pooled odds ratio is often presented as the outcome of primary interest in the resulting meta-analysis. We examine the use of 7 models for random-effects meta-analyses that have been proposed for this purpose. The first of these models is the conventional one that uses normal within-study approximations and a 2-stage approach. The other models are generalised linear mixed models that perform the analysis in 1 stage and have the potential to provide more accurate inference. We explore the implications of using these 7 models in the context of a Cochrane Review, and we also perform a simulation study. We conclude that generalised linear mixed models can result in better statistical inference than the conventional 2-stage approach but also that this type of model presents issues and difficulties. These challenges include more demanding numerical methods and determining the best way to model study specific baseline risks. One possible approach for analysts is to specify a primary model prior to performing the systematic review but also to present the results using other models in a sensitivity analysis. Only one of the models that we investigate is found to perform poorly so that any of the other models could be considered for either the primary or the sensitivity analysis.

Extending DerSimonian and Laird's methodology to perform network meta-analyses with random inconsistency effects

Network meta-analysis is becoming more popular as a way to compare multiple treatments simultaneously. Here, we develop a new estimation method for fitting models for network meta-analysis with random inconsistency effects. This method is an extension of the procedure originally proposed by DerSimonian and Laird. Our methodology allows for inconsistency within the network. The proposed procedure is semi-parametric, non-iterative, fast and highly accessible to applied researchers. The methodology is found to perform satisfactorily in a simulation study provided that the sample size is large enough and the extent of the inconsistency is not very severe. We apply our approach to two real examples.

Empirical evidence about inconsistency among studies in a pair-wise meta-analysis

This paper investigates how inconsistency (as measured by the I² statistic) among studies in a meta‐analysis may differ, according to the type of outcome data and effect measure. We used hierarchical models to analyse data from 3873 binary, 5132 continuous and 880 mixed outcome meta‐analyses within the Cochrane Database of Systematic Reviews. Predictive distributions for inconsistency expected in future meta‐analyses were obtained, which can inform priors for between‐study variance. Inconsistency estimates were highest on average for binary outcome meta‐analyses of risk differences and continuous outcome meta‐analyses. For a planned binary outcome meta‐analysis in a general research setting, the predictive distribution for inconsistency among log odds ratios had median 22% and 95% CI: 12% to 39%. For a continuous outcome meta‐analysis, the predictive distribution for inconsistency among standardized mean differences had median 40% and 95% CI: 15% to 73%. Levels of inconsistency were similar for binary data measured by log odds ratios and log relative risks. Fitted distributions for inconsistency expected in continuous outcome meta‐analyses using mean differences were almost identical to those using standardized mean differences. The empirical evidence on inconsistency gives guidance on which outcome measures are most likely to be consistent in particular circumstances and facilitates Bayesian meta‐analysis with an informative prior for heterogeneity. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

Approximate confidence intervals for moment-based estimators of the between-study variance in random effects meta-analysis

Moment‐based estimators of the between‐study variance are very popular when performing random effects meta‐analyses. This type of estimation has many advantages including computational and conceptual simplicity. Furthermore, by using these estimators in large samples, valid meta‐analyses can be performed without the assumption that the treatment effects follow a normal distribution. Recently proposed moment‐based confidence intervals for the between‐study variance are exact under the random effects model but are quite elaborate. Here, we present a much simpler method for calculating approximate confidence intervals of this type. This method uses variance‐stabilising transformations as its basis and can be used for a very wide variety of moment‐based estimators in both the random effects meta‐analysis and meta‐regression models. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

Advances in the meta-analysis of heterogeneous clinical trials I: The inverse variance heterogeneity model

This article examines an improved alternative to the random effects (RE) model for meta-analysis of heterogeneous studies. It is shown that the known issues of underestimation of the statistical error and spuriously overconfident estimates with the RE model can be resolved by the use of an estimator under the fixed effect model assumption with a quasi-likelihood based variance structure - the IVhet model. Extensive simulations confirm that this estimator retains a correct coverage probability and a lower observed variance than the RE model estimator, regardless of heterogeneity. When the proposed IVhet method is applied to the controversial meta-analysis of intravenous magnesium for the prevention of mortality after myocardial infarction, the pooled OR is 1.01 (95% CI 0.71-1.46) which not only favors the larger studies but also indicates more uncertainty around the point estimate. In comparison, under the RE model the pooled OR is 0.71 (95% CI 0.57-0.89) which, given the simulation results, reflects underestimation of the statistical error. Given the compelling evidence generated, we recommend that the IVhet model replace both the FE and RE models. To facilitate this, it has been implemented into free meta-analysis software called MetaXL which can be downloaded from www.epigear.com.

A non-iterative extension of the multivariate random effects meta-analysis

Multivariate methods in meta-analysis are becoming popular and more accepted in biomedical research despite computational issues in some of the techniques. A number of approaches, both iterative and non-iterative, have been proposed including the multivariate DerSimonian and Laird method by Jackson et al. (2010), which is non-iterative. In this study, we propose an extension of the method by Hartung and Makambi (2002) and Makambi (2001) to multivariate situations. A comparison of the bias and mean square error from a simulation study indicates that, in some circumstances, the proposed approach perform better than the multivariate DerSimonian-Laird approach. An example is presented to demonstrate the application of the proposed approach.

The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method

BACKGROUND

The DerSimonian and Laird approach (DL) is widely used for random effects meta-analysis, but this often results in inappropriate type I error rates. The method described by Hartung, Knapp, Sidik and Jonkman (HKSJ) is known to perform better when trials of similar size are combined. However evidence in realistic situations, where one trial might be much larger than the other trials, is lacking. We aimed to evaluate the relative performance of the DL and HKSJ methods when studies of different sizes are combined and to develop a simple method to convert DL results to HKSJ results.

METHODS

We evaluated the performance of the HKSJ versus DL approach in simulated meta-analyses of 2-20 trials with varying sample sizes and between-study heterogeneity, and allowing trials to have various sizes, e.g. 25% of the trials being 10-times larger than the smaller trials. We also compared the number of "positive" (statistically significant at p < 0.05) findings using empirical data of recent meta-analyses with > = 3 studies of interventions from the Cochrane Database of Systematic Reviews.

RESULTS

The simulations showed that the HKSJ method consistently resulted in more adequate error rates than the DL method. When the significance level was 5%, the HKSJ error rates at most doubled, whereas for DL they could be over 30%. DL, and, far less so, HKSJ had more inflated error rates when the combined studies had unequal sizes and between-study heterogeneity. The empirical data from 689 meta-analyses showed that 25.1% of the significant findings for the DL method were non-significant with the HKSJ method. DL results can be easily converted into HKSJ results.

CONCLUSIONS

Our simulations showed that the HKSJ method consistently results in more adequate error rates than the DL method, especially when the number of studies is small, and can easily be applied routinely in meta-analyses. Even with the HKSJ method, extra caution is needed when there are = <5 studies of very unequal sizes.

The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method

BACKGROUND

The DerSimonian and Laird approach (DL) is widely used for random effects meta-analysis, but this often results in inappropriate type I error rates. The method described by Hartung, Knapp, Sidik and Jonkman (HKSJ) is known to perform better when trials of similar size are combined. However evidence in realistic situations, where one trial might be much larger than the other trials, is lacking. We aimed to evaluate the relative performance of the DL and HKSJ methods when studies of different sizes are combined and to develop a simple method to convert DL results to HKSJ results.

METHODS

We evaluated the performance of the HKSJ versus DL approach in simulated meta-analyses of 2-20 trials with varying sample sizes and between-study heterogeneity, and allowing trials to have various sizes, e.g. 25% of the trials being 10-times larger than the smaller trials. We also compared the number of "positive" (statistically significant at p < 0.05) findings using empirical data of recent meta-analyses with > = 3 studies of interventions from the Cochrane Database of Systematic Reviews.

RESULTS

The simulations showed that the HKSJ method consistently resulted in more adequate error rates than the DL method. When the significance level was 5%, the HKSJ error rates at most doubled, whereas for DL they could be over 30%. DL, and, far less so, HKSJ had more inflated error rates when the combined studies had unequal sizes and between-study heterogeneity. The empirical data from 689 meta-analyses showed that 25.1% of the significant findings for the DL method were non-significant with the HKSJ method. DL results can be easily converted into HKSJ results.

CONCLUSIONS

Our simulations showed that the HKSJ method consistently results in more adequate error rates than the DL method, especially when the number of studies is small, and can easily be applied routinely in meta-analyses. Even with the HKSJ method, extra caution is needed when there are = <5 studies of very unequal sizes.

Confidence intervals for the between-study variance in random effects meta-analysis using generalised Cochran heterogeneity statistics

Statistical inference is problematic in the common situation in meta-analysis where the random effects model is fitted to just a handful of studies. In particular, the asymptotic theory of maximum likelihood provides a poor approximation, and Bayesian methods are sensitive to the prior specification. Hence, less efficient, but easily computed and exact, methods are an attractive alternative. Here, methodology is developed to compute exact confidence intervals for the between-study variance using generalised versions of Cochran's heterogeneity statistic. If some between-study is anticipated, but it is unclear how much, then a pragmatic approach is to use the reciprocals of the within-study standard errors as weights when computing the confidence interval.

A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews

Multivariate meta-analysis allows the joint synthesis of multiple correlated outcomes from randomised trials, and is an alternative to a separate univariate meta-analysis of each outcome independently. Usually not all trials report all outcomes; furthermore, outcome reporting bias (ORB) within trials, where an outcome is measured and analysed but not reported on the basis of the results, may cause a biased set of the evidence to be available for some outcomes, potentially affecting the significance and direction of meta-analysis results. The multivariate approach, however, allows one to 'borrow strength' across correlated outcomes, to potentially reduce the impact of ORB. Assuming ORB missing data mechanisms, we aim to investigate the magnitude of bias in the pooled treatment effect estimates for multiple outcomes using univariate meta-analysis, and to determine whether the 'borrowing of strength' from multivariate meta-analysis can reduce the impact of ORB. A simulation study was conducted for a bivariate fixed effect meta-analysis of two correlated outcomes. The approach is illustrated by application to a Cochrane systematic review. Results show that the 'borrowing of strength' from a multivariate meta-analysis can reduce the impact of ORB on the pooled treatment effect estimates. We also examine the use of the Pearson correlation as a novel approach for dealing with missing within-study correlations, and provide an extension to bivariate random-effects models that reduce ORB in the presence of heterogeneity.

Confidence intervals for random effects meta-analysis and robustness to publication bias

The DerSimonian-Laird confidence interval for the average treatment effect in meta-analysis is widely used in practice when there is heterogeneity between studies. However, it is well known that its coverage probability (the probability that the interval actually includes the true value) can be substantially below the target level of 95 per cent. It can also be very sensitive to publication bias. In this paper, we propose a new confidence interval that has better coverage than the DerSimonian-Laird method, and that is less sensitive to publication bias. The key idea is to note that fixed effects estimates are less sensitive to such biases than random effects estimates, since they put relatively more weight on the larger studies and relatively less weight on the smaller studies. Whereas the DerSimonian-Laird interval is centred on a random effects estimate, we centre our confidence interval on a fixed effects estimate, but allow for heterogeneity by including an assessment of the extra uncertainty induced by the random effects setting. Properties of the resulting confidence interval are studied by simulation and compared with other random effects confidence intervals that have been proposed in the literature. An example is briefly discussed.

Extending DerSimonian and Laird's methodology to perform multivariate random effects meta-analyses

Multivariate meta-analysis is increasingly used in medical statistics. In the univariate setting, the non-iterative method proposed by DerSimonian and Laird is a simple and now standard way of performing random effects meta-analyses. We propose a natural and easily implemented multivariate extension of this procedure which is accessible to applied researchers and provides a much less computationally intensive alternative to existing methods. In a simulation study, the proposed procedure performs similarly in almost all ways to the more established iterative restricted maximum likelihood approach. The method is applied to some real data sets and an extension to multivariate meta-regression is described.

Random-effects meta-analysis of correlations: Monte Carlo evaluation of mean estimators

Several authors have cautioned against using Fisher's z-transformation in random-effects meta-analysis of correlations, which seems to perform poorly in some situations, especially with substantial inter-study heterogeneity. Attributing this performance largely to the direct z-to-r transformation (DZRT) of Fisher z results (e.g. point estimate of mean correlation), in a previous paper Hafdahl (2009) proposed point and interval estimators of the mean Pearson r correlation that instead use an integral z-to-r transformation (IZRT). The present Monte Carlo study of these IZRT Fisher z estimators includes comparisons with their DZRT counterparts and with estimators based on Pearson r correlations. The IZRT point estimator was usually more accurate and efficient than its DZRT counterpart and comparable to the two Pearson r point estimators - better in some conditions but worse in others. Coverage probability for the IZRT confidence intervals (CIs) was often near nominal, much better than for the DZRT CIs, and comparable to coverage for the Pearson r CIs; every approach's CI fell markedly below nominal in some conditions. The IZRT estimators contradict warnings about Fisher z estimators' poor performance. Recommendations for practising research synthesists are offered, and an Appendix provides computing code to implement the IZRT as in the real-data example.

NONPARAMETRIC INFERENCE PROCEDURE FOR PERCENTILES OF THE RANDOM EFFECTS DISTRIBUTION IN META-ANALYSIS

To investigate whether treating cancer patients with erythropoiesis-stimulating agents (ESAs) would increase the mortality risk, Bennett et al. [Journal of the American Medical Association299 (2008) 914-924] conducted a meta-analysis with the data from 52 phase III trials comparing ESAs with placebo or standard of care. With a standard parametric random effects modeling approach, the study concluded that ESA administration was significantly associated with increased average mortality risk. In this article we present a simple nonparametric inference procedure for the distribution of the random effects. We re-analyzed the ESA mortality data with the new method. Our results about the center of the random effects distribution were markedly different from those reported by Bennett et al. Moreover, our procedure, which estimates the distribution of the random effects, as opposed to just a simple population average, suggests that the ESA may be beneficial to mortality for approximately a quarter of the study populations. This new meta-analysis technique can be implemented with study-level summary statistics. In contrast to existing methods for parametric random effects models, the validity of our proposal does not require the number of studies involved to be large. From the results of an extensive numerical study, we find that the new procedure performs well even with moderate individual study sample sizes.

A re-evaluation of the 'quantile approximation method' for random effects meta-analysis

The quantile approximation method has recently been proposed as a simple method for deriving confidence intervals for the treatment effect in a random effects meta-analysis. Although easily implemented, the quantiles used to construct intervals are derived from a single simulation study. Here it is shown that altering the study parameters, and in particular introducing changes to the distribution of the within-study variances, can have a dramatic impact on the resulting quantiles. This is further illustrated analytically by examining the scenario where all trials are assumed to be the same size. A more cautious approach is therefore suggested, where the conventional standard normal quantile is used in the primary analysis, but where the use of alternative quantiles is also considered in a sensitivity analysis. Copyright © 2008 John Wiley & Sons, Ltd.