The specification of the propensity score in multilevel observational studies

Conditioning on the Pre-Test versus Gain Score Modelling: Revisiting the Controversy in a Multilevel Setting

We consider estimating the effect of a treatment on a given outcome measured on subjects tested both before and after treatment assignment in observational studies. A vast literature compares the competing approaches of modelling the post-test score conditionally on the pre-test score versus modelling the difference, namely, the gain score. Our contribution lies in analyzing the merits and drawbacks of two approaches in a multilevel setting. This is relevant in many fields, such as education, where students are nested within schools. The multilevel structure raises peculiar issues related to contextual effects and the distinction between individual-level and cluster-level treatments. We compare the two approaches through a simulation study. For individual-level treatments, our findings align with existing literature. However, for cluster-level treatments, the scenario is more complex, as the cluster mean of the pre-test score plays a key role. Its reliability crucially depends on the cluster size, leading to potentially unsatisfactory estimators with small clusters.

Impact of crop commercialization on multidimensional poverty in rural Ethiopia: propensity score approach

Introduction

Reducing poverty through crop commercialization is one of the antipoverty efforts that helps promote health. This study explored the prevalence and the causal relationship between crop commercialization and rural Ethiopian households' multidimensional poverty using multilevel data.

Methods

The study uses data from the most recent nationally representative Ethiopian socioeconomic survey 2018/19 to calculate the rural multidimensional poverty index using the Alkire and Foster technique. The data show 2,714 rural households nested in 59 administrative zones of Ethiopia. Based on several parameters (nutrition and health, education, living standards, rural livelihoods and resources, and risk), the investigation looks into the multidimensional poverty levels of Ethiopian rural households and how they differ across Ethiopian administrative zones.

Results

The results indicate that 47.8% of the rural households of Ethiopians were multidimensionally poor in several dimensions; nutrition and health, education, living standards, rural livelihoods and resources, and risk. The living standard dimension is most deprivation-prone for the rural, multidimensional poor households. In addition, multidimensional poverty is more prevalent in Somali and Afar region rural areas. The best linear unbiased prediction estimates of multidimensional poverty vary substantially across Ethiopia's administrative zones. Specifically, the top poorest performing administrative zones concerning the likelihood of being multidimensional poor among rural households were Shebelle, Zone 2, Zone 3, Zone 4, and Konso special woreda.

Conclusion

The results of the generalized linear mixed-effects model show that crop-commercialized households have reduced the odds of being multidimensionally poorer than those who did not. This study recommends policymakers focus on rural mumyltidimensional poverty reduction strategies.

Cardinality matching versus propensity score matching for addressing cluster-level residual confounding in implantable medical device and surgical epidemiology: a parametric and plasmode simulation study

BACKGROUND

Rapid innovation and new regulations lead to an increased need for post-marketing surveillance of implantable devices. However, complex multi-level confounding related not only to patient-level but also to surgeon or hospital covariates hampers observational studies of risks and benefits. We conducted parametric and plasmode simulations to compare the performance of cardinality matching (CM) vs propensity score matching (PSM) to reduce confounding bias in the presence of cluster-level confounding.

METHODS

Two Monte Carlo simulation studies were carried out: 1) Parametric simulations (1,000 iterations) with patients nested in clusters (ratio 10:1, 50:1, 100:1, 200:1, 500:1) and sample size n = 10,000 were conducted with patient and cluster level confounders; 2) Plasmode simulations generated from a cohort of 9981 patients admitted for pancreatectomy between 2015 to 2019 from a US hospital database. CM with 0.1 standardised mean different constraint threshold (SMD) for confounders and PSM were used to balance the confounders for within-cluster and cross-cluster matching. Treatment effects were then estimated using logistic regression as the outcome model on the obtained matched sample.

RESULTS

CM yielded higher sample retention but more bias than PSM for cross-cluster matching in most scenarios. For instance, with ratio of 100:1, sample retention and relative bias were 97.1% and 26.5% for CM, compared to 82.5% and 12.2% for PSM. The results for plasmode simulation were similar.

CONCLUSIONS

CM offered better sample retention but higher bias in most scenarios compared to PSM. More research is needed to guide the use of CM particularly in constraint setting for confounders for medical device and surgical epidemiology.

The Association of Pregnancy With Outcomes Among Critically Ill Reproductive-Aged Women: A Propensity Score-Matched Retrospective Cohort Analysis

BACKGROUND

The maternal mortality rate in the United States is unacceptably high. However, the relative contribution of pregnancy to these outcomes is unknown. Studies comparing outcomes among pregnant vs nonpregnant critically ill patients show mixed results and are limited by small sample sizes.

RESEARCH QUESTION

What is the association of pregnancy with critical illness outcomes?

STUDY DESIGN AND METHODS

We performed a retrospective cohort study of women 18 to 55 years of age who received invasive mechanical ventilation (MV) on hospital day 0 or 1 or who demonstrated sepsis on admission (infection with organ failure) discharged from Premier Healthcare Database hospitals from 2008 through 2021. The exposure was pregnancy. The primary outcome was in-hospital mortality. We created propensity scores for pregnancy (using patient and hospital characteristics) and performed 1:1 propensity score matching without replacement within age strata (to ensure exact age matching). We performed multilevel multivariable mixed-effects logistic regression for propensity-matched pairs with pair as a random effect.

RESULTS

Three thousand ninety-three pairs were included in the matched MV cohort, and 13,002 pairs were included in the sepsis cohort. The characteristics of both cohorts were well balanced (all standard mean differences, < 0.1). Among matched pairs, unadjusted mortality was 8.0% vs 13.8% for MV and 1.4% vs 2.3% for sepsis among pregnant and nonpregnant patients, respectively. In adjusted regression, pregnancy was associated with lower odds of in-hospital mortality (MV: OR, 0.50; 95% CI, 0.41-0.60; P < .001; sepsis: OR, 0.52; 95% CI, 0.40-0.67; P < .001).

INTERPRETATION

In this large US cohort, critically ill pregnant women receiving MV or with sepsis showed better survival than propensity score-matched nonpregnant women. These findings must be interpreted in the context of likely residual confounding.

Protocol for a Prospective Observational Study of Revision Palatoplasty Versus Pharyngoplasty for Treatment of Velopharyngeal Insufficiency Following Cleft Palate Repair

OBJECTIVE

To present the design and methodology for an actively enrolling comparative effectiveness study of revision palatoplasty versus pharyngoplasty for the treatment of velopharyngeal insufficiency (VPI).

DESIGN

Prospective observational multicenter study.

SETTING

Twelve hospitals across the United States and Canada.

PARTICIPANTS

Individuals who are 3-23 years of age with a history of repaired cleft palate and a diagnosis of VPI, with a total enrollment target of 528 participants.

INTERVENTIONS

Revision palatoplasty and pharyngoplasty (either pharyngeal flap or sphincter pharyngoplasty), as selected for each participant by their treatment team.

MAIN OUTCOME MEASURE(S)

The primary outcome is resolution of hypernasality, defined as the absence of consistent hypernasality as determined by blinded perceptual assessment of a standard speech sample recorded twelve months after surgery. The secondary outcome is incidence of new onset obstructive sleep apnea. Statistical analyses will use propensity score matching to control for demographics, medical history, preoperative severity of hypernasality, and preoperative imaging findings.

RESULTS

Study recruitment began February 2021. As of September 2022, 148 participants are enrolled, and 78 have undergone VPI surgery. Enrollment is projected to continue into 2025. Collection of postoperative evaluations should be completed by the end of 2026, with dissemination of results soon thereafter.

CONCLUSIONS

Patients with VPI following cleft palate repair are being actively enrolled at sites across the US and Canada into a prospective observational study evaluating surgical outcomes. This study will be the largest and most comprehensive study of VPI surgery outcomes to date.

Propensity Score Weighting with Missing Data on Covariates and Clustered Data Structure

Propensity score (PS) analyses are increasingly popular in behavioral sciences. Two issues often add complexities to PS analyses, including missing data in observed covariates and clustered data structure. In previous research, methods for conducting PS analyses with considering either issue alone were examined. In practice, the two issues often co-occur; but the performance of methods for PS analyses in the presence of both issues has not been evaluated previously. In this study, we consider PS weighting analysis when data are clustered and observed covariates have missing values. A simulation study is conducted to evaluate the performance of different missing data handling methods (complete-case, single-level imputation, or multilevel imputation) combined with different multilevel PS weighting methods (fixed- or random-effects PS models, inverse-propensity-weighting or the clustered weighting, weighted single-level or multilevel outcome models). The results suggest that the bias in average treatment effect estimation can be reduced, by better accounting for clustering in both the missing data handling stage (such as with the multilevel imputation) and the PS analysis stage (such as with the fixed-effects PS model, clustered weighting, and weighted multilevel outcome model). A real-data example is provided for illustration.

Parametric and nonparametric propensity score estimation in multilevel observational studies

There has been growing interest in using nonparametric machine learning approaches for propensity score estimation in order to foster robustness against misspecification of the propensity score model. However, the vast majority of studies focused on single-level data settings, and research on nonparametric propensity score estimation in clustered data settings is scarce. In this article, we extend existing research by describing a general algorithm for incorporating random effects into a machine learning model, which we implemented for generalized boosted modeling (GBM). In a simulation study, we investigated the performance of logistic regression, GBM, and Bayesian additive regression trees for inverse probability of treatment weighting (IPW) when the data are clustered, the treatment exposure mechanism is nonlinear, and unmeasured cluster-level confounding is present. For each approach, we compared fixed and random effects propensity score models to single-level models and evaluated their use in both marginal and clustered IPW. We additionally investigated the performance of the standard Super Learner and the balance Super Learner. The results showed that when there was no unmeasured confounding, logistic regression resulted in moderate bias in both marginal and clustered IPW, whereas the nonparametric approaches were unbiased. In presence of cluster-level confounding, fixed and random effects models greatly reduced bias compared to single-level models in marginal IPW, with fixed effects GBM and fixed effects logistic regression performing best. Finally, clustered IPW was overall preferable to marginal IPW and the balance Super Learner outperformed the standard Super Learner, though neither worked as well as their best candidate model.

Tuning Random Forests for Causal Inference under Cluster-Level Unmeasured Confounding

Recently, there has been growing interest in using machine learning methods for causal inference due to their automatic and flexible ability to model the propensity score and the outcome model. However, almost all the machine learning methods for causal inference have been studied under the assumption of no unmeasured confounding and there is little work on handling omitted/unmeasured variable bias. This paper focuses on a machine learning method based on random forests known as Causal Forests and presents five simple modifications for tuning Causal Forests so that they are robust to cluster-level unmeasured confounding. Our simulation study finds that adjusting the default tuning procedure with the propensity score from fixed effects logistic regression or using variables that are centered to their cluster means produces estimates that are more robust to cluster-level unmeasured confounding. Also, when these parametric propensity score models are mis-specified, our modified machine learning methods remain robust to bias from cluster-level unmeasured confounders compared to existing parametric approaches based on propensity score weighting. We conclude by demonstrating our proposals in a real data study concerning the effect of taking an eighth-grade algebra course on math achievement scores from the Early Childhood Longitudinal Study.

The impact of Community Health Centers on inappropriate use of emergency services

Community Health Centers offer coordinated and comprehensive responses to primary care needs. Our study aims at assessing whether the introduction of such organizational model improved health outcomes measured by inappropriate emergency visits among diabetics in the Emilia-Romagna region of Italy. Using difference-in-differences methods within a staggered treatment setting, we estimate the effect of Community Health Center participation on inappropriate hospital emergency visits between year 2010 and year 2016. We distinguish between emergency department admissions for varying time spans, occurring at daytime during working days, at night-time, as well as during weekends. We show that, the causal effect of the adoption of the community care model leads to a reduction in the probability of inappropriate admissions by an amount ranging between 1.6 and 1.7% points during working days at daytime, with large facilities responsible for most gains by experiencing a decrease ranging between 4 and 3% points. Conversely, we detect no difference at night-time and during weekends. Our results point out that the coordinated care model increases appropriateness among vulnerable patients, and that extending opening hours and the range of services can further enhance such benefits.

Random effects modelling versus logistic regression for the inclusion of cluster-level covariates in propensity score estimation: A Monte Carlo simulation and registry cohort analysis

Purpose: Surgeon and hospital-related features, such as volume, can be associated with treatment choices and outcomes. Accounting for these covariates with propensity score (PS) analysis can be challenging due to the clustered nature of the data. We studied six different PS estimation strategies for clustered data using random effects modelling (REM) compared with logistic regression. Methods: Monte Carlo simulations were used to generate variable cluster-level confounding intensity [odds ratio (OR) = 1.01-2.5] and cluster size (20-1,000 patients per cluster). The following PS estimation strategies were compared: i) logistic regression omitting cluster-level confounders; ii) logistic regression including cluster-level confounders; iii) the same as ii) but including cross-level interactions; iv), v), and vi), similar to i), ii), and iii), respectively, but using REM instead of logistic regression. The same strategies were tested in a trial emulation of partial versus total knee replacement (TKR) surgery, where observational versus trial-based estimates were compared as a proxy for bias. Performance metrics included bias and mean square error (MSE). Results: In most simulated scenarios, logistic regression, including cluster-level confounders, led to the lowest bias and MSE, for example, with 50 clusters × 200 individuals and confounding intensity OR = 1.5, a relative bias of 10%, and MSE of 0.003 for (i) compared to 32% and 0.010 for (iv). The results from the trial emulation also gave similar trends. Conclusion: Logistic regression, including patient and surgeon-/hospital-level confounders, appears to be the preferred strategy for PS estimation.

A flexible approach for causal inference with multiple treatments and clustered survival outcomes

When drawing causal inferences about the effects of multiple treatments on clustered survival outcomes using observational data, we need to address implications of the multilevel data structure, multiple treatments, censoring, and unmeasured confounding for causal analyses. Few off-the-shelf causal inference tools are available to simultaneously tackle these issues. We develop a flexible random-intercept accelerated failure time model, in which we use Bayesian additive regression trees to capture arbitrarily complex relationships between censored survival times and pre-treatment covariates and use the random intercepts to capture cluster-specific main effects. We develop an efficient Markov chain Monte Carlo algorithm to draw posterior inferences about the population survival effects of multiple treatments and examine the variability in cluster-level effects. We further propose an interpretable sensitivity analysis approach to evaluate the sensitivity of drawn causal inferences about treatment effect to the potential magnitude of departure from the causal assumption of no unmeasured confounding. Expansive simulations empirically validate and demonstrate good practical operating characteristics of our proposed methods. Applying the proposed methods to a dataset on older high-risk localized prostate cancer patients drawn from the National Cancer Database, we evaluate the comparative effects of three treatment approaches on patient survival, and assess the ramifications of potential unmeasured confounding. The methods developed in this work are readily available in theR $$ \mathsf{R}\kern.15em $$ packageriAFTBART $$ \mathsf{riAFTBART} $$ .

Flexible propensity score estimation strategies for clustered data in observational studies

Existing studies have suggested superior performance of nonparametric machine learning over logistic regression for propensity score estimation. However, it is unclear whether the advantages of nonparametric propensity score modeling are carried to settings where there is clustering of individuals, especially when there is unmeasured cluster-level confounding. In this work we examined the performance of logistic regression (all main effects), Bayesian additive regression trees and generalized boosted modeling for propensity score weighting in clustered settings, with the clustering being accounted for by including either cluster indicators or random intercepts. We simulated data for three hypothetical observational studies of varying sample and cluster sizes. Confounders were generated at both levels, including a cluster-level confounder that is unobserved in the analyses. A binary treatment and a continuous outcome were generated based on seven scenarios with varying relationships between the treatment and confounders (linear and additive, nonlinear/nonadditive, nonadditive with the unobserved cluster-level confounder). Results suggest that when the sample and cluster sizes are large, nonparametric propensity score estimation may provide better covariate balance, bias reduction, and 95% confidence interval coverage, regardless of the degree of nonlinearity or nonadditivity in the true propensity score model. When the sample or cluster sizes are small, however, nonparametric approaches may become more vulnerable to unmeasured cluster-level confounding and thus may not be a better alternative to multilevel logistic regression. We applied the methods to the National Longitudinal Study of Adolescent to Adult Health data, estimating the effect of team sports participation during adolescence on adulthood depressive symptoms.

Causal Inference with Multilevel Data: A Comparison of Different Propensity Score Weighting Approaches

Propensity score methods are a widely recommended approach to adjust for confounding and to recover treatment effects with non-experimental, single-level data. This article reviews propensity score weighting estimators for multilevel data in which individuals (level 1) are nested in clusters (level 2) and nonrandomly assigned to either a treatment or control condition at level 1. We address the choice of a weighting strategy (inverse probability weights, trimming, overlap weights, calibration weights) and discuss key issues related to the specification of the propensity score model (fixed-effects model, multilevel random-effects model) in the context of multilevel data. In three simulation studies, we show that estimates based on calibration weights, which prioritize balancing the sample distribution of level-1 and (unmeasured) level-2 covariates, should be preferred under many scenarios (i.e., treatment effect heterogeneity, presence of strong level-2 confounding) and can accommodate covariate-by-cluster interactions. However, when level-1 covariate effects vary strongly across clusters (i.e., under random slopes), and this variation is present in both the treatment and outcome data-generating mechanisms, large cluster sizes are needed to obtain accurate estimates of the treatment effect. We also discuss the implementation of survey weights and present a real-data example that illustrates the different methods.

Propensity score methods for observational studies with clustered data: A review

Propensity score methods are a popular approach to mitigating confounding bias when estimating causal effects in observational studies. When study units are clustered (eg, patients nested within health systems), additional challenges arise such as accounting for unmeasured confounding at multiple levels and dependence between units within the same cluster. While clustered observational data are widely used to draw causal inferences in many fields, including medicine and healthcare, extensions of propensity score methods to clustered settings are still a relatively new area of research. This article presents a framework for estimating causal effects using propensity scores when study units are nested within clusters and are nonrandomly assigned to treatment conditions. We emphasize the need for investigators to examine the nature of the clustering, among other properties, of the observational data at hand in order to guide their choice of causal estimands and the corresponding propensity score approach.

Bridging the gap between multilevel modeling and economic methods

Many datasets used in the social sciences have a hierarchical structure, where lower units of aggregation are 'nested' in higher units. In many disciplines, such data are analyzed using multilevel modeling (MLM, also known as hierarchical linear modeling). However, MLM as a framework is relatively unknown in economics. Instead, economists use a range of separate econometric methods, including cluster-robust standard errors, fixed effects models, models with cross-level interactions, and estimated dependent variable models. Relying on an extensive literature review, this paper describes this methodological divide and provides a detailed comparison between MLM and 'economic methods' in their abilities to deal with three methodological challenges inherent in multilevel data ‒ clustering, omitted variables, and coefficients' heterogeneity across groups. We unfold the comparative advantages of these two methodological approaches and provide practical recommendations about which of them should be used, why, and in what settings.

Robust Machine Learning for Treatment Effects in Multilevel Observational Studies Under Cluster-level Unmeasured Confounding

Recently, machine learning (ML) methods have been used in causal inference to estimate treatment effects in order to reduce concerns for model mis-specification. However, many ML methods require that all confounders are measured to consistently estimate treatment effects. In this paper, we propose a family of ML methods that estimate treatment effects in the presence of cluster-level unmeasured confounders, a type of unmeasured confounders that are shared within each cluster and are common in multilevel observational studies. We show through simulation studies that our proposed methods are robust from biases from unmeasured cluster-level confounders in a variety of multilevel observational studies. We also examine the effect of taking an algebra course on math achievement scores from the Early Childhood Longitudinal Study, a multilevel observational educational study, using our methods. The proposed methods are available in the CURobustML R package.

A doubly robust method to handle missing multilevel outcome data with application to the China Health and Nutrition Survey

Missing data are common in longitudinal cohort studies and can lead to bias, particularly in studies with informative missingness. Many common methods for handling informatively missing data in survey samples require correctly specifying a model for missingness. Although doubly robust methods exist to provide unbiased regression coefficients in the presence of missing outcome data, these methods do not account for correlation due to clustering inherent in longitudinal or cluster-sampled studies. In this work, we developed a doubly robust method to estimate the regression of an outcome on a predictor in the presence of missing multilevel data on the outcome, which results in consistent estimation of regression coefficients assuming correct specification of either (1) the probability of missingness or (2) the outcome model. This method involves specification of separate hierarchical models for missingness and for the outcome, conditional on observed auxiliary variables and cluster-specific random effects, to account for correlation among observations. We showed this proposed estimator is doubly robust and derived its asymptotic distribution, conducted simulation studies to compare the method to an existing doubly robust method developed for independent data, and applied the method to data from the China Health and Nutrition Survey, an ongoing multilevel longitudinal cohort study.

Medium Cut-Off Versus High-Flux Hemodialysis Membranes and Clinical Outcomes: A Cohort Study Using Inverse Probability Treatment Weighting

Rationale & Objective

This study investigated the effects on patients’ outcomes of using medium cutoff (MCO) versus high-flux (HF) dialysis membranes.

Study Design

A retrospective, observational, multicenter, cohort study.

Setting & Participants

Patients aged greater than 18 years receiving hemodialysis at the Baxter Renal Care Services dialysis network in Colombia. The inception of the cohort occurred from September 1, 2017, to November 30, 2017, with follow-up to November 30, 2019.

Exposure

The patients were divided into 2 cohorts according to the dialyzer used at the inception: (1) MCO membrane or (2) HF membrane.

Outcomes

Primary outcomes were the hospitalization rate from any cause and hospitalization days per patient-year. Secondary outcomes were acute cardiovascular events and mortality rates from any cause and secondary to cardiovascular causes. Laboratory parameters were assessed throughout the 2-year follow-up period.

Analytical Approach

Descriptive statistics were used to report population characteristics. Inverse probability of treatment weighting was applied to each group before analysis. All categorical variables were compared using Pearson’s χ² test, and continuous variables were analyzed with the t test. Baseline differences between groups with a value of >10% were considered clinically meaningful. Laboratory variables were measured at 5 consecutive time points. A between-patient effect was analyzed using a split-plot factorial analysis of variance.

Results

The analysis included 1,098 patients, of whom 564 (51.3%) were dialyzed with MCO membranes and 534 (48.7%) with HF membranes. Patients receiving hemodialysis with MCO membranes had a lower all-cause hospitalization incidence rate (IR) per patient-year (IR = 0.93; 95% CI, 0.82-1.03) than those receiving hemodialysis with HF membranes (IR = 1.13; 95% CI, 0.96-1.30), corresponding to a significant incident rate ratio (MCO/HF) of 0.82 (95% CI, 0.68-0.99; P = 0.04). The frequency of nonfatal cardiovascular events showed statistical significance, with a lower incidence in the MCO group (incident rate ratio = 0.66; 95% CI, 0.46-0.96; P = 0.03). No statistically significant differences in all-cause time until death were observed (P = 0.48). Albumin levels were similar between the 2 dialyzer cohorts.

Limitations

Despite the robust statistical analysis, there remains the possibility that unmeasured variables may still generate residual imbalance and, therefore, skew the results.

Conclusions

The incidences of hospitalization and cardiovascular events in patients receiving hemodialysis were lower when dialyzed with MCO membranes than HF membranes. A randomized controlled trial would be desirable to confirm these results.

Trial Registration

Clinical Trials.gov, ISRCTN12403265.

Evaluating the performance of propensity score matching based approaches in individual patient data meta-analysis

Background

Individual-patient data meta-analysis (IPD-MA) is an increasingly popular approach because of its analytical benefits. IPD-MA of observational studies must overcome the problem of confounding, otherwise biased estimates of treatment effect may be obtained. One approach to reducing confounding bias could be the use of propensity score matching (PSM). IPD-MA can be considered as two-stage clustered data (patients within studies) and propensity score matching can be implemented within studies, across studies, and combining both.

Methods

This article focuses on implementation of four PSM-based approaches for the analysis of data structure that exploit IPD-MA in two ways: (i) estimation of propensity score model using single-level or random-effects logistic regression; and (ii) matching of propensity scores (PS) across studies, within studies or preferential-within studies. We investigated the performance of these approaches through a simulation study, which considers an IPD-MA that examined the success of different treatments for multidrug-resistant tuberculosis (MDR-TB). The simulation parameters were varied according to three treatment prevalences (according to studies, 50% and 30%), three levels of heterogeneity between studies (low, moderate and high) and three levels of pooled odds ratio (1, 1.5, 3).

Results

All approaches showed greater biases at the higher levels of heterogeneity regardless of the choices of treatment prevalences. However, matching of propensity scores using within-study and preferential-within study reported better performance compared to matching across studies when treatment prevalence varied across-studies. For fixed prevalences, a random-effect propensity score model to estimate propensity scores followed by matching of propensity scores across-studies achieved lower biases compared to other PSM-based approaches.

Conclusions

Propensity score matching has wide application in health research while only limited literature is available on the implementation of PSM methods in IPD-MA, and until now methodological performance of PSM methods have not been examined. We believe, this work offers an intuition to the applied researcher for the choice of the PSM-based approaches.

Random Forests Approach for Causal Inference with Clustered Observational Data

There is a growing interest in using machine learning (ML) methods for causal inference due to their (nearly) automatic and flexible ability to model key quantities such as the propensity score or the outcome model. Unfortunately, most ML methods for causal inference have been studied under single-level settings where all individuals are independent of each other and there is little work in using these methods with clustered or nested data, a common setting in education studies. This paper investigates using one particular ML method based on random forests known as Causal Forests to estimate treatment effects in multilevel observational data. We conduct simulation studies under different types of multilevel data, including two-level, three-level, and cross-classified data. Our simulation study shows that when the ML method is supplemented with estimated propensity scores from multilevel models that account for clustered/hierarchical structure, the modified ML method outperforms preexisting methods in a wide variety of settings. We conclude by estimating the effect of private math lessons in the Trends in International Mathematics and Science Study data, a large-scale educational assessment where students are nested within schools.

Effects of Compliance with the Early Management Bundle (SEP-1) on Mortality Changes among Medicare Beneficiaries with Sepsis: A Propensity Score Matched Cohort Study

BACKGROUND

U.S. hospitals have reported compliance with the SEP-1 quality measure to Medicare since 2015. Finding an association between compliance and outcomes is essential to gauge measure effectiveness.

RESEARCH QUESTION

What is the association between compliance with SEP-1 and 30-day mortality among Medicare beneficiaries?

STUDY DESIGN AND METHODS

Studying patient-level data reported to Medicare by 3,241 hospitals from October 1, 2015 to March 31, 2017, we used propensity score matching and a hierarchical general linear model (HGLM) to estimate the treatment effects associated with compliance with SEP-1. Compliance was defined as completion of all qualifying SEP-1 elements including lactate measurements, blood culture collection, broad-spectrum antibiotic administration, 30 ml/kg crystalloid fluid administration, application of vasopressors, and patient reassessment. The primary outcome was a change in 30-day mortality. Secondary outcomes included changes in length-of-stay.

RESULTS

We completed two matches to evaluate population-level treatment effects. In "Standard-match" 122,870 patients whose care was compliant were matched with the same number whose care was non-compliant. Compliance was associated with a reduction in 30-day mortality: 21.81% versus 27.48% yielding an ARR of 5.67% (95% confidence interval [CI]: 5.33-6.00; P < 0.001). In "Stringent-match" 107,016 patients whose care was compliant were matched with the same number whose care was non-compliant. Compliance was associated with a reduction in 30-day mortality: 22.22% versus 26.28% yielding an ARR of 4.06% (95% CI: 3.70-4.41; P < 0.001). At the subject-level, our HGLM model found compliance associated with lower 30-day risk-adjusted mortality (adjusted conditional odds ratio = 0.829; 95% CI: 0.812-0.846; P < 0001). Multiple elements correlated with lower mortality. Median length-of-stay was shorter among cases whose care was compliant (5 vs. 6 days; IQR: 3-9 vs. 4-10; P < 0.001).

INTERPRETATION

Compliance with SEP-1 was associated with lower 30-day mortality. Rendering SEP-1 compliant care may reduce the incidence of avoidable deaths.

Intra-operative autologous blood donation for cardiovascular surgeries in Japan: A retrospective cohort study

Intra-operative autologous blood donation is a blood conservation technique with limited evidence. We evaluated the association between intra-operative autologous blood donation and decrease in peri-operative transfusion in cardiovascular surgery based on evidence from a Japanese administrative database. We extracted the data of patients who had undergone cardiovascular surgery from the Diagnosis Procedure Combination database in Japan (2016–2019). Based on the surgery type, we examined the association of intra-operative autologous blood donation with the transfusion rate and amount of blood used in cardiac and aortic surgeries using multilevel propensity score matching. We enrolled 32,433 and 4,267 patients who underwent cardiac and aortic surgeries and received 5.0% and 6.7% intra-operative autologous blood donation with mean volumes of 557.68 mL and 616.96 mL, respectively. The red blood cell transfusion rates of the control and intra-operative autologous blood donation groups were 60.6% and 38.4%, respectively, in the cardiac surgery cohort (p < .001) and 91.4%, and 83.8%, respectively, in the aortic surgery cohort (p = .037). The transfusion amounts for the control and intra-operative autologous blood donation groups were 5.9 and 3.5 units of red blood cells, respectively, for cardiac surgery patients (p < .001) and 11.9 and 7.9 units, respectively, for aortic surgery patients (p < .001). Intra-operative autologous blood donation could reduce the transfusion rate or amount of red blood cells and fresh frozen plasma for patients undergoing index cardiovascular surgery and could be an effective blood transfusion strategy in cardiovascular surgery for Japanese patients.

Neural Networks to Estimate Generalized Propensity Scores for Continuous Treatment Doses

BACKGROUND

The generalized propensity score (GPS) addresses selection bias due to observed confounding variables and provides a means to demonstrate causality of continuous treatment doses with propensity score analyses. Estimating the GPS with parametric models obliges researchers to meet improbable conditions such as correct model specification, normal distribution of variables, and large sample sizes.

OBJECTIVES

The purpose of this Monte Carlo simulation study is to examine the performance of neural networks as compared to full factorial regression models to estimate GPS in the presence of Gaussian and skewed treatment doses and small to moderate sample sizes.

RESEARCH DESIGN

A detailed conceptual introduction of neural networks is provided, as well as an illustration of selection of hyperparameters to estimate GPS. An example from public health and nutrition literature uses residential distance as a treatment variable to illustrate how neural networks can be used in a propensity score analysis to estimate a dose-response function of grocery spending behaviors.

RESULTS

We found substantially higher correlations and lower mean squared error values after comparing true GPS with the scores estimated by neural networks. The implication is that more selection bias was removed using GPS estimated with neural networks than using GPS estimated with classical regression.

CONCLUSIONS

This study proposes a new methodological procedure, neural networks, to estimate GPS. Neural networks are not sensitive to the assumptions of linear regression and other parametric models and have been shown to be a contender against parametric approaches to estimate propensity scores for continuous treatments.

Propensity score matching for multilevel spatial data: accounting for geographic confounding in health disparity studies

BACKGROUND

Diabetes is a public health burden that disproportionately affects military veterans and racial minorities. Studies of racial disparities are inherently observational, and thus may require the use of methods such as Propensity Score Analysis (PSA). While traditional PSA accounts for patient-level factors, this may not be sufficient when patients are clustered at the geographic level and thus important confounders, whether observed or unobserved, vary by geographic location.

METHODS

We employ a spatial propensity score matching method to account for "geographic confounding", which occurs when the confounding factors, whether observed or unobserved, vary by geographic region. We augment the propensity score and outcome models with spatial random effects, which are assigned scaled Besag-York-Mollié priors to address spatial clustering and improve inferences by borrowing information across neighboring geographic regions. We apply this approach to a study exploring racial disparities in diabetes specialty care between non-Hispanic black and non-Hispanic white veterans. We construct multiple global estimates of the risk difference in diabetes care: a crude unadjusted estimate, an estimate based solely on patient-level matching, and an estimate that incorporates both patient and spatial information.

RESULTS

In simulation we show that in the presence of an unmeasured geographic confounder, ignoring spatial heterogeneity results in increased relative bias and mean squared error, whereas incorporating spatial random effects improves inferences. In our study of racial disparities in diabetes specialty care, the crude unadjusted estimate suggests that specialty care is more prevalent among non-Hispanic blacks, while patient-level matching indicates that it is less prevalent. Hierarchical spatial matching supports the latter conclusion, with a further increase in the magnitude of the disparity.

CONCLUSIONS

These results highlight the importance of accounting for spatial heterogeneity in propensity score analysis, and suggest the need for clinical care and management strategies that are culturally sensitive and racially inclusive.

Spillover HIV prevention effects of a cash transfer trial in East Zimbabwe: evidence from a cluster-randomised trial and general-population survey

BACKGROUND

Benefits of cash transfers (CTs) for HIV prevention have been demonstrated largely in purposively designed trials, commonly focusing on young women. It is less clear if CT interventions not designed for HIV prevention can have HIV-specific effects, including adverse effects. The cluster-randomised Manicaland Cash Transfer Trial (2010-11) evaluated effects of CTs on children's (2-17 years) development in eastern Zimbabwe. We evaluated whether this CT intervention with no HIV-specific objectives had unintended HIV prevention spillover effects (externalities).

METHODS

Data on 2909 individuals (15-54 years) living in trial households were taken from a general-population survey, conducted simultaneously in the same communities as the Manicaland Trial. Average treatment effects (ATEs) of CTs on sexual behaviour (any recent sex, condom use, multiple partners) and secondary outcomes (mental distress, school enrolment, and alcohol/cigarette/drug consumption) were estimated using mixed-effects logistic regressions (random effects for study site and intervention cluster), by sex and age group (15-29; 30-54 years). Outcomes were also evaluated with a larger synthetic comparison group created through propensity score matching.

RESULTS

CTs did not affect sexual debut but reduced having any recent sex (past 30 days) among young males (ATE: - 11.7 percentage points [PP] [95% confidence interval: -26.0PP, 2.61PP]) and females (- 5.68PP [- 15.7PP, 4.34PP]), with similar but less uncertain estimates when compared against the synthetic comparison group (males: -9.68PP [- 13.1PP, - 6.30PP]; females: -8.77PP [- 16.3PP, - 1.23PP]). There were no effects among older individuals. Young (but not older) males receiving CTs reported increased multiple partnerships (8.49PP [- 5.40PP, 22.4PP]; synthetic comparison: 10.3PP (1.27PP, 19.2PP). No impact on alcohol, cigarette, or drug consumption was found. There are indications that CTs reduced psychological distress among young people, although impacts were small. CTs increased school enrolment in males (11.5PP [3.05PP, 19.9PP]). Analyses with the synthetic comparison group (but not the original control group) further indicated increased school enrolment among females (5.50PP [1.62PP, 9.37PP]) and condom use among younger and older women receiving CTs (9.38PP [5.90PP, 12.9PP]; 5.95PP [1.46PP, 10.4PP]).

CONCLUSIONS

Non-HIV-prevention CT interventions can have HIV prevention outcomes, including reduced sexual activity among young people and increased multiple partnerships among young men. No effects on sexual debut or alcohol, cigarette, or drug consumption were observed. A broad approach is necessary to evaluate CT interventions to capture unintended outcomes, particularly in economic evaluations.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT00966849 . Registered August 27, 2009.

Propensity score methods for time-dependent cluster confounding

In observational studies, subjects are often nested within clusters. In medical studies, patients are often treated by doctors and therefore patients are regarded as nested or clustered within doctors. A concern that arises with clustered data is that cluster-level characteristics (e.g., characteristics of the doctor) are associated with both treatment selection and patient outcomes, resulting in cluster-level confounding. Measuring and modeling cluster attributes can be difficult and statistical methods exist to control for all unmeasured cluster characteristics. An assumption of these methods however is that characteristics of the cluster and the effects of those characteristics on the outcome (as well as probability of treatment assignment when using covariate balancing methods) are constant over time. In this paper, we consider methods that relax this assumption and allow for estimation of treatment effects in the presence of unmeasured time-dependent cluster confounding. The methods are based on matching with the propensity score and incorporate unmeasured time-specific cluster effects by performing matching within clusters or using fixed- or random-cluster effects in the propensity score model. The methods are illustrated using data to compare the effectiveness of two total hip devices with respect to survival of the device and a simulation study is performed that compares the proposed methods. One method that was found to perform well is matching within surgeon clusters partitioned by time. Considerations in implementing the proposed methods are discussed.

Propensity Score Methods in Health Technology Assessment: Principles, Extended Applications, and Recent Advances

Randomized clinical trials (RCT) are accepted as the gold-standard approaches to measure effects of intervention or treatment on outcomes. They are also the designs of choice for health technology assessment (HTA). Randomization ensures comparability, in both measured and unmeasured pretreatment characteristics, of individuals assigned to treatment and control or comparator. However, even adequately powered RCTs are not always feasible for several reasons such as cost, time, practical and ethical constraints, and limited generalizability. RCTs rely on data collected on selected, homogeneous population under highly controlled conditions; hence, they provide evidence on efficacy of interventions rather than on effectiveness. Alternatively, observational studies can provide evidence on the relative effectiveness or safety of a health technology compared to one or more alternatives when provided under the setting of routine health care practice. In observational studies, however, treatment assignment is a non-random process based on an individual’s baseline characteristics; hence, treatment groups may not be comparable in their pretreatment characteristics. As a result, direct comparison of outcomes between treatment groups might lead to biased estimate of the treatment effect. Propensity score approaches have been used to achieve balance or comparability of treatment groups in terms of their measured pretreatment covariates thereby controlling for confounding bias in estimating treatment effects. Despite the popularity of propensity scores methods and recent important methodological advances, misunderstandings on their applications and limitations are all too common. In this article, we present a review of the propensity scores methods, extended applications, recent advances, and their strengths and limitations.

Causal inference with interfering units for cluster and population level treatment allocation programs

Interference arises when an individual's potential outcome depends on the individual treatment level, but also on the treatment level of others. A common assumption in the causal inference literature in the presence of interference is partial interference, implying that the population can be partitioned in clusters of individuals whose potential outcomes only depend on the treatment of units within the same cluster. Previous literature has defined average potential outcomes under counterfactual scenarios where treatments are randomly allocated to units within a cluster. However, within clusters there may be units that are more or less likely to receive treatment based on covariates or neighbors' treatment. We define new estimands that describe average potential outcomes for realistic counterfactual treatment allocation programs, extending existing estimands to take into consideration the units' covariates and dependence between units' treatment assignment. We further propose entirely new estimands for population-level interventions over the collection of clusters, which correspond in the motivating setting to regulations at the federal (vs. cluster or regional) level. We discuss these estimands, propose unbiased estimators and derive asymptotic results as the number of clusters grows. For a small number of observed clusters, a bootstrap approach for confidence intervals is proposed. Finally, we estimate effects in a comparative effectiveness study of power plant emission reduction technologies on ambient ozone pollution.

Inverse probability weighted Cox model in multi-site studies without sharing individual-level data

The inverse probability weighted Cox proportional hazards model can be used to estimate the marginal hazard ratio. In multi-site studies, it may be infeasible to pool individual-level datasets due to privacy and other considerations. We propose three methods for making inference on hazard ratios without the need for pooling individual-level datasets across sites. The first method requires a summary-level eight-column risk-set table to produce the same hazard ratio estimate and robust sandwich variance estimate as those from the corresponding pooled individual-level data analysis (reference analysis). The second and third methods, which are based on two bootstrap re-sampling strategies, require a summary-level four-column risk-set table and bootstrap-based risk-set tables from each site to produce the same hazard ratio and bootstrap variance estimates as those from their reference analyses. All three methods require only one file transfer between the data-contributing sites and the analysis center. We justify these methods theoretically, illustrate their use, and demonstrate their statistical performance using both simulated and real-world data.

Analysis of racial differences in hospital stays in the presence of geographic confounding

Using recent methods for spatial propensity score modeling, we examine differences in hospital stays between non-Hispanic black and non-Hispanic white veterans with type 2 diabetes. We augment a traditional patient-level propensity score model with a spatial random effect to create a matched sample based on the estimated propensity score. We then use a spatial negative binomial hurdle model to estimate differences in both hospital admissions and inpatient days. We demonstrate that in the presence of unmeasured geographic confounding, spatial propensity score matching in addition to the spatial negative binomial hurdle outcome model yields improved performance compared to the outcome model alone. In the motivating application, we construct three estimates of racial differences in hospitalizations: the risk difference in admission, the mean difference in number of inpatient days among those hospitalized, and the mean difference in number of inpatient days across all patients (hospitalized and non-hospitalized). Results indicate that non-Hispanic black veterans with type 2 diabetes have a lower risk of hospital admission and a greater number of inpatient days on average. The latter result is especially important considering that we observed much smaller effect sizes in analyses that did not incorporate spatial matching. These results emphasize the need to address geographic confounding in health disparity studies.

Augmented Weighted Estimators Dealing with Practical Positivity Violation to Causal inferences in a Random Coefficient Model

The inverse probability of treatment weighted (IPTW) estimator can be used to make causal inferences under two assumptions: (1) no unobserved confounders (ignorability) and (2) positive probability of treatment and of control at every level of the confounders (positivity), but is vulnerable to bias if by chance, the proportion of the sample assigned to treatment, or proportion of control, is zero at certain levels of the confounders. We propose to deal with this sampling zero problem, also known as practical violation of the positivity assumption, in a setting where the observed confounder is cluster identity, i.e., treatment assignment is ignorable within clusters. Specifically, based on a random coefficient model assumed for the potential outcome, we augment the IPTW estimating function with the estimated potential outcomes of treatment (or of control) for clusters that have no observation of treatment (or control). If the cluster-specific potential outcomes are estimated correctly, the augmented estimating function can be shown to converge in expectation to zero and therefore yield consistent causal estimates. The proposed method can be implemented in the existing software, and it performs well in simulated data as well as with real-world data from a teacher preparation evaluation study.

Association of Overnight Extubation With Outcomes After Cardiac Surgery in the Intensive Care Unit

BACKGROUND

The frequency and safety of overnight extubation (OE) after cardiac surgery across intensive care units (ICUs) is unknown.

METHODS

We performed a retrospective study of adults (≥ 18 years) in US ICUs after coronary artery bypass grafting (CABG) or aortic valve replacement (AVR) or both, using The Society of Thoracic Surgery Adult Cardiac Surgery Database (July 2014 to June 2017); our primary cohort was elective CABGs. We assessed OE (7:00 pm to 6:59 am) frequency and used multilevel regression modelling to identify factors associated with OE. Within mechanical ventilation (MV) duration strata, we used propensity score matching to evaluate associations of OE with reintubations (primary outcome), mortality, and complications.

RESULTS

Among 142,225 patients with elective CABG, 42.2% had OEs. MV duration, cardiopulmonary bypass time, distal anastomosis number, and hospital of admission (median odds ratio [OR] 1.82, 95% confidence interval [CI]: 1.76 to 1.89) were independently associated with OE. After propensity matching, OE was associated with increased reintubation for patients with MV duration of 6 to 8 hours (2.2% vs 1.7%, OR 1.27, 95% CI: 1.04 to 1.56) and decreased reintubation for patients with MV duration of 15 to 17 hours (3.0% vs 4.2%, OR 0.70, 95% CI: 0.50 to 0.97) and 18 to 20 hours (2.3% vs 5.7%, OR 0.39, 95% CI: 0.21 to 0.72); OE was associated with increased ICU length of stay for patients with MV duration of 6 to 8 hours, but reduced length of stay for patients with MV duration of 9 to 20 hours. OE was not associated with increased mortality (hospital, 30-day). Other groups had similar OE rates (nonelective CABGs, 47.6%; elective AVR, 36.0%; elective CABG + AVRs, 51.0%) and outcomes.

CONCLUSIONS

OE is prevalent after cardiac surgery. OE is associated with little risk and reduces ICU length of stay for patients who require MV for more than 8 hours.

Effects of preoperative oral management by dentists on postoperative outcomes following esophagectomy: Multilevel propensity score matching and weighting analyses using the Japanese inpatient database

The purpose of this study was to investigate the effects of preoperative oral management (POM) by dentists on the incidence of postoperative pulmonary complications (PPCs), length of hospital stay, medical costs, and days of antibiotics administration following both open and thoracoscopic esophagectomy.Dental plaque is an established risk factor for postoperative pneumonia, which could be reduced by POM. However, few clinical guidelines for cancer treatment, including those for esophageal cancer, recommend POM as routine perioperative care.We extracted data of esophagectomy cases from the Japanese Diagnosis Procedure Combination database. We subsequently conducted propensity score (PS) analyses for multilevel data, including matching, inverse probability of treatment weighting (IPTW), and standardized mortality ratio weighting (SMRW), to estimate the effect of POM by dentists on the outcomes of esophagectomy.We analyzed 3412 esophagectomy cases of which 812 were open, and 2600 were thoracoscopic surgery. In IPTW analysis to estimate the average treatment effect, the risk difference of postoperative aspiration pneumonia ranged from -2.49% to -2.02% between the POM and control groups of both open and thoracoscopic esophagectomy cases. IPTW analyses indicated that the total medical costs of thoracoscopic esophagectomy were reduced by 221,200 to 253,100 Japanese Yen (equivalent to about $2000-$2200). In PS matching and SMRW analyses to estimate average treatment effect on treated, there was no difference in outcomes between the POM and control groups.Our results suggested that in patients undergoing open or thoracoscopic esophagectomy, POM by dentists prevented the occurrence of postoperative aspiration pneumonia. It could also reduce the total medical costs of thoracoscopic esophagectomy. Thus, POM by dentists can be considered as a routine perioperative care for all patients undergoing esophagectomy, regardless of the expected risk for PPC.

Aggression and violence in psychiatric hospitals with and without open door policies: A 15-year naturalistic observational study

Aggressive behavior and violence in psychiatric patients have often been quoted to justify more restrictive settings in psychiatric facilities. However, the effects of open vs. locked door policies on aggressive incidents remain unclear. This study had a naturalistic observational design and analyzed the occurrence of aggressive behavior as well as the use of seclusion or restraint in 21 German hospitals. The analysis included data from 1998 to 2012 and contained a total of n = 314,330 cases, either treated in one of 17 hospitals with (n = 68,135) or in one of 4 hospitals without an open door policy (n = 246,195). We also analyzed the data according to participants' stay on open, partially open, or locked wards. To compare hospital and ward types, we used generalized linear mixed-effects models on a propensity score matched subset (n = 126,268) and on the total dataset. The effect of open vs. locked door policy was non-significant in all analyses of aggressive behavior during treatment. Restraint or seclusion during treatment was less likely in hospitals with an open door policy. On open wards, any aggressive behavior and restraint or seclusion were less likely, whereas bodily harm was more likely than on closed wards. Hospitals with open door policies did not differ from hospitals with locked wards regarding different forms of aggression. Other restrictive interventions used to control aggression were significantly reduced in open settings. Open wards seem to have a positive effect on reducing aggression. Future research should focus on mental health care policies targeted at empowering treatment approaches, respecting the patient's autonomy and promoting reductions of institutional coercion.

Addressing geographic confounding through spatial propensity scores: a study of racial disparities in diabetes

Motivated by a study exploring differences in glycemic control between non-Hispanic black and non-Hispanic white veterans with type 2 diabetes, we aim to address a type of confounding that arises in spatially referenced observational studies. Specifically, we develop a spatial doubly robust propensity score estimator to reduce bias associated with geographic confounding, which occurs when measured or unmeasured confounding factors vary by geographic location, leading to imbalanced group comparisons. We augment the doubly robust estimator with spatial random effects, which are assigned conditionally autoregressive priors to improve inferences by borrowing information across neighboring geographic regions. Through a series of simulations, we show that ignoring spatial variation results in increased absolute bias and mean squared error, while the spatial doubly robust estimator performs well under various levels of spatial heterogeneity and moderate sample sizes. In the motivating application, we construct three global estimates of the risk difference between race groups: an unadjusted estimate, a doubly robust estimate that adjusts only for patient-level information, and a hierarchical spatial doubly robust estimate. Results indicate a gradual reduction in the risk difference at each stage, with the inclusion of spatial random effects providing a 20% reduction compared to an estimate that ignores spatial heterogeneity. Smoothed maps indicate poor glycemic control across Alabama and southern Georgia, areas comprising the so-called "stroke belt." These results suggest the need for community-specific interventions to target diabetes in geographic areas of greatest need.

Methodological approaches in analysing observational data: A practical example on how to address clustering and selection bias

BACKGROUND

Because not every scientific question on effectiveness can be answered with randomised controlled trials, research methods that minimise bias in observational studies are required. Two major concerns influence the internal validity of effect estimates: selection bias and clustering. Hence, to reduce the bias of the effect estimates, more sophisticated statistical methods are needed.

AIM

To introduce statistical approaches such as propensity score matching and mixed models into representative real-world analysis and to conduct the implementation in statistical software R to reproduce the results. Additionally, the implementation in R is presented to allow the results to be reproduced.

METHOD

We perform a two-level analytic strategy to address the problems of bias and clustering: (i) generalised models with different abilities to adjust for dependencies are used to analyse binary data and (ii) the genetic matching and covariate adjustment methods are used to adjust for selection bias. Hence, we analyse the data from two population samples, the sample produced by the matching method and the full sample.

RESULTS

The different analysis methods in this article present different results but still point in the same direction. In our example, the estimate of the probability of receiving a case conference is higher in the treatment group than in the control group. Both strategies, genetic matching and covariate adjustment, have their limitations but complement each other to provide the whole picture.

CONCLUSION

The statistical approaches were feasible for reducing bias but were nevertheless limited by the sample used. For each study and obtained sample, the pros and cons of the different methods have to be weighted.

Propensity score weighting for a continuous exposure with multilevel data

Propensity score methods (e.g., matching, weighting, subclassification) provide a statistical approach for balancing dissimilar exposure groups on baseline covariates. These methods were developed in the context of data with no hierarchical structure or clustering. Yet in many applications the data have a clustered structure that is of substantive importance, such as when individuals are nested within healthcare providers or within schools. Recent work has extended propensity score methods to a multilevel setting, primarily focusing on binary exposures. In this paper, we focus on propensity score weighting for a continuous, rather than binary, exposure in a multilevel setting. Using simulations, we compare several specifications of the propensity score: a random effects model, a fixed effects model, and a single-level model. Additionally, our simulations compare the performance of marginal versus cluster-mean stabilized propensity score weights. In our results, regression specifications that accounted for the multilevel structure reduced bias, particularly when cluster-level confounders were omitted. Furthermore, cluster mean weights outperformed marginal weights.

Impacts of nonstate, market-driven governance on Chilean forests

Global markets for agricultural products, timber, and minerals are critically important drivers of deforestation. The supply chains driving land use change may also provide opportunities to halt deforestation. Market campaigns, moratoria, and certification schemes have been promoted as powerful tools to achieve conservation goals. Despite their promise, there have been few opportunities to rigorously quantify the ability of these nonstate, market-driven (NSMD) governance regimes to deliver conservation outcomes. This study analyzes the impacts of three NSMD governance systems that sought to end the conversion of natural forests to plantations in Chile at the start of the 21st century. Using a multilevel, panel dataset of land use changes in Chile, we identify the impact of participation within each of the governance regimes by implementing a series of matched difference-in-differences analyses. Taking advantage of the mosaic of different NSMD regimes adopted in Chile, we explore the relative effectiveness of different policies. NSMD governance regimes reduced deforestation on participating properties by 2-23%. The NSMD governance regimes we studied included collaborative and confrontational strategies between environmental and industry stakeholders. We find that the more collaborative governance systems studied achieved better environmental performance than more confrontational approaches. Whereas many government conservation programs have targeted regions with little likelihood of conversion, we demonstrate that NSMD governance has the potential to alter behavior on high-deforestation properties.

Propensity score matching with clustered data. An application to the estimation of the impact of caesarean section on the Apgar score

This article focuses on the implementation of propensity score matching for clustered data. Different approaches to reduce bias due to cluster-level confounders are considered and compared using Monte Carlo simulations. We investigated methods that exploit the clustered structure of the data in two ways: in the estimation of the propensity score model (through the inclusion of fixed or random effects) or in the implementation of the matching algorithm. In addition to a pure within-cluster matching, we also assessed the performance of a new approach, 'preferential' within-cluster matching. This approach first searches for control units to be matched to treated units within the same cluster. If matching is not possible within-cluster, then the algorithm searches in other clusters. All considered approaches successfully reduced the bias due to the omission of a cluster-level confounder. The preferential within-cluster matching approach, combining the advantages of within-cluster and between-cluster matching, showed a relatively good performance both in the presence of big and small clusters, and it was often the best method. An important advantage of this approach is that it reduces the number of unmatched units as compared with a pure within-cluster matching. We applied these methods to the estimation of the effect of caesarean section on the Apgar score using birth register data. Copyright © 2016 John Wiley & Sons, Ltd.